shopping-hub

shopping-hub

Docker app from allornothing's Repository

Cross-marketplace product aggregation and price comparison.

Requirements

Internet access. A residential proxy is recommended for sustained crawling.
Overview

Cross-marketplace product aggregation and price comparison.

Crawls AliExpress, Alibaba, DHgate, Banggood, Made-in-China, Geekbuying, Chinavasion
and eBay, works out which listings are the same physical product, and publishes one
item page per product showing every site's price, minimum order quantity and
disclosed shipping - translated to English, with forwarding-agent links for the sites
that will not ship to the US.

Products are matched on five signals - perceptual image hash, normalised title,
brand, manufacturer part number and extracted specs - rather than titles alone, which
are close to useless on these sites.

Comparison is done on landed cost (unit price x minimum order quantity + freight +
duty), because the cheapest unit price often belongs to the most expensive order. A
US retail baseline from eBay is included, so the tool can tell you when importing is
not actually cheaper.

Also included: reverse image search, bulk/BOM costing, break-even analysis, price and
restock alerts, supplier trust scoring, category browsing and full-text search.

TWO SETTINGS THAT MATTER:

  1. Set SOURCEHUB_ADMIN_TOKEN below. The admin pages are disabled without it, by
    design - container traffic never looks like loopback to the app, so leaving it unset
    would otherwise expose an unauthenticated admin panel to your whole LAN. Sign in with
    the token as the password (any username). Browsing, search and crawling work either
    way.

  2. Leave Extra Parameters set to --shm-size=1g. Docker's default 64MB /dev/shm makes
    Chromium crash on heavy pages, and it fails silently - the browser-rendered sites just
    return zero listings.

Everything lives under the single /config mount: database, images, browser profile and
the editable YAML config. Back that up and you have backed up the app.

A slim image (allornothing/shopping-hub:slim, ~0.31GB vs ~1.17GB) is available if you
don't need the six JavaScript-rendered adapters or the eBay/break-even baseline --
change the Repository field above to use it.

Shopping Hub

Shopping Hub compares the same product across marketplaces like AliExpress, Alibaba, 1688, DHgate, LCSC, eBay and Best Buy. Each product gets one page with every site's price, minimum order quantity and shipping, translated to English, plus a US retail price so you can tell whether importing is actually cheaper.

What it does well:

  • Matches products across sites, so you see one item instead of fifteen listings.
  • Shows the real cost, including minimums, fees and a US baseline.
  • Handles forwarding agents for sites that won't ship to the US.

Image: allornothing/shopping-hub · Source: aon082910/shopping-hub

Install

Unraid: search shopping-hub in Apps, set an admin token, and keep --shm-size=1g in Extra Parameters. The full walkthrough is in UNRAID.md.

Docker:

docker run -d --name shopping-hub -p 8000:8000 --shm-size=1g \
  -v ./appdata:/config \
  -e SOURCEHUB_ADMIN_TOKEN=change-me \
  allornothing/shopping-hub:latest

Open http://127.0.0.1:8000. There's a docker-compose.yml in the repo too.

Setting Default What it's for
Port 8000 Web UI
/config your appdata folder Database and settings
SOURCEHUB_ADMIN_TOKEN none Admin access
SOURCEHUB_PROXY none Optional residential proxy for heavy crawling

Use the slim tag if you don't want the bundled browser (about 0.3 GB instead of 1.2 GB). It skips eBay, Banggood, 1688, Taobao and Tmall.

Read before crawling

Taobao, Tmall and 1688 need a logged-in session, and most marketplaces rate-limit scrapers and may forbid it in their terms. The defaults are deliberately slow. Check each site's terms and crawl politely.

Help

Open an issue. MIT licensed.

Media gallery

1 / 4

Categories

Download Statistics

1,307
Total Downloads

Related apps

Links

Details

Repository
allornothing/shopping-hub:latest
Last Updated2026-09-08
First Seen2026-09-03

Runtime arguments

Web UI
http://[IP]:[PORT:8000]/
Network
bridge
Shell
bash
Privileged
false
Extra Params
--shm-size=1g

Template configuration

WebUI PortPorttcp

Port for the web interface.

Target
8000
Default
8000
Value
8000
AppdataPathrw

Database, downloaded images, browser login profile and the editable YAML config all live here. Back this up.

Target
/config
Default
/mnt/user/appdata/shopping-hub
Value
/mnt/user/appdata/shopping-hub
Admin TokenVariable

REQUIRED to use the admin pages in a container. Any strong random string. Sign in with it as the password (any username). Leave blank and admin returns 403 - browsing and search still work.

Target
SOURCEHUB_ADMIN_TOKEN
ModeVariable

serve = web UI plus scheduled crawling (recommended). web = UI only, no crawling. schedule = crawler only, no UI.

Target
SOURCEHUB_MODE
Default
serve
Value
serve
TZVariable

Timezone. The crawl schedule in config.yaml is interpreted in this zone.

Default
America/New_York
Value
America/New_York
Translation ProviderVariable

claude, deepl, google_free or none. Without translation, Chinese listings cannot be title-matched and mostly stay on their own product pages.

Target
TRANSLATE_PROVIDER
Default
none
Value
none
Anthropic API KeyVariable

Used when Translation Provider is set to claude. Translations are cached by content hash, so a repeat crawl costs nothing.

Target
ANTHROPIC_API_KEY
PUIDVariable

User ID. 99 (nobody) is the Unraid default and keeps appdata editable from the host.

Default
99
Value
99
PGIDVariable

Group ID. 100 (users) is the Unraid default.

Default
100
Value
100
eBay Client IDVariable

Free from developer.ebay.com. Without it eBay falls back to browser scraping, which works but is slower.

Target
EBAY_CLIENT_ID
eBay Client SecretVariable

Free from developer.ebay.com.

Target
EBAY_CLIENT_SECRET
AliExpress App KeyVariable

From an AliExpress Affiliate app at portals.aliexpress.com (join the affiliate program, then create an app under its API section - the Dropshipping API is a different product and will not work here). Without this, AliExpress falls back to scraping its public search page. With it, the app also gets full category-tree crawling instead of only keyword search.

Target
ALIEXPRESS_APP_KEY
AliExpress App SecretVariable

From the same AliExpress Affiliate app as App Key.

Target
ALIEXPRESS_APP_SECRET
AliExpress Tracking IDVariable

Also from the Affiliate portal - required by the API on every call to attribute commission. Any tracking ID you create there works.

Target
ALIEXPRESS_TRACKING_ID
ProxyVariable

Optional. One http://user:pass@host:port, or several comma-separated for round-robin. Strongly recommended for sustained crawling.

Target
SOURCEHUB_PROXY
CN Provider KeyVariable

Optional API key for Taobao/Tmall/1688 data. Without it those three need a one-time browser login, which is awkward in a container.

Target
CN_PROVIDER_KEY
Trust Proxy HeadersVariable

Set true ONLY if SWAG, Nginx Proxy Manager, Traefik or similar sits in front of this container. Searching for something uncrawled starts a background crawl, and that is rate limited per caller by source address -- behind a proxy every request looks like it came from the proxy, so one visitor's limit is shared by everyone. Turning this on reads X-Forwarded-For instead. Leave it false when the port is exposed directly: the header is otherwise whatever the client typed, and trusting it lets one caller bypass the limit entirely.

Target
SOURCEHUB_TRUST_PROXY
Default
false