Admin access

Enter the admin PIN to view scraper health.

Scraper Health ยท workers ยท pipeline ยท last 30 days

๐Ÿฉบ Live health โ€”

Auto-refreshes every 15s. Pauses when this tab is hidden.

๐Ÿšฆ Pipeline diagnostics โ€”

Heuristic checks. Each row gives a verdict + remedy if it's not OK.

โฑ Currently running scrapes โ€”

Scrapes flagged "running" in the DB. Rows with red age = likely wedged from a worker crash.
What does "wedged" mean? โ–ธ
A scrape row that's been in status='running' for more than 60 minutes but no Celery worker is actually processing it. Most often caused by the worker getting SIGTERM'd (manual restart, OOM kill, system reboot) after the row was created but before a status update landed. The (niche, product_type) combo is otherwise idempotent โ€” the scheduler will re-attempt it on its next tick, so the safe fix is to mark these rows as failed. Click "Mark failed" on any row, or use the ๐Ÿ”ด "Resolve all wedged" button in Pipeline diagnostics above to bulk-clear them.

๐Ÿ“… Last 30 days โ€”

Per-day rollup of scrape attempts + ideas created. "Got stuck" fires when โ‰ฅ50 combos completed but <30 ideas landed (pipeline ran but didn't produce).

โŒ Recent failures (last 7 days) โ€”

Failed or timed-out scrape combos. Error messages truncated to first line โ€” full details in scrape_criteria_log.error_message.