What we won't do
- No CAPTCHA / bot-protection bypass. Targets that refuse the request return
TARGET_BLOCKED— this is a hard product boundary, not a missing feature. - Respect
robots.txtand site terms. LeanCrawl is for data you are allowed to fetch; abusive usage gets accounts suspended.
Common questions
Why is my crawl partial? It ran out of credits. Completed pages are billed
and returned; top up and re-run with a smaller limit or narrower
include_paths.
Why did my screenshot/PDF come back as a URL? Artifact endpoints return a
Bearer-authenticated URL (retained for a limited window). Fetch it with the same
Authorization header. In MCP, artifacts are embedded inline for you.
Do cached pages cost less? No — a cached hit costs the same as a fresh
capture. The 48 h cache buys latency, not discounts.
Where did my run go? GET /v1/account/activity lists every billed attempt
with endpoint, status, error_code, duration, and credit cost. Content is
retained ~90 days.
Rate limited? You get 429 + Retry-After. Honor it; for heavy work use
crawl instead of parallel capture calls.