Feeding live web pages to an LLM sounds trivial until you hit the real wall: JavaScript-rendered content, bot defenses, and pages that bury 200 tokens of signal under 20,000 tokens of nav and ads. The bet here is that scraping is plumbing developers shouldn't keep rebuilding — return LLM-ready markdown or typed JSON from one call. The team arrived at this from Mendable, where they kept rewriting the same brittle ingestion layer for every customer.
What Sets It Apart
- Output is shaped for tokens, not browsers: clean markdown or schema-defined JSON instead of raw HTML, so context is spent on content rather than markup.
- One API spans the whole pipeline — scrape a URL, crawl an entire site, map all its links, or batch many URLs — instead of gluing together a headless browser, a proxy pool, and a queue.
- Coverage is claimed at roughly 96% of the web including JS-heavy pages, with rotating proxies and rate-limit handling done server-side, so reliability becomes the vendor's problem rather than yours.
- Natural-language extract and agent modes pull structured fields by description, sidestepping per-site CSS-selector maintenance that breaks on every redesign.
Great Fit / Look Elsewhere
Great fit if you're building RAG pipelines, agents, or crawlers and want web data as a managed dependency — especially across many heterogeneous sites where keeping your own scraper alive is a constant tax. Look elsewhere if you only hit a handful of stable, static pages (a few lines of requests + BeautifulSoup is cheaper), if AGPL-3.0 on the self-hosted core clashes with your licensing, or if you need low-level control over the browser that an opinionated API deliberately abstracts away.
