AIAny
AI Others2024
Icon for item

Scrapling

Python web scraping framework that automatically relocates elements when a site's HTML changes, so selectors survive redesigns. Bundles Cloudflare Turnstile bypass, TLS fingerprint impersonation, and a Scrapy-like async spider for full crawls.

Introduction

Most scrapers break the moment a site ships a redesign — a renamed class or a reordered div, and your carefully tuned selectors return nothing. Scrapling's bet is that the element you scraped yesterday usually still exists today, just somewhere slightly different, and a framework can find it for you instead of making you rewrite XPath at 2am.

What Sets It Apart
  • Adaptive element relocation: when a selector stops matching, it fingerprints the original element (text, attributes, structural neighbors) and uses similarity scoring to find its new home — so scrapers self-heal across layout changes rather than failing silently.
  • Speed where it counts: the parser clocks roughly 2ms for text extraction, neck-and-neck with Parsel and hundreds of times faster than BeautifulSoup on the same benchmark, so the adaptive layer isn't paid for with a performance tax.
  • Anti-bot built in, not bolted on: Cloudflare Turnstile bypass, TLS fingerprint impersonation, and Playwright-backed stealth sessions ship together, so you rarely reach for a separate stealth stack.
  • A real spider, not just a parser: a Scrapy-like async crawler with per-domain throttling, checkpoint-based pause/resume, and streaming extraction handles full crawls, not just one-off page grabs.
Who It's For

Great fit if you maintain long-lived scrapers against sites you don't control and you're tired of selectors rotting on every redesign, or if you want stealth and crawling in one dependency instead of stitching together requests, a parser, and an anti-bot layer. Look elsewhere if your targets are stable APIs or static pages where a plain HTTP client plus Parsel is simpler — the adaptive machinery is overhead you won't use. It's also a younger project than Scrapy, so the ecosystem of middlewares and tutorials is thinner.

Information

  • Websitegithub.com
  • OrganizationsKarim Shoair
  • AuthorsKarim Shoair (D4Vinci)
  • Published date2024/10/13

Categories

More Items

GitHub
AI Others2026

Self-hosted personal finance manager that keeps all account data on your server — supports multi-currency accounts, bank sync, rules-based auto-categorization, budgets and asset tracking. Distinguishes itself by privacy-first design and optional self-hosted LLM agents for local data queries; suitable for users comfortable running a server.

GitHub
AI Others2020

Aggregates Summer 2026 internship postings across software engineering, data science/AI, quant, product and hardware; curated and updated daily by the Pitt Computer Science Club and Simplify. Lists 250+ active roles with categories, direct application links, and age-since-posting to help applicants find timely openings.

GitHub
AI Others2023

Offline, privacy-first English grammar checker implemented in Rust that lints documents in milliseconds and uses a fraction of LanguageTool's memory. Can run as a local binary, embed via WebAssembly, and offers integrations for editors and plugins.