AIAny
AI Others2024
Icon for item

Scrapling

Python web scraping framework that automatically relocates elements when a site's HTML changes, so selectors survive redesigns. Bundles Cloudflare Turnstile bypass, TLS fingerprint impersonation, and a Scrapy-like async spider for full crawls.

Introduction

Most scrapers break the moment a site ships a redesign — a renamed class or a reordered div, and your carefully tuned selectors return nothing. Scrapling's bet is that the element you scraped yesterday usually still exists today, just somewhere slightly different, and a framework can find it for you instead of making you rewrite XPath at 2am.

What Sets It Apart
  • Adaptive element relocation: when a selector stops matching, it fingerprints the original element (text, attributes, structural neighbors) and uses similarity scoring to find its new home — so scrapers self-heal across layout changes rather than failing silently.
  • Speed where it counts: the parser clocks roughly 2ms for text extraction, neck-and-neck with Parsel and hundreds of times faster than BeautifulSoup on the same benchmark, so the adaptive layer isn't paid for with a performance tax.
  • Anti-bot built in, not bolted on: Cloudflare Turnstile bypass, TLS fingerprint impersonation, and Playwright-backed stealth sessions ship together, so you rarely reach for a separate stealth stack.
  • A real spider, not just a parser: a Scrapy-like async crawler with per-domain throttling, checkpoint-based pause/resume, and streaming extraction handles full crawls, not just one-off page grabs.
Who It's For

Great fit if you maintain long-lived scrapers against sites you don't control and you're tired of selectors rotting on every redesign, or if you want stealth and crawling in one dependency instead of stitching together requests, a parser, and an anti-bot layer. Look elsewhere if your targets are stable APIs or static pages where a plain HTTP client plus Parsel is simpler — the adaptive machinery is overhead you won't use. It's also a younger project than Scrapy, so the ecosystem of middlewares and tutorials is thinner.

Information

  • Websitegithub.com
  • OrganizationsKarim Shoair
  • AuthorsKarim Shoair (D4Vinci)
  • Published date2024/10/13

Categories

More Items

GitHub
AI Others2014

Provides a complete, university-level computer science curriculum assembled from free online courses and books. Curates degree-aligned course sequences (Intro / Core / Advanced) with community support, project guidance, and checklists to track progress for self-directed learners.

GitHub
AI Others2025

Curated collection of 70 hands‑on cybersecurity projects, certification roadmaps and learning resources organized into Foundations/Beginner/Intermediate/Advanced tiers. Each project ships source code plus deep learn/ documentation; several focus on AI security (LLM prompt defenses, ML threat detection).

GitHub
AI Others2026

Provides a customizable React-based design system and component library designed for people and AI assistants to build together. Ships 150+ accessible components, a theme system, and a CLI; supports swizzling to eject source and className overrides so projects avoid styling lock-in.