D4Vinci/Scrapling
🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ and follow here for daily tips and tricks: https://x.com/Scrapling_dev observed · 2026-10-11
Health v2 · maintenance only
87/100
- Activity 99
- Release rhythm 92
- Longevity 52
How is this computed?
round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10) — computed 2026-10-21. Adoption (stars, forks) is never an input.
- gap_med: 8
- age_days: 737
- days_rel: 58
- days_push: 10
- n_releases_24m: 50
Adoption not part of the score
86704 stars · 8882 forks observed · 2026-10-11
What it is AI-extracted, prompt v1, taxonomy v1, 2026-09-21, confidence 0.97
Scrapling is a high‑performance Python library for web scraping and crawling that adapts to site changes automatically. It includes stealth fetchers, dynamic rendering support, and a full‑featured spider framework for scalable crawls.
Use cases
- scrape product listings from e‑commerce sites that use JavaScript rendering
- extract news articles from sites with frequent layout changes
- perform large‑scale crawls with automatic proxy rotation and rate‑limiting
- collect data from websites protected by Cloudflare Turnstile or other anti‑bot measures
- build an AI‑driven data collection pipeline that adapts to page structure changes
When to choose
- You need a Python‑based scraper that can handle dynamic, JavaScript‑heavy pages out of the box.
- You want adaptive selectors that automatically adjust when a website’s DOM changes.
- You require built‑in stealth features to bypass anti‑bot protections like Cloudflare.
- You plan to run concurrent, multi‑session crawls with pause/resume and automatic proxy management.
When to avoid
- You need a lightweight, single‑request scraper with minimal dependencies.
- Your project is not based on Python or cannot run Playwright/Chromium.
- You prefer a purely headless HTTP client without browser automation.
Facets
library · maturity active
web-scraping http-client web-development python cli adaptive-selectors stealth-mode concurrent-crawling proxy-rotation ai-agent-skill playwright-integration data-engineering docker
3 sources
- readme: https://github.com/D4Vinci/Scrapling · fetched 2026-10-11 · 34559550ac58
- homepage: https://scrapling.readthedocs.io/en/latest/ · fetched 2026-09-07 · 5a36bf675b8c
- registry_pypi: https://pypi.org/pypi/scrapling/json · fetched 2026-09-07 · 76311fe1a6d0
Member repositories
| Repository | Role | Health v2 |
|---|---|---|
| D4Vinci/Scrapling | main | 87 |
For agents
markdown · JSON · MCP: product_card(name="D4Vinci/Scrapling")
Data as of 2026-10-19T08:05:51.212177+00:00 · Report a problem