# D4Vinci/Scrapling

🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl!

Repository: https://github.com/D4Vinci/Scrapling
Canonical: https://ross.abutalabs.com/products/scrapling
Homepage: https://scrapling.readthedocs.io/en/latest/
Language: Python
License: BSD-3-Clause
License Family: permissive
Topics: crawler, crawling, crawling-python, playwright, python, scraping, selectors, stealth, web-scraper, web-scraping, web-scraping-python, webscraping, xpath, automation, ai, ai-scraping, data, data-extraction, mcp, mcp-server
Last push: 2026-08-25T18:39:16+00:00

## Health v2 (maintenance only)
Score: 89/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 99, release rhythm 99, longevity 49
- inputs: {"age_days": 689, "days_push": 8, "days_rel": 10, "gap_med": 7.5, "n_releases_24m": 53}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 76660, forks 7663 (observed 2026-08-28T04:12:21.610332+00:00)

## What it is
Scrapling is an adaptive Python web scraping framework that handles everything from single requests to full-scale concurrent crawls. It features self-healing selectors that relocate elements when page structures change, stealth fetchers that bypass anti-bot systems like Cloudflare, and a Scrapy-like spider API with proxy rotation and adaptive throttling.

## Use cases
- scrape product data from e-commerce sites
- crawl multiple pages concurrently with proxy rotation
- bypass Cloudflare and anti-bot protection when fetching pages
- extract data that survives website design changes
- scrape dynamic JavaScript-rendered websites
- build a spider to crawl an entire site
- expose web scraping to AI agents via MCP

## When to choose
- you need scraping that adapts when website HTML structure changes
- target sites use anti-bot systems like Cloudflare Turnstile
- you want a Scrapy-like crawling API with modern stealth fetching in one library
- you need both fast HTTP fetching and browser-based dynamic rendering

## When to avoid
- you only need simple static HTTP requests without parsing
- you need a distributed scraping cluster at massive scale
- your project is in a language other than Python
- you need strict legal compliance tooling around scraping permissions

## Facets
- artifact type: framework
- maturity: active
- function: web-scraping, parser, http-client, workflow-automation, mcp
- domain: crawlers, web-development, developer-tools
- platform: python, cross-platform
- tags: adaptive-scraping, anti-bot-bypass, stealth-fetching, spider-framework, playwright, xpath-selectors, data-extraction, automation, docker

## Member repositories
- D4Vinci/Scrapling (main) score 89

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:12:21.610332+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-29T16:14:20.759756+00:00, confidence not recorded.
  - readme: https://github.com/D4Vinci/Scrapling (fetched 2026-08-28T04:12:21.610332+00:00, sha 34559550ac58)
  - homepage: https://scrapling.readthedocs.io/en/latest/ (fetched 2026-08-28T17:43:58.477280+00:00, sha 31c9185255aa)
  - registry_pypi: https://pypi.org/pypi/scrapling/json (fetched 2026-08-28T17:43:58.486324+00:00, sha 76311fe1a6d0)
- Data as of 2026-08-30T08:39:29.467469+00:00.