ScrapeGraphAI/Scrapegraph-ai
Python scraper based on AI observed · 2026-08-28
Health v2 · maintenance only
88/100
- Activity 99
- Release rhythm 87
- Longevity 67
How is this computed?
round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10) — computed 2026-09-03. Adoption (stars, forks) is never an input.
- gap_med: 3
- age_days: 949
- days_rel: 10
- days_push: 10
- n_releases_24m: 104
Adoption not part of the score
29959 stars · 2975 forks observed · 2026-08-28
What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded
ScrapeGraphAI is a Python library that uses LLMs and direct graph logic to build web scraping pipelines for websites and local documents (XML, HTML, JSON, Markdown). Users specify what information to extract in natural language and the library generates the scraping flow. It also offers a hosted API with SDKs and integrations for LangChain, LlamaIndex, CrewAI, Agno, and Vercel AI SDK.
Use cases
- scrape structured data from websites using natural language prompts
- extract data from HTML, XML, JSON, and markdown documents with LLMs
- build AI-powered web crawling pipelines in Python
- feed structured web data into AI agents and RAG pipelines
- convert web pages to clean markdown for LLM consumption
- monitor web pages for changes on a schedule
- create datasets by crawling and extracting from multiple pages
When to choose
- you want LLM-driven scraping without writing CSS selectors or XPath by hand
- you need to extract structured data from pages with changing or unknown layouts
- you want scraping tools integrated into LangChain, LlamaIndex, CrewAI, or Vercel AI SDK agents
- you need to scrape local documents as well as live websites
- you prefer a Python library with an MIT license and active development
When to avoid
- you need high-volume scraping with zero LLM API costs — token costs add up quickly
- you need deterministic, fully predictable extraction where hallucination is unacceptable
- you require strict rate-limit compliance or lightweight scraping without heavy dependencies
- you want a fully self-contained open-source solution — the v2 hosted API requires an API key and paid plans for scale
- your target sites block automated access and you need advanced anti-bot bypassing
Facets
library · maturity active
web-scraping llm-inference rag data-generation parser crawlers large-language-models artificial-intelligence developer-tools python cross-platform ai-scraping llm-pipelines graph-logic data-extraction firecrawl-alternative langchain-integration web-crawler data-engineering nodejs
10 sources
- readme: https://github.com/ScrapeGraphAI/Scrapegraph-ai · fetched 2026-08-28 · 1ec968dae730
- homepage: https://scrapegraphai.com · fetched 2026-08-29 · 73440dd0807b
- site_page: https://docs.scrapegraphai.com/integrations/vercel_ai · fetched 2026-08-29 · 5ffc330dedc4
- site_page: https://docs.scrapegraphai.com/introduction · fetched 2026-08-29 · 53cdead6d8a7
- site_page: https://docs.scrapegraphai.com/ · fetched 2026-08-29 · 7228290eb6ba
- site_page: https://docs.scrapegraphai.com/integrations/langchain · fetched 2026-08-29 · b4a5fbc2ebe0
- site_page: https://docs.scrapegraphai.com/integrations/crewai · fetched 2026-08-29 · 0adb66ed5c47
- site_page: https://docs.scrapegraphai.com/integrations/llamaindex · fetched 2026-08-29 · c5ca9e58eeaa
- site_page: https://docs.scrapegraphai.com/integrations/agno · fetched 2026-08-29 · d3e58f6292aa
- site_page: https://scrapegraphai.com/pricing · fetched 2026-08-29 · 91b5f30dd3bb
Member repositories
| Repository | Role | Health v2 |
|---|---|---|
| ScrapeGraphAI/Scrapegraph-ai | main | 88 |
For agents
markdown · JSON · MCP: product_card(name="ScrapeGraphAI/Scrapegraph-ai")
Data as of 2026-08-30T08:39:29.467469+00:00 · Report a problem