# ScrapeGraphAI/Scrapegraph-ai

Python scraper based on AI

Repository: https://github.com/ScrapeGraphAI/Scrapegraph-ai
Canonical: https://ross.abutalabs.com/products/scrapegraph-ai
Homepage: https://scrapegraphai.com
Language: Python
License: MIT
License Family: permissive
Topics: scraping, scraping-python, llm, web-crawler, web-scraping, ai-scraping, crawler, markdown, rag, web-crawlers, ai-crawler, ai-search, large-language-model, web-data-extraction, web-search, web-scraper, data-extraction, web-data, webscraping, firecrawl-alternative
Last push: 2026-08-23T13:18:10+00:00

## Health v2 (maintenance only)
Score: 88/100 (v2, computed 2026-09-03T02:39:23.370411+00:00)
- activity 99, release rhythm 87, longevity 67
- inputs: {"age_days": 949, "days_push": 10, "days_rel": 10, "gap_med": 3, "n_releases_24m": 104}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 29959, forks 2975 (observed 2026-08-28T04:11:53.769655+00:00)

## What it is
ScrapeGraphAI is a Python library that uses LLMs and direct graph logic to build web scraping pipelines for websites and local documents (XML, HTML, JSON, Markdown). Users specify what information to extract in natural language and the library generates the scraping flow. It also offers a hosted API with SDKs and integrations for LangChain, LlamaIndex, CrewAI, Agno, and Vercel AI SDK.

## Use cases
- scrape structured data from websites using natural language prompts
- extract data from HTML, XML, JSON, and markdown documents with LLMs
- build AI-powered web crawling pipelines in Python
- feed structured web data into AI agents and RAG pipelines
- convert web pages to clean markdown for LLM consumption
- monitor web pages for changes on a schedule
- create datasets by crawling and extracting from multiple pages

## When to choose
- you want LLM-driven scraping without writing CSS selectors or XPath by hand
- you need to extract structured data from pages with changing or unknown layouts
- you want scraping tools integrated into LangChain, LlamaIndex, CrewAI, or Vercel AI SDK agents
- you need to scrape local documents as well as live websites
- you prefer a Python library with an MIT license and active development

## When to avoid
- you need high-volume scraping with zero LLM API costs — token costs add up quickly
- you need deterministic, fully predictable extraction where hallucination is unacceptable
- you require strict rate-limit compliance or lightweight scraping without heavy dependencies
- you want a fully self-contained open-source solution — the v2 hosted API requires an API key and paid plans for scale
- your target sites block automated access and you need advanced anti-bot bypassing

## Facets
- artifact type: library
- maturity: active
- function: web-scraping, llm-inference, rag, data-generation, parser
- domain: crawlers, large-language-models, artificial-intelligence, developer-tools
- platform: python, cross-platform
- tags: ai-scraping, llm-pipelines, graph-logic, data-extraction, firecrawl-alternative, langchain-integration, web-crawler, data-engineering, nodejs

## Member repositories
- ScrapeGraphAI/Scrapegraph-ai (main) score 88

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:11:53.769655+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-29T16:52:48.745855+00:00, confidence not recorded.
  - readme: https://github.com/ScrapeGraphAI/Scrapegraph-ai (fetched 2026-08-28T04:11:53.769655+00:00, sha 1ec968dae730)
  - homepage: https://scrapegraphai.com (fetched 2026-08-29T07:49:59.838079+00:00, sha 73440dd0807b)
  - site_page: https://docs.scrapegraphai.com/integrations/vercel_ai (fetched 2026-08-29T07:49:59.862843+00:00, sha 5ffc330dedc4)
  - site_page: https://docs.scrapegraphai.com/introduction (fetched 2026-08-29T07:49:59.852173+00:00, sha 53cdead6d8a7)
  - site_page: https://docs.scrapegraphai.com/ (fetched 2026-08-29T07:49:59.853833+00:00, sha 7228290eb6ba)
  - site_page: https://docs.scrapegraphai.com/integrations/langchain (fetched 2026-08-29T07:49:59.855479+00:00, sha b4a5fbc2ebe0)
  - site_page: https://docs.scrapegraphai.com/integrations/crewai (fetched 2026-08-29T07:49:59.857335+00:00, sha 0adb66ed5c47)
  - site_page: https://docs.scrapegraphai.com/integrations/llamaindex (fetched 2026-08-29T07:49:59.859268+00:00, sha c5ca9e58eeaa)
  - site_page: https://docs.scrapegraphai.com/integrations/agno (fetched 2026-08-29T07:49:59.861249+00:00, sha d3e58f6292aa)
  - site_page: https://scrapegraphai.com/pricing (fetched 2026-08-29T07:49:59.850353+00:00, sha 91b5f30dd3bb)
- Data as of 2026-08-30T08:39:29.467469+00:00.