ScrapeGraphAI
Some data may be outdated — see provenance labels.
Python scraper based on AI observed · 2026-09-28
Health v2 · maintenance only
87/100
- Activity 96
- Release rhythm 84
- Longevity 71
How is this computed?
round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10) — computed 2026-10-21. Adoption (stars, forks) is never an input.
- gap_med: 5.0
- age_days: 997
- days_rel: 25
- days_push: 25
- n_releases_24m: 81
Adoption not part of the score
31358 stars · 3165 forks observed · 2026-09-28
What it is AI-extracted, prompt v1, taxonomy v1, 2026-10-05, confidence 0.97
ScrapeGraphAI is a Python library that leverages large language models to automate web scraping and data extraction. It provides a high‑level SDK and API for fetching pages, extracting structured data, searching, crawling, and monitoring sites.
Use cases
- Extract product details from e‑commerce sites using natural language prompts
- Build a dataset of news articles by crawling and summarizing web pages
- Integrate web data extraction into LangChain or LlamaIndex agents
- Automate periodic monitoring of competitor websites for content changes
- Convert web pages to clean markdown or JSON for downstream AI pipelines
When to choose
- You need AI‑driven scraping that can understand arbitrary page layouts without writing custom parsers
- You want a single Python SDK that works with popular LLM frameworks like LangChain, CrewAI, or Vercel AI SDK
- You require both on‑demand extraction and scheduled crawling/monitoring capabilities
- You prefer a managed API with scalable credit‑based pricing for high‑volume use
When to avoid
- You need low‑level control over HTTP requests, headers, or proxy rotation beyond what the SDK offers
- Your project must run entirely offline without external API calls
- Legal or compliance constraints prohibit using LLM‑based extraction on certain websites
Facets
library · maturity active
sdk http-client web-scraping llm-inference rag web-development data-science artificial-intelligence python cloud self-hosted ai-scraping llm-integration markdown json html search-engine crawler
10 sources
- readme: https://github.com/ScrapeGraphAI/Scrapegraph-ai · fetched 2026-09-28 · 1ec968dae730
- homepage: https://scrapegraphai.com · fetched 2026-09-21 · 9206ef71a25e
- site_page: https://docs.scrapegraphai.com/integrations/langchain · fetched 2026-09-21 · b4a5fbc2ebe0
- site_page: https://docs.scrapegraphai.com/integrations/crewai · fetched 2026-09-21 · 0adb66ed5c47
- site_page: https://docs.scrapegraphai.com/integrations/llamaindex · fetched 2026-09-21 · c5ca9e58eeaa
- site_page: https://docs.scrapegraphai.com/integrations/agno · fetched 2026-09-21 · d3e58f6292aa
- site_page: https://docs.scrapegraphai.com/integrations/vercel_ai · fetched 2026-09-21 · 5ffc330dedc4
- site_page: https://docs.scrapegraphai.com/introduction · fetched 2026-09-21 · 53cdead6d8a7
- site_page: https://docs.scrapegraphai.com/ · fetched 2026-09-21 · 7228290eb6ba
- registry_pypi: https://pypi.org/pypi/toonify/json · fetched 2026-09-21 · d9640121c68c
Member repositories
| Repository | Role | Health v2 |
|---|---|---|
| ScrapeGraphAI/Scrapegraph-ai | main | 87 |
| ScrapeGraphAI/toonify | sdk | 53 |
| ScrapeGraphAI/scrapegraph-mcp | backend | 59 |
| ScrapeGraphAI/scrapegraph-py | sdk | 76 |
| ScrapeGraphAI/just-scrape | frontend | 59 |
For agents
markdown · JSON · MCP: product_card(name="ScrapeGraphAI/Scrapegraph-ai")
Data as of 2026-10-19T08:05:51.212177+00:00 · Report a problem