jamesturk/scrapeghost

Health v2 · maintenance only

Flags: no_license

Adoption not part of the score

1442 stars · 86 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

scrapeghost is an experimental Python library that uses OpenAI's GPT models to scrape structured data from websites without writing page-specific code. It supports Python-based schema definitions, HTML preprocessing, JSON/schema validation, hallucination checks, and cost controls, but is explicitly no longer maintained.

Use cases

• extract structured data from web pages using GPT
• scrape HTML into JSON without writing page-specific parsers
• define a schema in Python and let an LLM fill it from a URL
• validate LLM-extracted data against a pydantic schema
• check that scraped values actually appear on the page
• track and cap token costs when scraping with GPT

When to choose

• you want to experiment with LLM-based web scraping in Python
• you need one-off structured extraction from pages where writing selectors is impractical
• you want built-in cost tracking, budget limits, and hallucination checks for GPT scraping

When to avoid

• you need a maintained, production-ready scraping solution (the project is explicitly unmaintained)
• you want to avoid commercial LLM API costs and latency
• you need deterministic, guaranteed-accurate extraction
• your use case requires a permissive license (Hippocratic License 3.0)

Facets

library · maturity abandoned

web-scraping llm-inference parser nlp crawlers developer-tools python gpt openai html-to- schema-extraction experimental natural-language-processing

Member repositories

For agents

markdown · JSON · MCP: product_card(name="jamesturk/scrapeghost")