# jamesturk/scrapeghost

👻 Experimental library for scraping websites using OpenAI's GPT API.

Repository: https://github.com/jamesturk/scrapeghost
Canonical: https://ross.abutalabs.com/products/scrapeghost
Homepage: https://jamesturk.github.io/scrapeghost/
Language: Python
License: NOASSERTION
License Family: other
Topics: gpt, webscraping, openai-api
Last push: 2026-01-14T04:01:42+00:00

## Health v2 (maintenance only)
Score: 49/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 62, release rhythm 8, longevity 90
- inputs: {"age_days": 1265, "days_push": 231, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: no_license
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 1442, forks 86 (observed 2026-08-28T04:04:44.740030+00:00)

## What it is
scrapeghost is an experimental Python library that uses OpenAI's GPT models to scrape structured data from websites without writing page-specific code. It supports Python-based schema definitions, HTML preprocessing, JSON/schema validation, hallucination checks, and cost controls, but is explicitly no longer maintained.

## Use cases
- extract structured data from web pages using GPT
- scrape HTML into JSON without writing page-specific parsers
- define a schema in Python and let an LLM fill it from a URL
- validate LLM-extracted data against a pydantic schema
- check that scraped values actually appear on the page
- track and cap token costs when scraping with GPT

## When to choose
- you want to experiment with LLM-based web scraping in Python
- you need one-off structured extraction from pages where writing selectors is impractical
- you want built-in cost tracking, budget limits, and hallucination checks for GPT scraping

## When to avoid
- you need a maintained, production-ready scraping solution (the project is explicitly unmaintained)
- you want to avoid commercial LLM API costs and latency
- you need deterministic, guaranteed-accurate extraction
- your use case requires a permissive license (Hippocratic License 3.0)

## Facets
- artifact type: library
- maturity: abandoned
- function: web-scraping, llm-inference, parser, nlp
- domain: crawlers, developer-tools
- platform: python
- tags: gpt, openai, html-to-, schema-extraction, experimental, natural-language-processing

## Member repositories
- jamesturk/scrapeghost (main) score 49

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:04:44.740030+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T04:36:24.448750+00:00, confidence not recorded.
  - readme: https://github.com/jamesturk/scrapeghost (fetched 2026-08-28T04:04:44.740030+00:00, sha 29d49c14c72a)
  - homepage: https://jamesturk.github.io/scrapeghost/ (fetched 2026-08-29T11:46:32.435461+00:00, sha 449cf648d211)
- Data as of 2026-08-30T08:39:29.467469+00:00.