0xMassi/webclaw
Fast, local-first web content extraction for LLMs. Scrape, crawl, extract structured data — all from Rust. CLI, REST API, and MCP server. observed · 2026-08-28
Health v2 · maintenance only
77/100
- Activity 99
- Release rhythm 86
- Longevity 12
Flags: young
How is this computed?
round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10) — computed 2026-09-03. Adoption (stars, forks) is never an input.
- gap_med: 0.0
- age_days: 176
- days_rel: 17
- days_push: 7
- n_releases_24m: 61
Adoption not part of the score
2305 stars · 232 forks observed · 2026-08-28
What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded
webclaw is a Rust-based web extraction toolkit that turns any URL into clean, LLM-ready markdown, JSON, or token-optimized text, including bot-protected pages. It ships as three binaries sharing one engine: a CLI, an axum-based REST API server, and an MCP server with 14 tools for AI clients like Claude and Cursor.
Use cases
- scrape a webpage into clean markdown for an LLM prompt
- crawl an entire docs site and export every page as markdown
- convert HTML to LLM-optimized text with 90% fewer tokens
- extract structured JSON from a URL matching a schema
- plug live web access into Claude Desktop or Cursor via MCP
- monitor a webpage for changes with diffs
- self-host a Firecrawl alternative for RAG ingestion
When to choose
- you need LLM-ready web content instead of raw HTML boilerplate
- you want a self-hostable, fast Rust alternative to Firecrawl or Jina
- you need MCP integration so AI agents can read the live web
- you scrape bot-protected sites and need TLS fingerprinting
- you want CLI, REST API, and MCP access from one extraction engine
When to avoid
- you need a fully managed scraping service and don't want to self-host or pay for the cloud API
- you only need simple static HTML parsing in a language other than Rust
- your use case requires heavy JavaScript rendering at scale beyond what the engine handles
- you need a permissive license — the core is AGPL-3.0
Facets
cli-tool · maturity active
web-scraping parser rag mcp http-server cli search-engine web-development crawlers large-language-models developer-tools cli self-hosted cross-platform rust html-to-markdown firecrawl-alternative mcp-server llm-ready tls-fingerprinting rest-api web-crawler bot-protection-bypass ai-agents retrieval-augmented-generation natural-language-processing web-server
10 sources
- readme: https://github.com/0xMassi/webclaw · fetched 2026-08-28 · 9d8d7c7b67c1
- homepage: https://webclaw.io · fetched 2026-08-29 · e09be3f832ff
- site_page: https://webclaw.io/docs · fetched 2026-08-29 · 767286522a9c
- site_page: https://webclaw.io/features · fetched 2026-08-29 · 7351cded2cf6
- site_page: https://webclaw.io/docs/api/scrape · fetched 2026-08-29 · bd6f2c8e0dad
- site_page: https://webclaw.io/docs/api/crawl · fetched 2026-08-29 · f1b722e5d5f9
- site_page: https://webclaw.io/docs/api/search · fetched 2026-08-29 · ee67c220f196
- site_page: https://webclaw.io/pricing · fetched 2026-08-29 · b8641c565109
- site_page: https://webclaw.io/integrations · fetched 2026-08-29 · 1627f0ff9b7f
- site_page: https://webclaw.io/pricing?billing=yearly · fetched 2026-08-29 · b8641c565109
Member repositories
| Repository | Role | Health v2 |
|---|---|---|
| 0xMassi/webclaw | main | 77 |
For agents
Data as of 2026-08-30T08:39:29.467469+00:00 · Report a problem