coleam00/mcp-crawl4ai-rag

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

2245 stars · 576 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

An MCP server that combines Crawl4AI web crawling with RAG capabilities backed by a Supabase vector database, exposing tools for AI agents and coding assistants to scrape websites and query the crawled knowledge. It supports advanced RAG strategies including contextual embeddings, hybrid search, agentic RAG, reranking, and knowledge-graph-based hallucination detection.

Use cases

• crawl documentation sites and query them via RAG from an AI coding assistant
• give AI agents the ability to scrape web pages and store knowledge in a vector database
• extract code examples from crawled repositories for agentic RAG
• detect AI hallucinations using a knowledge graph of crawled content
• set up an MCP server for web crawling and retrieval-augmented generation
• build a knowledge engine for coding assistants like Cursor or Windsurf

When to choose

• you want an MCP server that crawls websites and serves the content over RAG to AI agents or coding assistants
• you need advanced retrieval strategies like contextual embeddings, hybrid search, or reranking
• you already use Supabase and want a vector-backed crawling pipeline
• you are building on or integrating with the Archon ecosystem

When to avoid

• you need a fully configurable, production-hardened solution with support for local embedding models like Ollama
• you want a general-purpose web scraper without RAG or vector storage
• you require active maintenance and fast issue resolution, as the author describes it as a testbed
• you prefer a managed service rather than self-hosting Python services with Supabase and OpenAI dependencies

Facets

service · maturity active

rag web-scraping vector-database mcp search-engine llm-inference artificial-intelligence large-language-models developer-tools python self-hosted mcp-server crawl4ai supabase contextual-embeddings hybrid-search reranking knowledge-graph ai-coding-assistants retrieval-augmented-generation ai-agents docker web-server

Member repositories

For agents

markdown · JSON · MCP: product_card(name="coleam00/mcp-crawl4ai-rag")