# coleam00/mcp-crawl4ai-rag

Web Crawling and RAG Capabilities for AI Agents and AI Coding Assistants

Repository: https://github.com/coleam00/mcp-crawl4ai-rag
Canonical: https://ross.abutalabs.com/products/mcp-crawl4ai-rag
Language: Python
License: MIT
License Family: permissive
Last push: 2025-07-25T15:17:13+00:00

## Health v2 (maintenance only)
Score: 34/100 (v2, computed 2026-09-02T17:46:02.011165+00:00)
- activity 33, release rhythm 35, longevity 34
- inputs: {"age_days": 487, "days_push": 404, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: no_releases
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 2245, forks 576 (observed 2026-08-28T04:06:30.111074+00:00)

## What it is
An MCP server that combines Crawl4AI web crawling with RAG capabilities backed by a Supabase vector database, exposing tools for AI agents and coding assistants to scrape websites and query the crawled knowledge. It supports advanced RAG strategies including contextual embeddings, hybrid search, agentic RAG, reranking, and knowledge-graph-based hallucination detection.

## Use cases
- crawl documentation sites and query them via RAG from an AI coding assistant
- give AI agents the ability to scrape web pages and store knowledge in a vector database
- extract code examples from crawled repositories for agentic RAG
- detect AI hallucinations using a knowledge graph of crawled content
- set up an MCP server for web crawling and retrieval-augmented generation
- build a knowledge engine for coding assistants like Cursor or Windsurf

## When to choose
- you want an MCP server that crawls websites and serves the content over RAG to AI agents or coding assistants
- you need advanced retrieval strategies like contextual embeddings, hybrid search, or reranking
- you already use Supabase and want a vector-backed crawling pipeline
- you are building on or integrating with the Archon ecosystem

## When to avoid
- you need a fully configurable, production-hardened solution with support for local embedding models like Ollama
- you want a general-purpose web scraper without RAG or vector storage
- you require active maintenance and fast issue resolution, as the author describes it as a testbed
- you prefer a managed service rather than self-hosting Python services with Supabase and OpenAI dependencies

## Facets
- artifact type: service
- maturity: active
- function: rag, web-scraping, vector-database, mcp, search-engine, llm-inference
- domain: artificial-intelligence, large-language-models, developer-tools
- platform: python, self-hosted
- tags: mcp-server, crawl4ai, supabase, contextual-embeddings, hybrid-search, reranking, knowledge-graph, ai-coding-assistants, retrieval-augmented-generation, ai-agents, docker, web-server

## Member repositories
- coleam00/mcp-crawl4ai-rag (main) score 34

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:06:30.111074+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T02:43:59.987821+00:00, confidence not recorded.
  - readme: https://github.com/coleam00/mcp-crawl4ai-rag (fetched 2026-08-28T04:06:30.111074+00:00, sha f67effbb008d)
- Data as of 2026-08-30T08:39:29.467469+00:00.