# gocolly/colly

Elegant Scraper and Crawler Framework for Golang

Repository: https://github.com/gocolly/colly
Canonical: https://ross.abutalabs.com/products/colly
Homepage: https://go-colly.org/
Language: Go
License: Apache-2.0
License Family: permissive
Topics: golang, scraper, framework, crawler, scraping, crawling, spider, go
Last push: 2026-08-14T14:37:32+00:00

## Health v2 (maintenance only)
Score: 66/100 (v2, computed 2026-09-02T17:46:02.011165+00:00)
- activity 97, release rhythm 8, longevity 100
- inputs: {"age_days": 3260, "days_push": 19, "days_rel": 524, "gap_med": null, "n_releases_24m": 1}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 25483, forks 1855 (observed 2026-08-28T04:11:38.171532+00:00)

## What it is
Colly is a fast and elegant web scraping and crawling framework for Go. It provides a clean callback-based API for making HTTP requests, parsing HTML, and extracting structured data, with built-in support for rate limiting, caching, cookies, and distributed scraping.

## Use cases
- scrape product prices from e-commerce sites
- crawl a website and extract all links
- build a spider to collect structured data from web pages
- mine data from websites for analysis
- archive web content automatically
- scrape news headlines or comments from sites like Hacker News or Reddit

## When to choose
- you need a high-performance scraper in Go
- you want built-in rate limiting, concurrency control, and robots.txt compliance
- you need sync, async, or distributed crawling
- you prefer a clean callback-based API for HTML extraction

## When to avoid
- you need to scrape JavaScript-rendered single-page apps (use a headless browser tool instead)
- your project is in Python (Scrapy/BeautifulSoup may fit better)
- you only need a one-off simple HTTP fetch without crawling logic

## Facets
- artifact type: framework
- maturity: stable
- function: web-scraping, http-client, parser
- domain: crawlers, web-development, developer-tools
- platform: go, cross-platform
- tags: scraper, crawler, spider, html-parsing, distributed-scraping, robots-txt, data-engineering

## Member repositories
- gocolly/colly (main) score 66

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:11:38.171532+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-29T16:56:03.085991+00:00, confidence not recorded.
  - readme: https://github.com/gocolly/colly (fetched 2026-08-28T04:11:38.171532+00:00, sha 4aba788dfdfb)
  - homepage: https://go-colly.org/ (fetched 2026-08-29T07:52:48.965513+00:00, sha 9a4dcc826b6e)
  - site_page: https://go-colly.org/docs (fetched 2026-08-29T07:52:49.010594+00:00, sha 4488744c1021)
- Data as of 2026-08-30T08:39:29.467469+00:00.