geekan/scrapy-examples resource

Health v2 · maintenance only

Flags: no_releases no_license

Adoption not part of the score

3253 stars · 1016 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

A collection of example Scrapy spiders demonstrating web scraping patterns for sites like Douban, LinkedIn, Zhihu, Alexa, and GitHub. It includes reusable helpers such as CSS-rule-based parsing and proxy/agent middleware to speed up writing new spiders.

Use cases

• learn scrapy by example
• scrape book listings from douban
• write a spider quickly with css rules
• scrape with rotating proxies and user agents
• bootstrap a new scrapy project
• extract data from linkedin or zhihu profiles

When to choose

• you are learning Scrapy and want working spider examples
• you need a template with proxy and user-agent middleware already wired up
• you want a quick project scaffold for a new scraper

When to avoid

• you need a production scraping framework rather than example code
• you require a maintained project with a license and active support
• you are scraping sites whose terms of service prohibit it

Facets

learning-resource · maturity maintenance

web-scraping developer-tools crawlers developer-tools tutorials python cli cross-platform scrapy spiders example-code proxy-support web-crawling

Member repositories

For agents

markdown · JSON · MCP: product_card(name="geekan/scrapy-examples")