LiuRoy/zhihu_spider

Health v2 · maintenance only

Flags: no_releases no_license

Adoption not part of the score

1282 stars · 611 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

A Scrapy-based web crawler that scrapes Zhihu user profiles and their follower/following relationship graphs, storing data in MongoDB. It serves primarily as an educational example of building a login-aware Scrapy spider with Celery-based async image downloading.

Use cases

• scrape zhihu user profiles
• crawl social network follower relationships
• learn scrapy with a real example
• extract user data from zhihu
• build a social graph from a website
• scrape user avatars asynchronously

When to choose

• you want a worked Scrapy example with login, CSRF handling, and pagination
• you need to crawl Zhihu user data and relationship topology
• you want to see Scrapy combined with MongoDB, RabbitMQ, and Celery

When to avoid

• you need a maintained tool - the last release was 2016 and Zhihu's site has changed
• you want scraping without providing your own Zhihu account credentials
• you need a general-purpose scraping framework rather than a site-specific spider

Facets

cli-tool · maturity abandoned

web-scraping parser crawlers social-media python cli scrapy zhihu mongodb rabbitmq celery social-graph data-engineering docker

Member repositories

For agents

markdown · JSON · MCP: product_card(name="LiuRoy/zhihu_spider")