SpiderClub/weibospider
:zap: A distributed crawler for weibo, building with celery and requests. observed · 2026-08-28
Health v2 · maintenance only
32/100
- Activity 0
- Release rhythm 35
- Longevity 100
Flags: no_releases
How is this computed?
round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10) — computed 2026-09-02. Adoption (stars, forks) is never an input.
- gap_med: n/a
- age_days: 3719
- days_rel: n/a
- days_push: 2244
- n_releases_24m: 0
Adoption not part of the score
4793 stars · 1181 forks observed · 2026-08-28
What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded
A distributed web crawler for Sina Weibo (Chinese microblogging platform) built with Python, Celery, and requests. It scrapes user profiles, keyword search results, posts, comments, and repost relationships, storing data in MySQL with an optional Django-based admin UI for configuration.
Use cases
- scrape weibo user profiles and posts
- collect weibo search results for keywords
- harvest comments and reposts from weibo
- build a weibo dataset for social media analysis
- run a distributed crawler for sina weibo data
When to choose
- you need structured weibo data (users, posts, comments, reposts) at scale
- you want a battle-tested crawler with account management and error handling built in
- you prefer a self-hosted Python stack with Celery workers and MySQL storage
When to avoid
- you need data from platforms other than Weibo
- you want an actively maintained project - the last release was in 2020 and Weibo's API/pages have likely changed
- you need a simple one-off scrape without setting up MySQL, Redis, and Celery infrastructure
Facets
application · maturity maintenance
web-scraping etl data-science social-media crawlers data-science big-data python self-hosted weibo distributed-crawler celery sina-weibo scraping django-admin mysql redis crawlers linux docker
1 source
- readme: https://github.com/SpiderClub/weibospider · fetched 2026-08-28 · 8499b5685da1
Member repositories
| Repository | Role | Health v2 |
|---|---|---|
| SpiderClub/weibospider | main | 32 |
For agents
markdown · JSON · MCP: product_card(name="SpiderClub/weibospider")
Data as of 2026-08-30T08:39:29.467469+00:00 · Report a problem