# srx-2000/spider_collection

python爬虫，目前库存：网易云音乐歌曲爬取，B站视频爬取，知乎问答爬取，壁纸爬取，xvideos视频爬取，有声书爬取，微博爬虫，安居客信息爬取+数据可视化，哔哩哔哩视频封面提取器，ip代理池封装，知乎百万级用户爬虫+数据分析，github用户爬虫

Repository: https://github.com/srx-2000/spider_collection
Canonical: https://ross.abutalabs.com/products/spider_collection
Language: Python
License: MIT
License Family: permissive
Topics: python, spider, crawler
Last push: 2024-04-23T02:28:05+00:00

## Health v2 (maintenance only)
Score: 32/100 (v2, computed 2026-09-03T02:39:23.370411+00:00)
- activity 0, release rhythm 35, longevity 100
- inputs: {"age_days": 2402, "days_push": 863, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: no_releases
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 1646, forks 239 (observed 2026-08-28T04:05:16.281495+00:00)

## What it is
A collection of Python web crawler scripts targeting sites like Bilibili, Zhihu, Weibo, NetEase Music, GitHub, and Anjuke, built with requests, Scrapy, parsel, and BeautifulSoup. It also includes a re-wrapped IP proxy pool and simple data analysis/visualization examples using pandas and matplotlib.

## Use cases
- download bilibili videos or extract video covers
- scrape zhihu answers and user profiles for analysis
- collect weibo user info
- download netease cloud music playlists
- scrape rental listings from anjuke and visualize them
- learn how to build python crawlers with scrapy and requests
- set up a wrapped ip proxy pool for anti-scraping

## When to choose
- you want ready-made example crawlers for popular Chinese websites
- you are learning web scraping techniques like threading, proxy pools, and scrapy
- you need a starting point to adapt a crawler for bilibili, zhihu, or weibo

## When to avoid
- you need a production-grade, maintained scraping framework
- you need a general-purpose crawler with a stable API
- the target site has changed its layout and the script is outdated (some spiders are marked deprecated)

## Facets
- artifact type: library
- maturity: active
- function: web-scraping, data-visualization, caching, developer-tools
- domain: crawlers, data-science, social-media, developer-tools
- platform: python, cross-platform
- tags: scrapy, requests, proxy-pool, bilibili, zhihu, weibo, netease-music, multi-spider-collection, learning-project, automation

## Member repositories
- srx-2000/spider_collection (main) score 32

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:05:16.281495+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T03:45:24.115536+00:00, confidence not recorded.
  - readme: https://github.com/srx-2000/spider_collection (fetched 2026-08-28T04:05:16.281495+00:00, sha 5b598cff5b9f)
- Data as of 2026-08-30T08:39:29.467469+00:00.