aivarsk/scrapy-proxies

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

1667 stars · 421 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

A Scrapy downloader middleware that routes requests through random proxies from a configurable list to avoid IP bans. It supports multiple proxy modes, including per-request rotation, a single fixed proxy, and a custom proxy set in settings.

Use cases

• rotate proxies randomly while scraping with scrapy
• avoid getting IP banned during web crawling
• assign a single proxy to all scrapy requests
• use a custom proxy in scrapy settings
• retry failed requests through different proxies

When to choose

• you already use Scrapy and need simple file-based proxy rotation
• you want a lightweight middleware without extra dependencies
• you need configurable proxy modes (per-request, fixed, or custom)

When to avoid

• you need proxy health checking, scoring, or automatic proxy list fetching
• you use a framework other than Scrapy
• you need actively maintained software with recent releases

Facets

library · maturity maintenance

middleware web-scraping proxy web-development crawlers developer-tools python cross-platform scrapy proxy-rotation downloader-middleware ip-ban-avoidance

Member repositories

For agents

markdown · JSON · MCP: product_card(name="aivarsk/scrapy-proxies")