howie6879/ruia

Health v2 · maintenance only

Adoption not part of the score

1738 stars · 185 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Ruia is an async web scraping micro-framework for Python 3.6+ built on asyncio and aiohttp. It offers declarative Item/Field extraction (XPath, CSS selectors), spider control, and extensibility through middlewares and plugins.

Use cases

• scrape product prices from e-commerce sites
• crawl multiple pages concurrently with asyncio
• extract structured data from HTML using CSS selectors or XPath
• build a custom web crawler with middlewares
• fetch and parse Hacker News-style listing pages
• write an async spider without heavy frameworks like Scrapy

When to choose

• you want a lightweight, asyncio-native alternative to Scrapy
• you prefer declarative item definitions with XPath/CSS extraction
• you need fine-grained control via middlewares and plugins
• your scraping workload benefits from async concurrency

When to avoid

• you need distributed crawling out of the box (still a TODO)
• you require heavy JavaScript rendering without adding plugins
• you need a framework with a large ecosystem and long-term commercial support
• you are on Python older than 3.6

Facets

framework · maturity maintenance

web-scraping http-client middleware plugin-system crawlers web-development python windows cross-platform asyncio aiohttp spider crawler declarative uvloop micro-framework automation linux macos

Member repositories

For agents

markdown · JSON · MCP: product_card(name="howie6879/ruia")