ssssssss-team/spider-flow

Health v2 · maintenance only

Adoption not part of the score

11352 stars · 2169 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

Spider-Flow is a self-hosted Java-based web crawler platform that lets users define scraping workflows visually as flowcharts without writing code. It supports XPath/JsonPath/CSS selector/regex extraction, JS-rendered pages, proxies, database persistence, and extension via plugins like Selenium, Redis, MongoDB, and OCR.

Use cases

• scrape websites without writing code
• build a visual web crawler with a drag-and-drop flowchart
• extract data from JS-rendered or AJAX pages
• schedule and monitor crawling tasks with logs
• save scraped data automatically to a database or files
• scrape sites behind proxies or captchas with OCR

When to choose

• you want a no-code/low-code scraping platform with a visual editor
• your team needs to maintain many crawlers with monitoring and debugging UI
• you need extensible scraping with plugins for Selenium, proxies, or OCR
• you want a self-hosted Java crawler platform under an MIT license

When to avoid

• you need a lightweight code-first scraping library instead of a full platform
• your project requires active development or recent updates (last release mid-2023)
• you need distributed large-scale crawling beyond a single self-hosted instance
• you prefer Python-based scraping ecosystems like Scrapy

Facets

application · maturity maintenance

web-scraping parser workflow-automation plugin-system crawlers web-development developer-tools jvm self-hosted visual-crawler-builder low-code flow-based-programming xpath path css-selectors jsoup selenium proxy-support ocr automation web-server

Member repositories

For agents

markdown · JSON · MCP: product_card(name="ssssssss-team/spider-flow")