# kkangert/kspider

Kspider 是一个爬虫平台，以图形化方式定义爬虫流程，无需代码即可实现一个爬虫流程，Kspider不仅限爬虫，也可用于WEB自动化测试，更多功能等你探索。

Repository: https://github.com/kkangert/kspider
Canonical: https://ross.abutalabs.com/products/kspider
Homepage: https://kkangert.github.io/kspider-docs/
Language: Java
License: MIT
License Family: permissive
Last push: 2024-08-19T03:06:36+00:00

## Health v2 (maintenance only)
Score: 14/100 (v2, computed 2026-09-02T17:46:02.011165+00:00)
- activity 0, release rhythm 8, longevity 56
- inputs: {"age_days": 787, "days_push": 744, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 1269, forks 124 (observed 2026-08-28T04:04:11.743726+00:00)

## What it is
Kspider is a self-hosted visual web scraping platform written in Java where users define crawler workflows as flowcharts without writing backend code, featuring xpath/css/regex/JSON extraction, cookie management, proxies, Selenium-based rendering of JS pages, and extensible plugins. The same graphical workflow engine also serves as a no-code web automation testing tool, with scheduled tasks, result export to multiple databases or CSV, and containerized deployment.

## Use cases
- scrape websites without writing code
- build a visual web crawler with a flowchart editor
- extract data from javascript-rendered pages
- automate browser testing without programming
- schedule recurring scraping tasks
- save scraped data to mysql, postgresql, or csv
- self-hosted alternative to commercial scraping tools

## When to choose
- You want to assemble crawlers or web automation flows visually with minimal code
- You need built-in handling of cookies, proxies, random user agents, and JS-rendered pages out of the box
- You want to route extracted results to multiple destinations such as relational databases and CSV files
- You need a self-hosted, plugin-extensible platform in the spirit of SpiderFlow with an online management UI

## When to avoid
- You need large-scale distributed scraping, which is still on the TODO list
- You prefer fully programmatic scraping frameworks like Scrapy or Playwright for fine-grained control
- You require OCR extraction or a multi-user management system, which are planned but not yet implemented
- You need long-term enterprise-grade support or extensive English documentation, as the project and docs are primarily Chinese

## Facets
- artifact type: application
- maturity: active
- function: web-scraping, workflow-automation, e2e-testing, scheduling, proxy, plugin-system
- domain: crawlers, testing, developer-tools, self-hosted
- platform: self-hosted, jvm, windows
- tags: no-code, visual-programming, flowchart-editor, selenium, xpath, css-selectors, browser-automation, data-extraction, proxy-support, user-agent-rotation, spring-boot, java, chinese-language, crawler-platform, automation, data-engineering, docker, web-server, linux, macos

## Member repositories
- kkangert/kspider (main) score 14

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:04:11.743726+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T05:03:32.249280+00:00, confidence not recorded.
  - readme: https://github.com/kkangert/kspider (fetched 2026-08-28T04:04:11.743726+00:00, sha 1f60a0a74fe1)
  - homepage: https://kkangert.github.io/kspider-docs/ (fetched 2026-08-29T12:14:59.385351+00:00, sha f9b55629e4ad)
- Data as of 2026-08-30T08:39:29.467469+00:00.