# brianway/webporter

基于 webmagic 的 Java 爬虫应用

Repository: https://github.com/brianway/webporter
Canonical: https://ross.abutalabs.com/products/webporter
Language: Java
License Family: other
Topics: elasticsearch, kibana, zhihu
Last push: 2022-01-08T13:14:10+00:00

## Health v2 (maintenance only)
Score: 23/100 (v2, computed 2026-09-02T17:46:02.011165+00:00)
- activity 0, release rhythm 8, longevity 100
- inputs: {"age_days": 3569, "days_push": 1698, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: no_license
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 2766, forks 847 (observed 2026-08-28T04:07:18.652787+00:00)

## What it is
webporter is a Java crawler application built on the webmagic framework that demonstrates a complete pipeline of data crawling, persistence, and visualization. It ships a Zhihu user-data collector example with modules for core logic, Elasticsearch bulk import, and site-specific collection configured via JSON.

## Use cases
- scrape zhihu user profiles into elasticsearch
- learn how to build a java crawler with webmagic
- bulk import crawled json data into elasticsearch
- visualize scraped data with kibana aggregations
- example of a full crawl-store-analyze pipeline
- configure a crawler without changing source code

## When to choose
- you want a worked Java example of crawling plus Elasticsearch ingestion
- you need to collect Zhihu user data for analysis or learning
- you prefer JSON configuration over code changes

## When to avoid
- you need a production-grade crawler - use webmagic or scrapy instead
- you need a general-purpose scraping framework rather than a sample application
- you need an actively maintained project with a clear license

## Facets
- artifact type: application
- maturity: maintenance
- function: web-scraping, search-engine, data-visualization, etl
- domain: crawlers, data-science, developer-tools
- platform: jvm, cli
- tags: zhihu, webmagic, elasticsearch, kibana, crawler-example, json-config, multi-threaded, search, docker

## Member repositories
- brianway/webporter (main) score 23

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:07:18.652787+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T08:16:45.192968+00:00, confidence not recorded.
  - readme: https://github.com/brianway/webporter (fetched 2026-08-28T04:07:18.652787+00:00, sha 5ff238f39b2a)
- Data as of 2026-08-30T08:39:29.467469+00:00.