kingname/GeneralNewsExtractor
新闻网页正文通用抽取器 Beta 版. observed · 2026-08-28
Health v2 · maintenance only
81/100
- Activity 78
- Release rhythm 74
- Longevity 100
How is this computed?
round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10) — computed 2026-09-03. Adoption (stars, forks) is never an input.
- gap_med: 0
- age_days: 2551
- days_rel: 179
- days_push: 134
- n_releases_24m: 2
Adoption not part of the score
3788 stars · 540 forks observed · 2026-08-28
What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded
GNE (GeneralNewsExtractor) is a Python library that extracts article title, author, publish time, body text, and images from news page HTML using a text and symbol density algorithm. It takes already-fetched HTML as input and returns a structured dictionary, deliberately excluding any crawling/request functionality.
Use cases
- extract article body text from news pages
- scrape news articles without site-specific rules
- get title author and publish time from html
- extract article listings from an index page
- clean noise like comments and ads out of article html
- build a news aggregator pipeline
When to choose
- you already have rendered HTML and need structured article data
- you want a lightweight extractor with no heavy dependencies
- you target Chinese news sites or general news layouts without writing per-site selectors
When to avoid
- you need a full crawler that fetches pages from websites
- you need extraction of non-news page types like e-commerce or forums
- you require JavaScript rendering - you must fetch and render HTML yourself first
Facets
library · maturity active
parser web-scraping nlp web-development crawlers python cross-platform news-extraction html-parsing text-density content-extraction metadata-extraction natural-language-processing
3 sources
- readme: https://github.com/kingname/GeneralNewsExtractor · fetched 2026-08-28 · 47f88965b8c7
- homepage: http://gne.kingname.info/ · fetched 2026-08-29 · 501a159e32c6
- registry_pypi: https://pypi.org/pypi/generalnewsextractor/json · fetched 2026-08-29 · 369874815e1b
Member repositories
| Repository | Role | Health v2 |
|---|---|---|
| kingname/GeneralNewsExtractor | main | 81 |
For agents
markdown · JSON · MCP: product_card(name="kingname/GeneralNewsExtractor")
Data as of 2026-08-30T08:39:29.467469+00:00 · Report a problem