# kingname/GeneralNewsExtractor

新闻网页正文通用抽取器 Beta 版.

Repository: https://github.com/kingname/GeneralNewsExtractor
Canonical: https://ross.abutalabs.com/products/generalnewsextractor
Homepage: http://gne.kingname.info/
Language: Python
License: GPL-3.0
License Family: copyleft
Topics: python3, webcrawler, webspider
Last push: 2026-04-21T23:03:46+00:00

## Health v2 (maintenance only)
Score: 81/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 78, release rhythm 74, longevity 100
- inputs: {"age_days": 2551, "days_push": 134, "days_rel": 179, "gap_med": 0, "n_releases_24m": 2}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 3788, forks 540 (observed 2026-08-28T04:08:18.667129+00:00)

## What it is
GNE (GeneralNewsExtractor) is a Python library that extracts article title, author, publish time, body text, and images from news page HTML using a text and symbol density algorithm. It takes already-fetched HTML as input and returns a structured dictionary, deliberately excluding any crawling/request functionality.

## Use cases
- extract article body text from news pages
- scrape news articles without site-specific rules
- get title author and publish time from html
- extract article listings from an index page
- clean noise like comments and ads out of article html
- build a news aggregator pipeline

## When to choose
- you already have rendered HTML and need structured article data
- you want a lightweight extractor with no heavy dependencies
- you target Chinese news sites or general news layouts without writing per-site selectors

## When to avoid
- you need a full crawler that fetches pages from websites
- you need extraction of non-news page types like e-commerce or forums
- you require JavaScript rendering - you must fetch and render HTML yourself first

## Facets
- artifact type: library
- maturity: active
- function: parser, web-scraping, nlp
- domain: web-development, crawlers
- platform: python, cross-platform
- tags: news-extraction, html-parsing, text-density, content-extraction, metadata-extraction, natural-language-processing

## Member repositories
- kingname/GeneralNewsExtractor (main) score 81

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:08:18.667129+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-29T18:27:28.030584+00:00, confidence not recorded.
  - readme: https://github.com/kingname/GeneralNewsExtractor (fetched 2026-08-28T04:08:18.667129+00:00, sha 47f88965b8c7)
  - homepage: http://gne.kingname.info/ (fetched 2026-08-29T09:22:11.796676+00:00, sha 501a159e32c6)
  - registry_pypi: https://pypi.org/pypi/generalnewsextractor/json (fetched 2026-08-29T09:22:11.805515+00:00, sha 369874815e1b)
- Data as of 2026-08-30T08:39:29.467469+00:00.