fxsjy/jieba

Health v2 · maintenance only

Adoption not part of the score

35130 stars · 6686 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

Jieba is the most popular Python library for Chinese word segmentation, supporting precise, full, search-engine, and PaddlePaddle-based segmentation modes. It includes traditional Chinese support, custom dictionaries, part-of-speech tagging, and new word detection via an HMM model.

Use cases

• segment chinese text into words in python
• tokenize chinese text for search engine indexing
• add custom domain words to a chinese tokenizer
• do part-of-speech tagging for chinese sentences
• split chinese text for text analysis
• handle traditional chinese text segmentation
• detect new or out-of-vocabulary words in chinese text

When to choose

• you need reliable, battle-tested Chinese word segmentation in Python
• you want multiple segmentation modes including a search-engine-friendly mode
• you need custom dictionaries or traditional Chinese support
• you want a lightweight pure-Python solution with no heavy dependencies

When to avoid

• you need segmentation for languages other than Chinese
• you need state-of-the-art neural segmentation and are willing to run larger deep learning models
• you need actively developed features - the project is largely in maintenance mode

Facets

library · maturity maintenance

nlp parser search-engine localization python cross-platform chinese-word-segmentation text-segmentation tokenization traditional-chinese custom-dictionary hmm natural-language-processing search

Member repositories

For agents

markdown · JSON · MCP: product_card(name="fxsjy/jieba")