shibing624/text2vec

Health v2 · maintenance only

Adoption not part of the score

4974 stars · 428 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

A Python library that converts text (words, sentences, paragraphs) into vector embeddings, implementing Word2Vec, RankBM25, BERT, Sentence-BERT, and CoSENT models. It provides ready-to-use text representation and semantic similarity computation, with pretrained Chinese and multilingual models and a CLI for batch vectorization.

Use cases

• compute semantic similarity between two sentences
• generate sentence embeddings for Chinese text
• build a semantic search or retrieval system
• encode documents into vectors for clustering
• train a custom text matching model with CoSENT
• find duplicate or paraphrase questions in a corpus

When to choose

• you need out-of-the-box sentence embeddings, especially for Chinese or multilingual text
• you want to compare multiple text similarity models (BM25, SBERT, CoSENT) in one library
• you need pretrained matching models with a simple Python API or CLI

When to avoid

• you need production-scale vector database features like ANN indexing and filtering
• you only need English embeddings and prefer the broader sentence-transformers ecosystem
• you need the latest LLM-based embedding models rather than BERT-era encoders

Facets

library · maturity active

nlp machine-learning search-engine machine-learning python embeddings sentence-embeddings text-similarity word2vec sentence-bert cosent chinese-nlp semantic-search natural-language-processing search

Member repositories

For agents

markdown · JSON · MCP: product_card(name="shibing624/text2vec")