dleemiller/WordLlama

Health v2 · maintenance only

Adoption not part of the score

1451 stars · 48 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

WordLlama is a fast, lightweight Python NLP toolkit built on LLM token embeddings for tasks like similarity computation, ranking, fuzzy deduplication, clustering, filtering, and semantic text splitting. It runs on CPU with minimal dependencies, making it suitable for resource-constrained environments.

Use cases

• compute semantic similarity between two texts
• rank documents by relevance to a query
• deduplicate a large corpus of near-duplicate texts
• cluster documents by topic
• split long documents into semantically coherent chunks
• top-k retrieval of similar documents
• embed text cheaply on CPU without a GPU

When to choose

• you need lightweight semantic text similarity on CPU without heavy inference dependencies
• you want fast fuzzy deduplication or clustering of text corpora
• you need semantic chunking/splitting for RAG pipelines on limited hardware

When to avoid

• you need state-of-the-art full transformer embedding quality
• you require multilingual embeddings beyond the model's training
• you need GPU-accelerated or fine-tunable embedding models

Facets

library · maturity active

nlp machine-learning search-engine data-science machine-learning developer-tools python cross-platform embeddings semantic-similarity fuzzy-deduplication clustering semantic-splitting cpu-optimized token-embeddings llm-derived natural-language-processing cpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="dleemiller/WordLlama")