huggingface/text-embeddings-inference
A blazing fast inference solution for text embeddings models observed · 2026-08-28
Health v2 · maintenance only
84/100
- Activity 94
- Release rhythm 76
- Longevity 75
How is this computed?
round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10) — computed 2026-09-02. Adoption (stars, forks) is never an input.
- gap_med: 26
- age_days: 1055
- days_rel: 163
- days_push: 40
- n_releases_24m: 16
Adoption not part of the score
5028 stars · 426 forks observed · 2026-08-28
What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded
Text Embeddings Inference (TEI) is a Rust-based toolkit from Hugging Face for deploying and serving open-source text embedding, re-ranker, and sequence classification models at high performance. It supports dynamic batching, Flash Attention, Safetensors/ONNX weight loading, and production features like Prometheus metrics and OpenTelemetry tracing.
Use cases
- serve embedding models for a RAG pipeline
- host a fast embeddings API endpoint for semantic search
- deploy re-ranker models behind an HTTP or gRPC API
- run embeddings inference on GPU, CPU, or Apple Silicon locally
- self-host embeddings instead of paying per-token API costs
- batch-embed large document collections efficiently
When to choose
- you need production-grade, low-latency serving of supported embedding models like BGE, GTE, E5, Qwen3, or Nomic
- you want small Docker images, fast boot times, and serverless-friendly deployment
- you need dynamic batching, tracing, and metrics out of the box
- you want to serve private or gated Hugging Face models on your own infrastructure
When to avoid
- you need to serve embedding model architectures not on the supported list
- you only need occasional embeddings and prefer a hosted API without infrastructure
- you need text generation inference rather than embeddings (use TGI instead)
- you want a Python-native stack for experimentation rather than a Rust server
Facets
service · maturity active
llm-inference http-server machine-learning rag machine-learning large-language-models artificial-intelligence developer-tools rust self-hosted embeddings huggingface rerankers sequence-classification flash-attention safetensors onnx grpc prometheus open-telemetry retrieval-augmented-generation docker linux macos gpu web-server
10 sources
- readme: https://github.com/huggingface/text-embeddings-inference · fetched 2026-08-28 · a752b8732804
- homepage: https://huggingface.co/docs/text-embeddings-inference/quick_tour · fetched 2026-08-29 · c8d318cfef15
- site_page: https://huggingface.co/docs · fetched 2026-08-29 · bdec26667b98
- site_page: https://huggingface.co/docs/text-embeddings-inference/index · fetched 2026-08-29 · 258947bb5df4
- site_page: https://huggingface.co/docs/text-embeddings-inference/supported_models · fetched 2026-08-29 · a299924ec629
- site_page: https://huggingface.co/docs/text-embeddings-inference/local_cpu · fetched 2026-08-29 · f8fc18d6d5d6
- site_page: https://huggingface.co/docs/text-embeddings-inference/local_metal · fetched 2026-08-29 · 3060e91b11c1
- site_page: https://huggingface.co/docs/text-embeddings-inference/local_gpu · fetched 2026-08-29 · f95a39ec6f64
- site_page: https://huggingface.co/docs/text-embeddings-inference/private_models · fetched 2026-08-29 · 4e7c87089927
- site_page: https://huggingface.co/pricing · fetched 2026-08-29 · de6b7a178be5
Member repositories
| Repository | Role | Health v2 |
|---|---|---|
| huggingface/text-embeddings-inference | main | 84 |
For agents
markdown · JSON · MCP: product_card(name="huggingface/text-embeddings-inference")
Data as of 2026-08-30T08:39:29.467469+00:00 · Report a problem