# michaelfeil/infinity

Infinity is a high-throughput, low-latency serving engine for text-embeddings, reranking models, clip, clap and colpali

Repository: https://github.com/michaelfeil/infinity
Canonical: https://ross.abutalabs.com/products/michaelfeil-infinity
Homepage: https://michaelfeil.github.io/infinity/
Language: Python
License: MIT
License Family: permissive
Topics: llm, text-embeddings, bert-embeddings
Last push: 2026-03-24T03:59:47+00:00

## Health v2 (maintenance only)
Score: 63/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 73, release rhythm 44, longevity 75
- inputs: {"age_days": 1057, "days_push": 162, "days_rel": 376, "gap_med": 3.5, "n_releases_24m": 23}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 2917, forks 197 (observed 2026-08-28T04:07:30.051502+00:00)

## What it is
Infinity is a high-throughput, low-latency serving engine that exposes text-embedding, reranking, CLIP, CLAP and ColPali models via an OpenAI-compatible REST API. It is built on FastAPI with PyTorch, ONNX/TensorRT and CTranslate2 backends, supporting dynamic batching across CUDA, ROCm, CPU, AWS Inf2 and Apple MPS.

## Use cases
- self-host an OpenAI-compatible embeddings API
- serve sentence-transformer models for a RAG pipeline
- deploy reranking models behind a REST endpoint
- run CLIP image and text embeddings on GPU
- batch-serve multiple embedding models from one server
- host ColPali vision retrieval models

## When to choose
- you need a self-hosted, OpenAI-spec-compatible embedding server
- you want to mix multiple embedding/reranking/multimodal models in one service
- you need high throughput with dynamic batching and hardware acceleration
- you want an MIT-licensed alternative to proprietary embedding APIs

## When to avoid
- you only need embeddings inside a single Python app without a server
- you need full LLM text generation, not embeddings or reranking
- you want a fully managed hosted service rather than self-hosting

## Facets
- artifact type: service
- maturity: active
- function: llm-inference, http-server, api-framework, machine-learning, rag
- domain: machine-learning, large-language-models, apis, self-hosted, backend
- platform: python, windows, self-hosted, cross-platform
- tags: embeddings, reranking, clip, sentence-transformers, openai-compatible-api, text-embeddings-inference, fastapi, docker, linux, macos, gpu

## Member repositories
- michaelfeil/infinity (main) score 63

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:07:30.051502+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T07:34:04.975419+00:00, confidence not recorded.
  - readme: https://github.com/michaelfeil/infinity (fetched 2026-08-28T04:07:30.051502+00:00, sha 816365ff1961)
  - homepage: https://michaelfeil.github.io/infinity/ (fetched 2026-08-29T09:49:18.713466+00:00, sha 6c24e505942a)
- Data as of 2026-08-30T08:39:29.467469+00:00.