superlinked/sie

Health v2 · maintenance only

Adoption not part of the score

2830 stars · 280 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

SIE (Superlinked Inference Engine) is an open-source, self-hosted inference server and production cluster that serves 100+ open models (embedders, rerankers, extractors, LLMs) behind one OpenAI-compatible API. It handles multi-model GPU sharing, on-demand model loading, and ships a full production stack with KEDA autoscaling and Terraform for GKE, EKS, and AKS.

Use cases

• self-host embeddings and rerankers for RAG
• run open LLM inference in my own cloud
• replace per-token embedding API costs with self-hosted models
• serve many small ML models on one GPU
• add semantic search embeddings to Qdrant or Weaviate
• deploy an OpenAI-compatible inference endpoint on Kubernetes
• extract entities and OCR documents for agent pipelines
• air-gapped inference for agents

When to choose

• You need self-hosted, privacy-preserving inference for embeddings, reranking, extraction, and small LLMs behind one API
• You run many small models on shared GPUs and need fast model switching
• You want OpenAI-compatible endpoints for drop-in migration from hosted APIs
• You need production deployment with autoscaling on GKE, EKS, or AKS

When to avoid

• You only need to serve one very large model across many GPUs - tools like vLLM or SGLang alone fit better
• You want a fully managed hosted API with no infrastructure to run
• You need non-Python/TypeScript SDKs or integrations beyond the supported vector stores and agent frameworks

Facets

service · maturity active

llm-inference rag machine-learning nlp search-engine ocr api-framework http-server gpu-computing sdk machine-learning large-language-models self-hosted developer-tools python self-hosted cloud cross-platform inference-server embeddings reranking vector-search openai-compatible-api model-serving rag-pipeline gpu-sharing semantic-search splade colbert mlops retrieval-augmented-generation natural-language-processing search ai-agents docker kubernetes gpu linux macos

Member repositories

For agents

markdown · JSON · MCP: product_card(name="superlinked/sie")