basetenlabs/truss

Health v2 · maintenance only

Adoption not part of the score

1199 stars · 122 forks observed · 2026-09-03

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Truss is a Python CLI and packaging framework for deploying and serving AI/ML models in production, primarily on the Baseten platform. It handles containerization, dependency management, and GPU configuration, supporting frameworks like vLLM, SGLang, TensorRT-LLM, transformers, and PyTorch.

Use cases

• deploy a hugging face model as a production api endpoint
• serve an llm on gpu infrastructure
• package pytorch model for inference in production
• deploy stable diffusion or whisper model as an api
• run vllm or sglang model server without writing dockerfiles
• iterate on model serving code with live reload
• turn a model checkpoint into an openai-compatible api

When to choose

• you want to deploy custom or fine-tuned models to Baseten with minimal configuration
• you need GPU-backed model serving with autoscaling and secrets management
• you want a config-file-only path to production for open-source LLMs
• you need the same model server behavior in development and production

When to avoid

• you only need to call hosted models via API without deploying your own
• you want a fully cloud-agnostic serving stack with no Baseten dependency
• you need non-Python model serving workflows
• you prefer managing your own Kubernetes and Docker setup directly

Facets

cli-tool · maturity active

llm-inference machine-learning deployment container-runtime cli sdk machine-learning artificial-intelligence large-language-models cloud-computing developer-tools python cli cloud model-serving inference-server model-deployment baseten gpu-inference vllm containerization mlops docker gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="basetenlabs/truss")