# basetenlabs/truss

The simplest way to serve AI/ML models in production

Repository: https://github.com/basetenlabs/truss
Canonical: https://ross.abutalabs.com/products/truss
Homepage: https://truss.baseten.co
Language: Python
License: MIT
License Family: permissive
Topics: machine-learning, artificial-intelligence, easy-to-use, inference-api, inference-server, model-serving, open-source, packaging, falcon, stable-diffusion, whisper, wizardlm
Last push: 2026-09-02T20:53:34+00:00

## Health v2 (maintenance only)
Score: 96/100 (v2, computed 2026-09-03T02:39:23.370411+00:00)
- activity 100, release rhythm 88, longevity 100
- inputs: {"age_days": 1519, "days_push": 0, "days_rel": 1, "gap_med": 1, "n_releases_24m": 222}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 1199, forks 122 (observed 2026-09-03T02:15:09.561480+00:00)

## What it is
Truss is a Python CLI and packaging framework for deploying and serving AI/ML models in production, primarily on the Baseten platform. It handles containerization, dependency management, and GPU configuration, supporting frameworks like vLLM, SGLang, TensorRT-LLM, transformers, and PyTorch.

## Use cases
- deploy a hugging face model as a production api endpoint
- serve an llm on gpu infrastructure
- package pytorch model for inference in production
- deploy stable diffusion or whisper model as an api
- run vllm or sglang model server without writing dockerfiles
- iterate on model serving code with live reload
- turn a model checkpoint into an openai-compatible api

## When to choose
- you want to deploy custom or fine-tuned models to Baseten with minimal configuration
- you need GPU-backed model serving with autoscaling and secrets management
- you want a config-file-only path to production for open-source LLMs
- you need the same model server behavior in development and production

## When to avoid
- you only need to call hosted models via API without deploying your own
- you want a fully cloud-agnostic serving stack with no Baseten dependency
- you need non-Python model serving workflows
- you prefer managing your own Kubernetes and Docker setup directly

## Facets
- artifact type: cli-tool
- maturity: active
- function: llm-inference, machine-learning, deployment, container-runtime, cli, sdk
- domain: machine-learning, artificial-intelligence, large-language-models, cloud-computing, developer-tools
- platform: python, cli, cloud
- tags: model-serving, inference-server, model-deployment, baseten, gpu-inference, vllm, containerization, mlops, docker, gpu

## Member repositories
- basetenlabs/truss (main) score 96

## Provenance
- Observed fields: from GitHub, fetched 2026-09-03T02:15:09.561480+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T06:22:09.348881+00:00, confidence not recorded.
  - readme: https://github.com/basetenlabs/truss (fetched 2026-09-03T02:15:09.561480+00:00, sha 23ed39cb485a)
  - homepage: https://truss.baseten.co (fetched 2026-08-29T12:29:36.716680+00:00, sha e83f80aae721)
  - site_page: https://docs.baseten.co/quickstart (fetched 2026-08-29T12:29:36.719371+00:00, sha 6b75450d6a88)
  - site_page: https://docs.baseten.co/inference/model-apis/pricing-and-limits (fetched 2026-08-29T12:29:36.721354+00:00, sha 0f67cde41748)
  - site_page: https://docs.baseten.co/inference/integrations (fetched 2026-08-29T12:29:36.723462+00:00, sha 1e80b421f59d)
  - site_page: https://docs.baseten.co/overview (fetched 2026-08-29T12:29:36.725333+00:00, sha eb1d364e501d)
  - site_page: https://www.baseten.co/changelog (fetched 2026-08-29T12:29:36.727240+00:00, sha b3c89f738cbe)
- Data as of 2026-08-30T08:39:29.467469+00:00.