# bentoml/BentoML

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

Repository: https://github.com/bentoml/BentoML
Canonical: https://ross.abutalabs.com/products/bentoml
Homepage: https://bentoml.com
Language: Python
License: Apache-2.0
License Family: permissive
Topics: model-serving, mlops, llmops, generative-ai, llm-inference, model-inference-service, inference-platform, deep-learning, llm-serving, machine-learning, python, multimodal, ml-engineering, llm, ai-inference
Last push: 2026-08-26T07:56:48+00:00

## Health v2 (maintenance only)
Score: 94/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 99, release rhythm 83, longevity 100
- inputs: {"age_days": 2711, "days_push": 7, "days_rel": 118, "gap_med": 9, "n_releases_24m": 58}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 8808, forks 1015 (observed 2026-08-28T04:10:25.830744+00:00)

## What it is
BentoML is a Python framework for building online serving systems for AI apps and model inference, turning model inference scripts into REST API servers with minimal code. It packages models and dependencies into reproducible Docker containers and provides serving optimizations like dynamic batching, multi-model orchestration, and scaling features.

## Use cases
- serve a machine learning model as a REST API
- deploy LLM inference endpoints with vLLM
- build multi-model inference pipelines
- package ML models into Docker containers for deployment
- run batch inference job queues
- deploy Stable Diffusion or RAG applications as APIs
- scale model inference with autoscaling and dynamic batching
- serve models on GPU with optimized utilization

## When to choose
- you need to turn Python model inference code into production API servers quickly
- you want reproducible Docker packaging and deployment for ML models
- you need high-performance serving with batching, parallelism, and multi-model composition
- you serve LLMs or generative AI models and want day-one support for new open models
- you want to deploy to Kubernetes, your own cloud, or a managed inference platform

## When to avoid
- you only need lightweight model training or experimentation without serving
- you need a non-Python serving stack
- you want a fully managed inference service with zero infrastructure code
- your workload is simple enough that a plain FastAPI wrapper suffices

## Facets
- artifact type: framework
- maturity: stable
- function: llm-inference, machine-learning, web-framework, api-framework, deployment, container-runtime, monitoring, gpu-computing
- domain: machine-learning, large-language-models, deep-learning, artificial-intelligence, cloud-computing, developer-tools
- platform: python, cloud, self-hosted, windows
- tags: model-serving, mlops, llmops, inference-api, model-deployment, dynamic-batching, task-queues, docker-packaging, multi-model-serving, generative-ai, docker, kubernetes, gpu, linux, macos

## Member repositories
- bentoml/BentoML (main) score 94

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:10:25.830744+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-29T17:25:10.810004+00:00, confidence not recorded.
  - readme: https://github.com/bentoml/BentoML (fetched 2026-08-28T04:10:25.830744+00:00, sha 31b7ed7eaebc)
  - homepage: https://bentoml.com (fetched 2026-08-29T08:25:23.909480+00:00, sha c9b0ba7568d6)
  - site_page: https://docs.bentoml.com/ (fetched 2026-08-29T08:25:23.912068+00:00, sha de09c4f72293)
  - registry_pypi: https://pypi.org/pypi/bentoml/json (fetched 2026-08-29T08:25:23.915880+00:00, sha e41bfa635bfb)
  - site_page: https://www.bentoml.com/blog/neurolabs-faster-time-to-market-and-save-cost-with-bentoml (fetched 2026-08-29T08:25:23.914095+00:00, sha ee2030b43e29)
- Data as of 2026-08-30T08:39:29.467469+00:00.