awslabs/multi-model-server

Health v2 · maintenance only

Flags: archived

Adoption not part of the score

1024 stars · 230 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Multi Model Server (MMS) is a tool for serving deep learning model inference over HTTP endpoints, supporting models from any ML/DL framework including MXNet and ONNX. It provides a server CLI and pre-configured Docker images to deploy models as inference services.

Use cases

• serve deep learning models for inference over http
• deploy neural network models as a rest api
• host onnx model inference server
• serve mxnet models in production
• package and serve pytorch or mxnet model archives
• run a multi-model inference server in docker

When to choose

• you need HTTP endpoints for model inference requests
• you want to serve models from multiple DL frameworks
• you want pre-configured Docker images for model serving
• you are on Linux or macOS with Java 8 and Python available

When to avoid

• you need a modern, actively developed serving stack (consider TorchServe, its successor)
• you need first-class Windows support
• you want GPU-optimized LLM serving with batching and quantization
• you need a framework-free client-side inference library

Facets

service · maturity maintenance

http-server machine-learning llm-inference deep-learning machine-learning apis self-hosted python jvm cross-platform model-serving inference-server mxnet onnx deep-learning-serving http-endpoints linux macos docker

Member repositories

For agents

markdown · JSON · MCP: product_card(name="awslabs/multi-model-server")