# awslabs/multi-model-server

Multi Model Server is a tool for serving neural net models for inference

Repository: https://github.com/awslabs/multi-model-server
Canonical: https://ross.abutalabs.com/products/multi-model-server
Language: Java
License: Apache-2.0
License Family: permissive
Topics: mxnet, deep-learning, inference, ai, neural-network, onnx, server
Archived: true
Last push: 2024-05-20T19:29:13+00:00

## Health v2 (maintenance only)
Score: 10/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 0, release rhythm 8, longevity 100
- inputs: {"age_days": 3255, "days_push": 835, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: archived
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 1024, forks 230 (observed 2026-08-28T04:03:16.357786+00:00)

## What it is
Multi Model Server (MMS) is a tool for serving deep learning model inference over HTTP endpoints, supporting models from any ML/DL framework including MXNet and ONNX. It provides a server CLI and pre-configured Docker images to deploy models as inference services.

## Use cases
- serve deep learning models for inference over http
- deploy neural network models as a rest api
- host onnx model inference server
- serve mxnet models in production
- package and serve pytorch or mxnet model archives
- run a multi-model inference server in docker

## When to choose
- you need HTTP endpoints for model inference requests
- you want to serve models from multiple DL frameworks
- you want pre-configured Docker images for model serving
- you are on Linux or macOS with Java 8 and Python available

## When to avoid
- you need a modern, actively developed serving stack (consider TorchServe, its successor)
- you need first-class Windows support
- you want GPU-optimized LLM serving with batching and quantization
- you need a framework-free client-side inference library

## Facets
- artifact type: service
- maturity: maintenance
- function: http-server, machine-learning, llm-inference
- domain: deep-learning, machine-learning, apis, self-hosted
- platform: python, jvm, cross-platform
- tags: model-serving, inference-server, mxnet, onnx, deep-learning-serving, http-endpoints, linux, macos, docker

## Member repositories
- awslabs/multi-model-server (main) score 10

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:03:16.357786+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T07:08:08.067834+00:00, confidence not recorded.
  - readme: https://github.com/awslabs/multi-model-server (fetched 2026-08-28T04:03:16.357786+00:00, sha 223d4d4ac3cc)
- Data as of 2026-08-30T08:39:29.467469+00:00.