vllm-project/vllm-omni

Health v2 · maintenance only

Adoption not part of the score

6369 stars · 1554 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

vLLM-Omni is a Python framework extending vLLM for efficient inference and serving of omni-modality models, including diffusion transformers, TTS, image/video generation, and robot-policy models. It provides pipelined stage execution, distributed parallelism, streaming outputs, and an OpenAI-compatible API server.

Use cases

• serve a text-to-image diffusion model behind an OpenAI-compatible API
• run offline batched inference with Qwen3-Omni or MiniCPM-o
• deploy TTS models like CosyVoice3 with streaming audio output
• serve video generation models like Wan2.2 or MiniMax H3
• host robot-policy and action models for robotics inference
• run full-duplex realtime voice serving with streaming audio input and output

When to choose

• you need high-throughput serving of multimodal, diffusion, TTS, or action models on GPU
• you want vLLM-style performance (KV cache, batching, parallelism) for non-autoregressive models
• you need an OpenAI-compatible server with streaming multimodal outputs

When to avoid

• you only need plain text LLM inference, where core vLLM suffices
• you need Windows or macOS support, since it targets Linux
• you want a lightweight single-model pipeline without distributed serving complexity

Facets

framework · maturity active

llm-inference machine-learning image-processing audio-processing video-processing tts speech-recognition http-server api-framework large-language-models machine-learning artificial-intelligence image-processing speech-processing robotics gpu-computing python cloud diffusion multimodal model-serving openai-compatible-api dit world-model robot-policy vllm audio video linux gpu docker

Member repositories

For agents

markdown · JSON · MCP: product_card(name="vllm-project/vllm-omni")