VITA-MLLM/VITA

Health v2 · maintenance only

Flags: no_releases no_license

Adoption not part of the score

2534 stars · 182 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

VITA is an open-source interactive omni multimodal large language model (VITA-1.5) that supports real-time vision and speech interaction, similar to GPT-4o. It provides training code, inference scripts, and demos for English and Chinese multimodal (image, video, audio) understanding and conversation.

Use cases

• build a real-time voice and vision assistant like GPT-4o
• run an open-source omni multimodal LLM locally
• understand and chat about videos and images with speech input
• fine-tune a multimodal LLM on custom data
• evaluate a multimodal model on MLLM benchmarks like Video-MME
• demo real-time interactive AI conversation with camera and microphone

When to choose

• you need an open-source GPT-4o-style real-time multimodal assistant
• you want to train or fine-tune an omni-modal LLM with vision and speech
• you need video understanding combined with voice interaction in English or Chinese

When to avoid

• you only need text-only LLM inference
• you lack GPU resources for large multimodal models
• you need a production-ready commercial product with licensing guarantees (license is custom/unclear)

Facets

library · maturity active

machine-learning deep-learning llm-inference speech-recognition computer-vision chatbot large-language-models artificial-intelligence computer-vision speech-processing python cross-platform multimodal omni-modal vision-language-model real-time-interaction video-understanding speech-interaction neurips-2025 gpt-4o-alternative natural-language-processing video gpu linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="VITA-MLLM/VITA")