HITsz-TMG/Uni-MoE

Health v2 · maintenance only

Flags: no_releases no_license

Adoption not part of the score

1116 stars · 71 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Uni-MoE is a family of open-source Mixture-of-Experts (MoE) based omnimodal large language models that understand and generate across text, images, speech, audio, and video. The repository provides model weights, training code, and evaluation integration (e.g., LMMs-Eval) for versions including Uni-MoE-2.0-Omni built on Qwen2.5-7B.

Use cases

• run an omnimodal LLM that understands images, speech, and video
• generate speech, images, and text from a single unified model
• fine-tune a MoE multimodal model on custom data
• evaluate a multimodal LLM with lmms-eval
• research mixture-of-experts architectures for multimodal learning
• convert speech to text and text to speech with one model

When to choose

• you need a single open model handling cross-modal understanding and generation
• you want to study or extend MoE-based multimodal architectures
• you need audio generation unifying speech and music

When to avoid

• you need a lightweight model for CPU-only or edge deployment
• you need a commercially licensed model (no license specified)
• you only need text-only LLM inference with minimal setup

Facets

library · maturity active

machine-learning deep-learning llm-training speech-recognition tts image-processing video-processing audio-processing large-language-models deep-learning artificial-intelligence speech-processing python mixture-of-experts multimodal omnimodal model-weights research natural-language-processing gpu linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="HITsz-TMG/Uni-MoE")