HumanMLLM/R1-Omni

Health v2 · maintenance only

Flags: no_releases no_license

Adoption not part of the score

1022 stars · 73 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

R1-Omni is a research project applying Reinforcement Learning with Verifiable Reward (RLVR) to an omni-multimodal large language model for explainable emotion recognition from video and audio. It releases model weights (HumanOmni-0.5B variants) and training/inference code built on the HumanOmni base model.

Use cases

• recognize emotions from video with audio and visual cues
• train a multimodal LLM with reinforcement learning and verifiable rewards
• get explainable reasoning for emotion recognition predictions
• fine-tune an omni-modal model on emotion datasets like MAFW and DFEW
• evaluate emotion recognition generalization on out-of-distribution data
• run inference with a 0.5B multimodal emotion model

When to choose

• you need multimodal (audio+visual) emotion recognition with interpretable reasoning
• you want to experiment with RLVR training on multimodal LLMs
• you need open model weights for emotion recognition research

When to avoid

• you need a production-ready, well-documented pipeline (setup and reproduction docs are incomplete)
• you need single-modality (video-only or audio-only) inference, which is not yet supported
• you require a permissive license - no license is specified

Facets

library · maturity experimental

machine-learning deep-learning llm-training speech-recognition video-processing nlp artificial-intelligence large-language-models machine-learning python emotion-recognition multimodal reinforcement-learning rlvr omni-model research audio video gpu linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="HumanMLLM/R1-Omni")