thu-ml/Motus

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

1246 stars · 73 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Motus is the official implementation of a unified latent action world model for robotics, combining a video generation model, a vision-language model, and action/understanding experts via a Mixture-of-Transformers architecture. It supports multiple modeling modes including world modeling, vision-language-action prediction, inverse dynamics, and video-action joint prediction, with pretrained checkpoints and training/inference code.

Use cases

• train a vision-language-action model for robotic manipulation
• generate future video predictions conditioned on robot actions
• pretrain robot policies on large-scale unlabeled video using latent actions
• run inverse dynamics model inference to extract actions from video
• evaluate multi-task robot policies in RoboTwin 2.0 simulation
• experiment with unified diffusion-based world models

When to choose

• you need a unified world model and VLA in one codebase
• you want to leverage pretrained video and vision-language backbones for robotics
• you need cross-embodiment action pretraining from raw video via optical-flow latent actions

When to avoid

• you need a lightweight real-time robot controller rather than a research model
• you lack GPU resources for an ~8B parameter diffusion model
• you need a production-ready deployment stack rather than research code

Facets

library · maturity active

machine-learning deep-learning video-processing simulation llm-inference robotics artificial-intelligence machine-learning python world-model vision-language-action diffusion-model latent-actions robotic-manipulation optical-flow mixture-of-transformers video-generation video gpu linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="thu-ml/Motus")