facebookresearch/multimodal

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

1732 stars · 174 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

TorchMultimodal is a PyTorch library from Meta for training state-of-the-art multimodal multi-task models at scale, covering both content understanding and generative models. It provides modular building blocks (fusion layers, losses, datasets), pretrained model classes like CLIP, FLAVA, BLIP-2, and DALL-E 2, and example scripts replicating published research baselines.

Use cases

• train a multimodal vision-language model at scale
• fine-tune CLIP or FLAVA on my own dataset
• replicate BLIP-2 or CoCa research baselines in PyTorch
• build a text-to-image diffusion model like DALL-E 2
• get pretrained weights for multimodal models
• train an audio MAE model
• compose custom fusion layers and losses for multi-task learning

When to choose

• you want PyTorch-native implementations of state-of-the-art multimodal models with pretrained weights
• you need modular, composable building blocks for multimodal research
• you want scalable training baselines for vision-language or generative multimodal tasks

When to avoid

• you need a production inference API rather than a research training library
• you want a framework-agnostic or non-PyTorch solution
• you need a stable, fully supported product rather than a beta research library

Facets

library · maturity active

machine-learning deep-learning llm-training machine-learning deep-learning artificial-intelligence computer-vision python pytorch multimodal vision-language diffusion-models pretrained-models research natural-language-processing gpu linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="facebookresearch/multimodal")