apple/ml-4m

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

1808 stars · 114 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

4M is a framework from Apple and EPFL for training any-to-any multimodal foundation models using masked modeling over discrete tokens across tens of modalities. It includes official implementations, training code, tokenizers, and pre-trained models (4M-7, 4M-21) for versatile vision and multimodal generation tasks.

Use cases

• train any-to-any multimodal foundation models
• run a single vision model across tens of tasks and modalities
• generate images conditioned on text or other modalities
• perform multimodal image editing and in-painting
• fine-tune a generalist vision model for downstream tasks
• tokenize images, text, geometry, and semantics into discrete tokens
• download pretrained multimodal transformer checkpoints

When to choose

• you need one unified model handling many vision modalities and tasks
• you want to experiment with multimodal masked modeling research
• you need steerable multimodal generation or editing capabilities
• you want pretrained any-to-any vision models with open weights

When to avoid

• you need a lightweight single-task vision model
• you lack GPU resources for large transformer training or inference
• you need production-ready multimodal APIs rather than a research framework
• your use case is text-only NLP

Facets

library · maturity active

machine-learning deep-learning image-processing nlp data-science machine-learning computer-vision deep-learning artificial-intelligence image-processing python multimodal masked-modeling foundation-models transformer any-to-any tokenization pretrained-models vision-models generative-models pytorch gpu linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="apple/ml-4m")