microsoft/mup

Health v2 · maintenance only

Adoption not part of the score

1753 stars · 105 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

The `mup` Python package implements Maximal Update Parametrization (μP) for PyTorch models, enabling optimal hyperparameters to remain stable across model widths. It supports μTransfer: tuning hyperparameters on a small proxy model and zero-shot transferring them to much larger models.

Use cases

• tune hyperparameters on a small model and transfer them to a large transformer
• implement maximal update parametrization in PyTorch models
• reduce hyperparameter tuning cost for large neural networks
• run coord checks to verify parametrization correctness
• scale up MLPs or transformers without retuning learning rates

When to choose

• you are pretraining large transformers or other wide neural networks and want cheap hyperparameter tuning
• you want to reduce fragility when scaling models from exploration to production size
• you use PyTorch and can modify your model's readout layer and optimizers

When to avoid

• you train small models where hyperparameter tuning is already cheap
• you cannot modify your model architecture to use μP layers
• you use frameworks other than PyTorch

Facets

library · maturity stable

machine-learning deep-learning llm-training machine-learning deep-learning large-language-models python pytorch mup mutransfer hyperparameter-tuning transformers parametrization

Member repositories

For agents

markdown · JSON · MCP: product_card(name="microsoft/mup")