KellerJordan/Muon

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

2801 stars · 129 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Muon is a PyTorch optimizer for the hidden layers of neural networks, based on orthogonalized momentum updates via Newton-Schulz iteration. It is designed to be combined with AdamW for embeddings, heads, and biases, and has set training speed records on GPT-2 and CIFAR-10 benchmarks.

Use cases

• train transformers faster than with AdamW
• optimize hidden layer weights of a neural network
• speed up GPT-2 style LLM pretraining on a budget
• set CIFAR-10 training speed records
• replace AdamW for hidden weights in a PyTorch model
• scale learning rate across model sizes with muP-style scaling

When to choose

• you are training transformers or convnets in PyTorch and want faster convergence per FLOP
• you want a drop-in optimizer for hidden weights alongside AdamW
• you are running training speedrun experiments like modded-nanogpt

When to avoid

• you need a battle-tested general-purpose optimizer for all parameter types
• you are not using PyTorch
• you need embeddings, heads, and biases optimized by the same optimizer

Facets

library · maturity active

machine-learning llm-training deep-learning machine-learning deep-learning large-language-models python optimizer pytorch newton-schulz training-speedrun muon

Member repositories

For agents

markdown · JSON · MCP: product_card(name="KellerJordan/Muon")