MoonshotAI/Moonlight

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

1546 stars · 103 forks observed · 2026-09-22

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Moonlight is a 3B/16B Mixture-of-Experts LLM trained with the Muon optimizer, released by Moonshot AI along with a memory- and communication-efficient distributed Muon implementation. It includes pretrained, instruction-tuned, and intermediate checkpoints plus scaling law experiments showing ~2x compute efficiency over AdamW.

Use cases

• train large language models with the Muon optimizer
• run distributed Muon training with ZeRO-1 memory optimization
• compare Muon vs AdamW scaling law experiments
• fine-tune or study an open MoE model checkpoint
• reproduce compute-optimal LLM training research

When to choose

• you want to experiment with Muon for large-scale LLM training
• you need an efficient distributed optimizer implementation for research
• you want open MoE model checkpoints for study or fine-tuning

When to avoid

• you need a production-ready training framework with broad ecosystem support
• you lack multi-GPU infrastructure for large model training
• you only need inference rather than training

Facets

library · maturity active

llm-training machine-learning deep-learning large-language-models deep-learning machine-learning python muon-optimizer mixture-of-experts scaling-laws distributed-training model-checkpoints gpu linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="MoonshotAI/Moonlight")