facebookresearch/schedule_free

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

2323 stars · 78 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

A PyTorch library implementing schedule-free optimizers (SGD, AdamW, RAdam variants) that remove the need for learning rate schedules or specifying training length in advance. Based on the 'The Road Less Scheduled' paper, it replaces momentum with interpolation and averaging.

Use cases

• train deep learning models without tuning learning rate decay schedules
• avoid specifying total training steps in advance
• replace cosine decay schedules in PyTorch training loops
• train LLMs with schedule-free AdamW
• get better test loss than scheduled optimizers at any stopping point

When to choose

• you use PyTorch and want to eliminate learning rate schedule tuning
• you don't know the total number of training steps ahead of time
• you want to match or beat cosine/linear decay schedules without scheduling

When to avoid

• you need a Jax implementation (use Optax's version instead)
• you rely on optimizers incompatible with the train/eval mode switching pattern
• you need minimal memory overhead beyond the base optimizer with the reference variants

Facets

library · maturity active

machine-learning llm-training machine-learning deep-learning large-language-models python pytorch-optimizers schedule-free-learning learning-rate-schedule adamw sgd research

Member repositories

For agents

markdown · JSON · MCP: product_card(name="facebookresearch/schedule_free")