MoonshotAI/checkpoint-engine

Health v2 · maintenance only

Adoption not part of the score

1005 stars · 106 forks observed · 2026-09-03

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Checkpoint-engine is a lightweight Python middleware for updating model weights in-place across LLM inference engines, a critical step in reinforcement learning workflows. It provides efficient Broadcast and P2P weight transfer implementations, updating a 1-trillion-parameter model across thousands of GPUs in about 20 seconds.

Use cases

• update LLM weights in inference engines during RL training
• synchronize model weights across thousands of GPUs
• push new weights to dynamically added inference instances
• broadcast sharded model weights from CPU memory to inference clusters
• integrate weight syncing between training and serving for RLHF

When to choose

• you run reinforcement learning loops that need to refresh inference engine weights in-place
• you serve very large models (e.g., trillion-parameter MoE) across many GPUs and need fast weight sync
• you need to update weights on inference instances added mid-serving without disrupting existing workloads

When to avoid

• you only need to save or load checkpoints to disk rather than live weight updates
• your inference engine is not supported/integrated with checkpoint-engine
• you need a full RL training framework rather than a weight-update middleware

Facets

library · maturity active

middleware llm-inference llm-training reinforcement-learning gpu-computing large-language-models machine-learning reinforcement-learning microservices developer-tools python weight-update parameter-server rlhf model-serving broadcast p2p-transfer cuda-ipc zeromq gpu linux docker

Member repositories

For agents

markdown · JSON · MCP: product_card(name="MoonshotAI/checkpoint-engine")