deepseek-ai/DualPipe

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

2998 stars · 333 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

DualPipe is a Python library implementing a bidirectional pipeline parallelism algorithm that overlaps forward and backward computation with communication to reduce pipeline bubbles in large model training. It also includes DualPipeV, a V-shaped schedule variant, and was introduced in the DeepSeek-V3 technical report.

Use cases

• reduce pipeline bubbles when training large language models
• overlap computation and communication in distributed PyTorch training
• implement bidirectional pipeline parallel schedules for MoE models
• compare pipeline parallelism strategies like 1F1B, ZB1P, and DualPipe
• train DeepSeek V3/R1 style models efficiently across many GPUs

When to choose

• you are training very large models with pipeline parallelism and communication is a bottleneck
• you use PyTorch 2.0+ and can implement a custom overlapped_forward_backward for your module
• you want to reproduce DeepSeek-style training efficiency

When to avoid

• you only train small models where pipeline bubbles are negligible
• you need a turnkey training framework rather than a scheduling algorithm
• your stack does not support PyTorch 2.0 or multi-GPU pipeline parallelism

Facets

library · maturity active

llm-training machine-learning gpu-computing large-language-models deep-learning microservices gpu-computing python pipeline-parallelism distributed-training pytorch computation-communication-overlap deepseek gpu linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="deepseek-ai/DualPipe")