deepseek-ai/DeepEP

Health v2 · maintenance only

Adoption not part of the score

10263 stars · 1466 forks observed · 2026-10-12

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

DeepEP is a high-performance GPU communication library for expert parallelism (EP) in MoE training and inference, providing high-throughput and low-latency all-to-all dispatch/combine kernels with FP8 support. It uses runtime JIT compilation with an NCCL Gin backend and experimental primitives for pipeline, context, and remote-memory parallelism with zero or minimal SM occupation.

Use cases

• dispatch tokens to experts across GPUs for MoE training
• combine expert outputs with low-latency all-to-all kernels
• run FP8 dispatching and BF16 combining for large-scale MoE models
• train mixture-of-experts models at EP2048 scale without auto-tuning
• overlap expert-parallel communication with minimal SM usage
• add pipeline or context parallelism primitives to a distributed training stack

When to choose

• you are training or serving MoE models and need efficient expert-parallel all-to-all communication
• you need near-hardware-bandwidth dispatch/combine on NVLink and RDMA (InfiniBand/RoCE) clusters
• you want to minimize SM occupation so GPUs stay free for compute
• you prefer runtime JIT kernels with no CUDA compilation step at install time
• you are building on Hopper (SM90) or newer (SM100) NVIDIA hardware with NCCL available

When to avoid

• your workload is single-GPU or uses only data/tensor parallelism with no expert parallelism
• you need the legacy 0-SM RDMA low-latency EP mode, which is no longer supported in V2
• you cannot accept the larger buffer memory footprint of V2
• you target non-NVIDIA GPUs or platforms without NCCL/RDMA support
• you need production-stable pipeline, context, or Engram remote-memory features, which are still experimental

Facets

library · maturity active

machine-learning llm-training llm-inference gpu-computing concurrency machine-learning deep-learning large-language-models gpu-computing microservices developer-tools python cpp expert-parallelism mixture-of-experts moe all-to-all-communication rdma nvlink nccl nvshmem jit-compilation fp8 low-precision pipeline-parallelism context-parallelism cuda-kernels high-performance-computing sm-occupation elastic-buffer deepseek linux gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="deepseek-ai/DeepEP")