bytedance/flux

Health v2 · maintenance only

Adoption not part of the score

1354 stars · 115 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Flux is a GPU kernel library from ByteDance that overlaps computation with communication for tensor and expert parallelism in dense and MoE model training and inference. It provides high-performance, pluggable CUDA kernels compatible with PyTorch across multiple Nvidia GPU architectures.

Use cases

• overlap allreduce communication with gemm computation during tensor parallel training
• speed up MoE expert parallelism by hiding dispatch/combine communication
• integrate fast communication-overlapping kernels into an existing PyTorch training framework
• reduce GPU idle time in multi-node LLM training
• train dense or mixture-of-experts models more efficiently on Hopper or Ampere GPUs

When to choose

• you are training or serving large dense or MoE models with tensor/expert parallelism on Nvidia GPUs
• you need pluggable PyTorch-compatible kernels to overlap communication with computation
• you want to squeeze more performance out of multi-GPU or multi-node training

When to avoid

• you are doing single-GPU workloads with no distributed communication to overlap
• you use non-Nvidia GPUs or unsupported CUDA architectures
• you need a full training framework rather than a kernel library

Facets

library · maturity active

gpu-computing machine-learning llm-training concurrency deep-learning large-language-models gpu-computing microservices machine-learning python cpp communication-overlapping tensor-parallelism expert-parallelism moe nvshmem cutlass pytorch-kernels gpu cuda linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="bytedance/flux")