ByteDance-Seed/Triton-distributed

Health v2 · maintenance only

Flags: prerelease_only

Adoption not part of the score

1559 stars · 176 forks observed · 2026-10-09

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Triton-distributed is a distributed compiler built on OpenAI Triton for computation-communication overlapping on multi-GPU systems. It lets developers write high-performance distributed kernels comparable to hand-optimized libraries like Distributed-GEMM and FLUX, targeting Nvidia and AMD GPUs.

Use cases

• write distributed GPU kernels with computation-communication overlap
• implement expert parallelism for MoE models
• build low-latency all-to-all communication kernels
• optimize GEMM operations across multiple GPUs
• profile intra-kernel performance on GPUs
• port distributed kernel compilation to custom hardware

When to choose

• you need custom distributed kernels beyond what standard Triton offers
• you are training or serving large models with expert parallelism
• you want Python-level kernel development instead of CUDA/CUTLASS
• you need computation and communication to overlap in a single kernel

When to avoid

• you only need single-GPU kernels (plain Triton suffices)
• you want a turnkey distributed training framework rather than a kernel compiler
• you target CPUs or non-GPU accelerators without porting effort

Facets

library · maturity active

compiler gpu-computing machine-learning llm-training benchmarking gpu-computing microservices deep-learning large-language-models developer-tools python triton distributed-kernels computation-communication-overlap moe expert-parallelism nvidia-gpu amd-gpu gpu linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="ByteDance-Seed/Triton-distributed")