bytedance/byteps

Health v2 · maintenance only

Flags: archived no_license

Adoption not part of the score

3717 stars · 491 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

BytePS is a high-performance parameter server framework for distributed deep neural network training, supporting TensorFlow, Keras, PyTorch, and MXNet over TCP or RDMA networks. It achieves significantly better scaling efficiency than Horovod+NCCL in many scenarios, such as ~90% scaling efficiency on BERT-large with 256 GPUs.

Use cases

• train BERT-large across hundreds of GPUs with high scaling efficiency
• speed up distributed deep learning training beyond Horovod+NCCL
• run distributed training over RDMA networks
• add gradient compression to distributed training
• train TensorFlow, PyTorch, Keras, or MXNet models on a cluster

When to choose

• you need maximum multi-GPU/multi-node training throughput on Ethernet or RDMA
• you train large models like BERT at scale and Horovod is a bottleneck
• you want a cross-framework parameter server for TensorFlow, PyTorch, or MXNet

When to avoid

• you need a framework with frequent updates and broad community support
• your training is small-scale and single-node
• you require the latest framework features or newest PyTorch/TensorFlow versions

Facets

framework · maturity maintenance

machine-learning deep-learning llm-training machine-learning deep-learning gpu-computing microservices python cloud parameter-server distributed-training rdma horovod-alternative gradient-compression linux gpu docker

Member repositories

For agents

markdown · JSON · MCP: product_card(name="bytedance/byteps")