volcengine/veScale

Health v2 · maintenance only

Flags: no_releases

Adoption not part of the score

1036 stars · 64 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

veScale is a PyTorch distributed training library from ByteDance for hyperscale training of large language models and reinforcement learning workloads. It provides eager-mode SPMD tensor programming and a flexible, high-performance FSDP implementation.

Use cases

• train large language models across many GPUs
• run distributed FSDP training at scale
• train RL models with PyTorch distributed
• shard tensors with ragged sharding in DTensor
• scale PyTorch training to hyperscale clusters

When to choose

• you need a PyTorch-native distributed LLM training library
• you want a flexible high-performance alternative to stock FSDP
• you train LLMs or RL workloads on large GPU clusters

When to avoid

• you need a stable, feature-complete framework today (the repo is mid-migration with old code moved to legacy/)
• you train small models on a single GPU
• you need a framework-agnostic solution outside PyTorch

Facets

library · maturity experimental

llm-training machine-learning deep-learning machine-learning deep-learning large-language-models gpu-computing python pytorch distributed-training fsdp reinforcement-learning spmd dtensor gpu linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="volcengine/veScale")