NousResearch/DisTrO

Health v2 · maintenance only

Flags: no_releases no_license

Adoption not part of the score

1058 stars · 59 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

DisTrO is a family of low-latency distributed optimizers that reduce inter-GPU communication requirements by three to four orders of magnitude, enabling large model training over the internet. It grew out of the DeMo optimization research and has been used to train models like a 15B and a 40B LLM.

Use cases

• train large language models across geographically distributed GPUs
• reduce inter-GPU communication bandwidth for distributed training
• run collaborative LLM training over the internet
• train models without a high-speed interconnect cluster
• research distributed optimization algorithms

When to choose

• you want to train large models on GPUs not connected by fast interconnects
• network bandwidth is the bottleneck in your distributed training
• you want to participate in decentralized community training runs

When to avoid

• you need a battle-tested, production-supported training framework with vendor support
• your training fits on a single machine or standard cluster with InfiniBand
• you need a permissive license - the repository has no license file

Facets

library · maturity active

llm-training machine-learning rpc large-language-models deep-learning microservices machine-learning python rust distributed-optimizers decentralized-training low-latency-communication research gpu linux

Member repositories

For agents

markdown · JSON · MCP: product_card(name="NousResearch/DisTrO")