# NousResearch/DisTrO

Distributed Training Over-The-Internet

Repository: https://github.com/NousResearch/DisTrO
Canonical: https://ross.abutalabs.com/products/distro
License Family: other
Last push: 2025-10-14T15:37:16+00:00

## Health v2 (maintenance only)
Score: 44/100 (v2, computed 2026-09-02T17:46:02.011165+00:00)
- activity 47, release rhythm 35, longevity 52
- inputs: {"age_days": 737, "days_push": 323, "days_rel": null, "gap_med": null, "n_releases_24m": 0}
- flags: no_releases, no_license
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 1058, forks 59 (observed 2026-08-28T04:03:25.168749+00:00)

## What it is
DisTrO is a family of low-latency distributed optimizers that reduce inter-GPU communication requirements by three to four orders of magnitude, enabling large model training over the internet. It grew out of the DeMo optimization research and has been used to train models like a 15B and a 40B LLM.

## Use cases
- train large language models across geographically distributed GPUs
- reduce inter-GPU communication bandwidth for distributed training
- run collaborative LLM training over the internet
- train models without a high-speed interconnect cluster
- research distributed optimization algorithms

## When to choose
- you want to train large models on GPUs not connected by fast interconnects
- network bandwidth is the bottleneck in your distributed training
- you want to participate in decentralized community training runs

## When to avoid
- you need a battle-tested, production-supported training framework with vendor support
- your training fits on a single machine or standard cluster with InfiniBand
- you need a permissive license - the repository has no license file

## Facets
- artifact type: library
- maturity: active
- function: llm-training, machine-learning, rpc
- domain: large-language-models, deep-learning, microservices, machine-learning
- platform: python, rust
- tags: distributed-optimizers, decentralized-training, low-latency-communication, research, gpu, linux

## Member repositories
- NousResearch/DisTrO (main) score 44

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:03:25.168749+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T06:57:32.709679+00:00, confidence not recorded.
  - readme: https://github.com/NousResearch/DisTrO (fetched 2026-08-28T04:03:25.168749+00:00, sha 73b8bd40f8bd)
- Data as of 2026-08-30T08:39:29.467469+00:00.