# learning-at-home/hivemind

Decentralized deep learning in PyTorch. Built to train models on thousands of volunteers across the world.

Repository: https://github.com/learning-at-home/hivemind
Canonical: https://ross.abutalabs.com/products/hivemind
Language: Python
License: MIT
License Family: permissive
Topics: deep-learning, pytorch, volunteer-computing, mixture-of-experts, distributed-training, distributed-systems, asynchronous-programming, asyncio, dht, hivemind, neural-networks, machine-learning
Last push: 2026-01-11T21:13:45+00:00

## Health v2 (maintenance only)
Score: 59/100 (v2, computed 2026-09-03T02:20:16.233290+00:00)
- activity 61, release rhythm 32, longevity 100
- inputs: {"age_days": 2379, "days_push": 234, "days_rel": 242, "gap_med": 257, "n_releases_24m": 2}
- flags: none
- formula: round(0.45*activity + 0.35*rhythm + 0.20*longevity); archived -> min(score, 10)

## Adoption (not part of the score)
Stars 2515, forks 232 (observed 2026-08-28T04:06:57.658324+00:00)

## What it is
Hivemind is a PyTorch library for decentralized deep learning across the Internet, enabling training of large models on hundreds of volunteer computers without a master node. It provides a distributed hash table, fault-tolerant backpropagation, decentralized parameter averaging, and Decentralized Mixture-of-Experts for splitting large models across participants.

## Use cases
- train a large neural network collaboratively across volunteers' computers
- run distributed deep learning without a central parameter server
- train a transformer model with a mixture-of-experts layer spread across machines
- fine-tune large language models in a peer-to-peer network like Petals
- collaboratively pretrain a language model with researchers worldwide
- tolerate slow or unresponsive nodes during distributed backpropagation

## When to choose
- you want to train a large model across unreliable, geographically distributed machines without a master node
- you need fault-tolerant, asynchronous distributed training in PyTorch
- your model is too large for one GPU and can be sharded via mixture-of-experts
- you are building volunteer or collaborative training platforms

## When to avoid
- you need classic tightly-coupled data-parallel training on a reliable cluster - use torch.distributed, DeepSpeed, or Horovod instead
- your workload is small enough for single-machine training
- you need strict synchronization or deterministic training runs
- you require production support or a large enterprise ecosystem

## Facets
- artifact type: library
- maturity: active
- function: deep-learning, machine-learning, llm-training, concurrency, networking
- domain: deep-learning, machine-learning, microservices, gpu-computing
- platform: python, cross-platform
- tags: pytorch, distributed-training, decentralized, dht, mixture-of-experts, volunteer-computing, asyncio, peer-to-peer, gpu

## Member repositories
- learning-at-home/hivemind (main) score 59

## Provenance
- Observed fields: from GitHub, fetched 2026-08-28T04:06:57.658324+00:00.
- Health v2: computed from the inputs above; adoption is never an input.
- Inferred fields (summary, facets, guidance): AI-extracted, prompt v1, taxonomy v1, on 2026-08-30T02:26:12.164004+00:00, confidence not recorded.
  - readme: https://github.com/learning-at-home/hivemind (fetched 2026-08-28T04:06:57.658324+00:00, sha 60f3907e1707)
  - registry_pypi: https://pypi.org/pypi/hivemind/json (fetched 2026-08-29T10:08:29.069748+00:00, sha d86f3c8081ed)
- Data as of 2026-08-30T08:39:29.467469+00:00.