learning-at-home/hivemind

Health v2 · maintenance only

Adoption not part of the score

2515 stars · 232 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-30, confidence not recorded

Hivemind is a PyTorch library for decentralized deep learning across the Internet, enabling training of large models on hundreds of volunteer computers without a master node. It provides a distributed hash table, fault-tolerant backpropagation, decentralized parameter averaging, and Decentralized Mixture-of-Experts for splitting large models across participants.

Use cases

• train a large neural network collaboratively across volunteers' computers
• run distributed deep learning without a central parameter server
• train a transformer model with a mixture-of-experts layer spread across machines
• fine-tune large language models in a peer-to-peer network like Petals
• collaboratively pretrain a language model with researchers worldwide
• tolerate slow or unresponsive nodes during distributed backpropagation

When to choose

• you want to train a large model across unreliable, geographically distributed machines without a master node
• you need fault-tolerant, asynchronous distributed training in PyTorch
• your model is too large for one GPU and can be sharded via mixture-of-experts
• you are building volunteer or collaborative training platforms

When to avoid

• you need classic tightly-coupled data-parallel training on a reliable cluster - use torch.distributed, DeepSpeed, or Horovod instead
• your workload is small enough for single-machine training
• you need strict synchronization or deterministic training runs
• you require production support or a large enterprise ecosystem

Facets

library · maturity active

deep-learning machine-learning llm-training concurrency networking deep-learning machine-learning microservices gpu-computing python cross-platform pytorch distributed-training decentralized dht mixture-of-experts volunteer-computing asyncio peer-to-peer gpu

Member repositories

For agents

markdown · JSON · MCP: product_card(name="learning-at-home/hivemind")