bigscience-workshop/petals

Health v2 · maintenance only

Adoption not part of the score

10521 stars · 642 forks observed · 2026-08-28

What it is AI-extracted, prompt v1, taxonomy v1, 2026-08-29, confidence not recorded

Petals is a Python library that lets you run and fine-tune large language models (Llama 3.1, Mixtral, Falcon, BLOOM) on a BitTorrent-style distributed network of volunteer GPUs. It integrates with Hugging Face Transformers so models load and generate as if they were local, while layers are served across a community swarm.

Use cases

• run 100B+ parameter LLMs without a high-end GPU
• generate text with Llama 405B from a laptop or Google Colab
• fine-tune large language models on consumer hardware
• build a chatbot backed by a distributed LLM
• contribute idle GPU capacity to a community model-serving swarm
• experiment with model hidden states and custom inference paths

When to choose

• you want to run or fine-tune very large models without expensive hardware
• you need PyTorch/Transformers flexibility rather than a fixed hosted API
• you want to donate GPU resources to a public inference network
• you're doing research on distributed inference or model parallelism

When to avoid

• you need guaranteed latency or throughput for production workloads
• you handle sensitive data that can't leave your machine (unless you set up a private swarm)
• you need a fully self-contained offline deployment
• you require strict SLAs or enterprise support

Facets

library · maturity active

llm-inference llm-training machine-learning deep-learning nlp large-language-models machine-learning deep-learning microservices python distributed-inference volunteer-computing bittorrent-style transformers pytorch fine-tuning swarm bloom llama mixtral falcon natural-language-processing linux macos gpu docker

Member repositories

For agents

markdown · JSON · MCP: product_card(name="bigscience-workshop/petals")