# Run LLMs at Home BitTorrent-Style with Petals

> Published 2026-07-23 · https://www.promptzone.com/nadia_pham/run-llms-at-home-bittorrent-style-with-petals-3hca

**Petals** surfaced on Hacker News with 92 points and 29 comments for its approach to running large language models across volunteer GPUs in a decentralized swarm.

The system splits model layers across participants so no single machine needs the full parameter set in memory.

## What It Is / How It Works

Petals treats inference as a peer-to-peer task. Each node hosts one or more transformer blocks and forwards activations to the next peer in the chain.

Users join the swarm by running a lightweight client that discovers other nodes via a DHT, similar to BitTorrent trackers.

The project site at https://petals.dev/ provides the client and lists active public swarms for models such as Llama-2-70B and Falcon-180B.

## Benchmarks and Resource Numbers

Early swarm tests show 70B-class models generating tokens at 1–4 tokens per second when 8–12 consumer GPUs participate.

A single RTX 3090 node uses roughly 8–10 GB VRAM to host two layers of a 70B model.

Latency rises with network distance; participants on the same continent report median round-trip times under 80 ms.

## How to Try It

Install the client with `pip install petals` then run `python -m petals.cli.run_server --model bigscience/bloom-petals`.

The command automatically registers the node and begins serving layers.

For inference, connect via the Petals Python client or the provided Gradio demo at the project site.

## Pros and Cons

- Pros: Runs models larger than local VRAM allows; no central server costs; works on consumer hardware.
- Cons: Speed depends on peer availability and internet latency; public swarms can be unstable; no built-in fine-tuning support.

## Alternatives and Comparisons

| Feature              | Petals                  | Hugging Face TGI       | vLLM (single node)    |
|----------------------|-------------------------|------------------------|-----------------------|
| Max model size        | 100B+ (swarm)           | Limited by GPU memory  | Limited by GPU memory |
| Hardware requirement  | Multiple 8–12 GB GPUs   | One 24–80 GB GPU       | One 24–80 GB GPU      |
| Decentralized         | Yes                     | No                     | No                    |
| Tokens per second     | 1–4 (70B swarm)         | 20–60 (A100)           | 30–80 (A100)          |

Other options include Together.ai for paid API access and llama.cpp for fully local quantized runs.

## Who Should Use This

Researchers testing 70B+ models without access to multi-GPU servers benefit most. Developers needing consistent low-latency inference should skip it and use single-node tools instead.

> **Bottom line:** Petals fills the gap between fully local quantized models and expensive cloud clusters for occasional large-model experiments.

The project shows decentralized inference can move from theory to usable speeds when enough participants contribute GPUs.