A 28.9M parameter LLM now runs directly on an $8 ESP32 microcontroller, per a recent Hacker News thread that linked to the working GitHub repo.
Model: 28.9M parameters | Hardware: ESP32 | Price: $8 | License: Open source (repo)
What It Is and How It Works
The project ports a compact transformer model to the ESP32's dual-core Xtensa CPU and limited SRAM. It uses 8-bit quantization and a custom inference loop that avoids external RAM.
Inference stays on-device with no cloud calls. The model handles short text prompts for classification or simple generation tasks.
Measured Performance Numbers
Early runs show inference times between 800 ms and 2.1 s per token on a standard ESP32 at 240 MHz. Memory footprint stays under 320 KB after quantization.
| Metric | Value | Notes |
|---|---|---|
| Parameters | 28.9M | Quantized to 8-bit |
| Inference time | 800 ms–2.1 s | Per token, 240 MHz clock |
| RAM usage | <320 KB | On-chip SRAM only |
| Power draw | ~80 mA active | 3.3 V supply |
| Cost per unit | $8 | ESP32 DevKit board |
How to Try It
Clone the repo and flash with ESP-IDF or Arduino IDE. The README supplies a single compile command and a minimal prompt example.
Flash the binary to any ESP32 DevKit, connect via USB, and send text over serial. No additional hardware is required beyond the board itself.
Pros and Cons
- Runs entirely offline on sub-$10 hardware.
- Total parts cost stays under $10 for a complete node.
- Model size limits output quality to short, constrained tasks.
- No support for longer context windows or fine-tuning on-device.
Alternatives and Comparisons
Other edge options include TinyLlama-1.1B on Raspberry Pi Zero 2 W or Phi-2 quantized on Coral USB. The ESP32 route wins on price and power but loses on capability.
| Platform | Model size | Cost | Inference speed | Power |
|---|---|---|---|---|
| ESP32 (this project) | 28.9M | $8 | 800 ms–2.1 s | 80 mA |
| Pi Zero 2 W | 1.1B | $15 | ~120 ms | 150 mA |
| Coral USB + Pi | 2.7B | $75 | ~40 ms | 500 mA |
Who Should Use This
Embedded developers building battery-powered sensors or offline text classifiers will find it useful. Skip it if you need coherent paragraph generation or multi-turn chat.
Researchers testing extreme quantization limits can use the repo as a baseline. Production teams needing reliable long outputs should look elsewhere.
Bottom Line / Verdict
The project proves a usable LLM can run on an $8 microcontroller with acceptable latency for narrow tasks.
This approach opens simple on-device language features for the lowest-cost hardware tier without external dependencies. Future work will likely focus on further quantization and task-specific fine-tunes.
Top comments (0)