# Can You Run a 28.9M LLM on an $8 ESP32?

> Published 2026-07-26 · https://www.promptzone.com/diego_banerjee/can-you-run-a-289m-llm-on-an-8-esp32-53h4

A 28.9M parameter LLM now runs directly on an **$8 ESP32** microcontroller, per [a recent Hacker News thread](https://github.com/slvDev/esp32-ai) that linked to the working GitHub repo.

> **Model:** 28.9M parameters | **Hardware:** ESP32 | **Price:** $8 | **License:** Open source (repo)

## What It Is and How It Works

The project ports a compact transformer model to the ESP32's dual-core Xtensa CPU and limited SRAM. It uses 8-bit quantization and a custom inference loop that avoids external RAM.

Inference stays on-device with no cloud calls. The model handles short text prompts for classification or simple generation tasks.

## Measured Performance Numbers

Early runs show inference times between 800 ms and 2.1 s per token on a standard ESP32 at 240 MHz. Memory footprint stays under 320 KB after quantization.

| Metric              | Value          | Notes                     |
|---------------------|----------------|---------------------------|
| Parameters          | 28.9M          | Quantized to 8-bit        |
| Inference time      | 800 ms–2.1 s   | Per token, 240 MHz clock  |
| RAM usage           | <320 KB        | On-chip SRAM only         |
| Power draw          | ~80 mA active  | 3.3 V supply              |
| Cost per unit       | $8             | ESP32 DevKit board        |

## How to Try It

Clone the repo and flash with ESP-IDF or Arduino IDE. The README supplies a single compile command and a minimal prompt example.

Flash the binary to any ESP32 DevKit, connect via USB, and send text over serial. No additional hardware is required beyond the board itself.

## Pros and Cons

- Runs entirely offline on sub-$10 hardware.
- Total parts cost stays under $10 for a complete node.
- Model size limits output quality to short, constrained tasks.
- No support for longer context windows or fine-tuning on-device.

## Alternatives and Comparisons

Other edge options include **TinyLlama-1.1B** on Raspberry Pi Zero 2 W or **Phi-2** quantized on Coral USB. The ESP32 route wins on price and power but loses on capability.

| Platform              | Model size | Cost | Inference speed | Power |
|-----------------------|------------|------|-----------------|-------|
| ESP32 (this project)  | 28.9M      | $8   | 800 ms–2.1 s    | 80 mA |
| Pi Zero 2 W           | 1.1B       | $15  | ~120 ms         | 150 mA|
| Coral USB + Pi        | 2.7B       | $75  | ~40 ms          | 500 mA|

## Who Should Use This

Embedded developers building battery-powered sensors or offline text classifiers will find it useful. Skip it if you need coherent paragraph generation or multi-turn chat.

Researchers testing extreme quantization limits can use the repo as a baseline. Production teams needing reliable long outputs should look elsewhere.

## Bottom Line / Verdict

The project proves a usable LLM can run on an $8 microcontroller with acceptable latency for narrow tasks.

This approach opens simple on-device language features for the lowest-cost hardware tier without external dependencies. Future work will likely focus on further quantization and task-specific fine-tunes.