# GateGPT FPGA Hits 56k Tokens per Second

> Published 2026-06-16 · https://www.promptzone.com/seojun_sullivan/gategpt-fpga-hits-56k-tokens-per-second-23bp

GateGPT implements a Transformer with KV cache that reaches **56k tokens per second** on an FPGA running at **80 MHz**, according to discussion flagged on Hacker News.

The result was shared via [a recent Hacker News thread](https://twitter.com/fguzmanai/status/2065832668172845209) that drew 27 points and 8 comments.

> **Model:** GateGPT | **Speed:** 56k tokens/s | **Hardware:** FPGA | **Clock:** 80 MHz

## What GateGPT Delivers on FPGA Hardware

GateGPT maps the attention KV cache directly onto FPGA fabric. The design keeps matrix operations and memory access on-chip at a fixed 80 MHz clock.

This removes the need for high-frequency GPU schedulers while sustaining high token throughput for inference workloads that reuse cached keys and values.


![GateGPT FPGA Hits 56k Tokens per Second](https://www.jeffgeerling.com/blog/2024/testing-raspberry-pis-ai-kit-13-tops-70/pi-ai-kit.jpeg)

## Key Performance Metrics

The reported figure of **56k tokens per second** comes from a single FPGA implementation. No batch-size or sequence-length details were provided in the source post.

Early HN comments noted the low clock rate relative to typical GPU boost clocks above 1 GHz.

## Tradeoffs of FPGA Inference

- Fixed 80 MHz clock limits peak frequency but simplifies power and cooling requirements.
- KV-cache mapping reduces external memory traffic compared with standard GPU attention kernels.
- Reconfiguring the FPGA for new model sizes requires synthesis time that GPUs avoid.

## Comparing FPGA to GPU Alternatives

Standard GPU inference engines such as vLLM or TensorRT-LLM target consumer and data-center cards. The table below places the reported GateGPT numbers against typical published throughputs for similar KV-cache workloads.

| Hardware       | Tokens/s (KV cache) | Clock     | Notes                     |
|----------------|---------------------|-----------|---------------------------|
| GateGPT FPGA   | 56,000             | 80 MHz    | Single reported result    |
| A100 (FP16)    | 8,000–15,000       | ~1.4 GHz  | vLLM typical figures      |
| RTX 4090       | 4,000–9,000        | ~2.5 GHz  | TensorRT-LLM batch=1      |

GateGPT shows higher tokens per clock cycle than the GPU baselines listed.

## Who Should Use This Approach

Teams building fixed-function inference pipelines with stable model architectures can benefit. Researchers exploring low-power or deterministic latency setups may also test the design.

Users who need frequent model updates or large-batch training should continue with GPU frameworks.

## Final Assessment

GateGPT demonstrates that an 80 MHz FPGA can exceed common GPU token rates for KV-cache inference when the architecture stays simple and on-chip.

> **Bottom line:** The result highlights a viable path for high-throughput inference on modest FPGA hardware when model changes remain infrequent.