Llmspeed launched as a web demo that shows token generation speeds for popular local LLMs on specific GPUs and CPUs. The project first appeared on Hacker News, where the thread reached 243 points and 77 comments.
Tool: Llmspeed | Purpose: Hardware speed preview | Source: llmspeed.dev | HN stats: 243 points, 77 comments
What Llmspeed Measures
The site runs pre-recorded inference traces for models such as Llama 3 8B and Mistral 7B. Users select a GPU model and batch size, then see tokens per second and estimated VRAM use without installing anything.
It focuses on consumer hardware profiles: RTX 3060 12 GB, RTX 4070 12 GB, and M2 MacBook Pro. Each profile lists measured latency and throughput numbers taken from actual runs.
How the Tool Works
Llmspeed replays benchmark logs collected on real systems. The interface shows a timeline of prompt processing and token generation phases.
No model weights are downloaded. The demo uses stored timing data to produce the speed readout in under two seconds.
Benchmarks Shown on the Site
The landing page lists concrete figures for common setups:
- Llama 3 8B on RTX 4070: 48 tokens/s at 4-bit quantization
- Mistral 7B on RTX 3060: 31 tokens/s
- Llama 3 8B on M2 MacBook Pro: 22 tokens/s
These numbers match reports from early testers in the Hacker News thread.
Pros and Cons
- Immediate feedback on hardware suitability before purchase
- No installation or model download required
- Covers both NVIDIA and Apple Silicon profiles
- Limited to pre-benchmarked models and hardware combinations
- Does not support custom prompts or fine-tuned checkpoints
- Relies on static logs rather than live runs
Alternatives and Comparisons
| Tool | Live Runs | Hardware Profiles | Install Needed | Link |
|---|---|---|---|---|
| Llmspeed | No | 8 listed | No | llmspeed.dev |
| Ollama | Yes | User hardware | Yes | ollama.com |
| LM Studio | Yes | User hardware | Yes | lmstudio.ai |
Ollama and LM Studio require local installation but run the actual model on the user's machine. Llmspeed trades live execution for instant preview.
Who Should Use Llmspeed
Developers evaluating a new GPU or laptop benefit most. Researchers planning hardware purchases can quickly rule out underpowered configurations.
Users who already own hardware and need live inference should skip directly to Ollama or LM Studio.
Verdict
Llmspeed fills the gap between marketing specs and real token throughput by giving concrete numbers for common consumer setups before any money is spent.
The tool's value lies in its zero-friction access to measured performance data rather than simulated estimates.
Top comments (0)