LLM VRAM Calculator: Can Your GPU Run It? (2026)
Check any local model against your GPU — or flip it around and see everything your card can run. VRAM math, best quantization, live cloud-rental prices when it doesn't fit, and the exact ollama command.
Model specs last verified July 16, 2026 — updated with every major model release. Cloud rental prices shown are live from provider APIs.
VRAM requirements for popular local LLMs (Q4_K_M, 8k context)
| Model | VRAM needed | Minimum GPU that fits it |
|---|---|---|
| Llama 3.2 3B | ~3.5 GB | Any 6 GB card |
| Qwen3 8B / Llama 3.1 8B | ~7 GB | RTX 3060 12GB / any 8 GB card |
| Gemma 3 12B | ~10 GB | RTX 3060 12GB |
| Phi-4 / Qwen3 14B | ~12 GB | RTX 4060 Ti 16GB |
| GPT-OSS 20B (MoE) | ~14 GB | RTX 4060 Ti 16GB |
| Mistral Small 3.2 (24B) | ~17 GB | RTX 3090 / 4090 24GB |
| Gemma 3 27B | ~19 GB | RTX 3090 / 4090 24GB |
| Qwen3 32B / R1 Distill 32B | ~23 GB | RTX 3090 / 4090 24GB |
| Llama 3.3 70B / R1 Distill 70B | ~45 GB | RTX 5090 32GB + offload, A6000 48GB, or 2× 24 GB |
| GPT-OSS 120B (MoE) | ~73 GB | Mac Studio 128GB or multi-GPU rig |
| Qwen3 235B-A22B (MoE) | ~145 GB | Mac Studio 192GB+ or server GPUs |
| DeepSeek R1 671B (MoE) | ~410 GB | Server cluster (or run a distill instead) |
Figures include model weights, 8k-token KV cache, and runtime overhead at Q4_K_M quantization.
How the math works
Model weights: parameters × bits-per-weight ÷ 8. A 8B model at Q4_K_M (~4.85 bits/weight) is about 5 GB; at FP16 it's 16 GB. KV cache grows linearly with context: roughly 2 × layers × KV-dimension × context-tokens × 2 bytes — about 1 GB for an 8B model at 8k, but 13 GB+ for a 70B at 32k. Runtime overhead adds ~1 GB. If weights don't fit in VRAM, llama.cpp and Ollama can offload layers to system RAM — it works, but expect single-digit tokens/sec.
On Apple Silicon, unified memory serves as VRAM — a 64 GB Mac gives roughly 48 GB usable for the model, which comfortably runs 70B-class models at Q4. With two identical GPUs, tensor-parallel runtimes give you close to double the usable VRAM (we count 190% to reflect real-world overhead).
FAQ
How much VRAM do I need to run a 70B model locally?
About 45 GB at Q4_K_M with 8k context — that means an RTX A6000 (48 GB), two 24 GB cards, a 64 GB Mac, or a 24-32 GB GPU with CPU offload at reduced speed.
What is the best local LLM for a 24 GB GPU (RTX 3090/4090)?
Qwen3 32B or DeepSeek R1 Distill 32B at Q4_K_M (~22 GB) are the strongest dense fits. Gemma 3 27B and Mistral Small 3.2 leave more headroom for longer context.
What is the best local LLM for 12 GB of VRAM?
Gemma 3 12B or Qwen3 8B at Q4/Q5 run fully on-GPU with room for context. Phi-4 (14B) fits at Q4 with a shorter context window.
Does quantization hurt quality?
Q5_K_M and Q4_K_M lose very little on most tasks and are the standard for local use. Below Q4 (Q3, IQ2) degradation becomes noticeable — prefer a smaller model at Q4 over a bigger one at Q2.
Why do MoE models like GPT-OSS 20B run fast on modest GPUs?
Mixture-of-experts models store all parameters (so VRAM needs stay high-ish) but only activate a few billion per token, so generation speed resembles a much smaller model.
Built by PromptZone. Model too big for your card? Check live cloud GPU prices. Setup guides: Local LLMs 2026 complete guide. Estimates only — actual usage varies by runtime and settings.
