# How much VRAM do you need for 7B, 14B, and 32B models? > VRAM needed for 7B, 14B, and 32B LLMs – weights, quantization, KV cache, and context length, with a quick-reference table. *Source: https://velstech.net/how-much-vram-for-llm · Updated: 2026-08-27 · Category: AI · Tags: VRAM, LLM, GPU, Quantization, VelsTech Lab* *Markdown version of [How much VRAM do you need for 7B, 14B, and 32B models?](https://velstech.net/how-much-vram-for-llm). [Read the full guide with interactive tools](https://velstech.net/how-much-vram-for-llm).* *Also as Markdown: [Hindi](https://velstech.net/how-much-vram-for-llm.hi.md) · [Tamil](https://velstech.net/how-much-vram-for-llm.ta.md).* --- The single most common question in local AI is "can my GPU run this model?" The answer comes down to one number: **VRAM**. Here's how to estimate it yourself in about thirty seconds – and a cheat sheet for the popular model sizes. 🔧 Skip the math – use the calculator Pick your model, quantization, and context to get an exact VRAM estimate with a fit check for your GPU. [Open the LLM VRAM Calculator →](https://velstech.net/llm-vram-calculator) ## The quick answer VRAM needed for the model alone (weights), plus ~1–3 GB for KV cache + overhead at 8K context: ``` Model size Q4_K_M Q8_0 FP16 ────────────────────────────────────────────── 7–8B ~4–5 GB ~8–9 GB ~15–16 GB 13–14B ~9–10 GB ~16–17 GB ~30 GB 27–32B ~18–21 GB ~32–35 GB ~60+ GB 70B ~40–45 GB ~75 GB cloud only ``` Translate that to GPUs: an **8 GB** card runs 7B Q4. A **16 GB** card is the sweet spot – 14B Q4 fully, 32B with light offload. A **24 GB** card runs 32B Q4 fully. That's why "16 GB" keeps showing up in our [best GPU for local LLMs](https://velstech.net/best-gpu-for-local-llm) picks. ## What actually eats VRAM Three things, and you need to add all of them: - Model weights – the big one. Roughly parameters × bytes per weight. At Q4_K_M that's ~0.6 bytes/weight; at Q8_0 ~1.06; at FP16 exactly 2. - KV cache – grows with context length and batch size. Small at 8K, significant at 128K. For a 7B it's ~0.5 GB at 8K and ~3–4 GB at 64K+. - Overhead – CUDA context, engine buffers, tokenizer: usually ~0.5–1.5 GB on top. So a 7B at Q4_K_M with 8K context ≈ 4.5 GB weights + 0.5 GB KV + ~0.5 GB overhead ≈ **5.5 GB**. Comfortable on an 8 GB card, tight on a 6 GB one. ## 7B models: the entry level - Q4_K_M (~4.5 GB) – runs on an 8 GB GPU (RTX 4060, RX 7600) with room for context. - Q8_0 (~8 GB) – an 8 GB card is right at the edge; 12 GB is comfortable. - FP16 (~16 GB) – you need a 16 GB card; at that point you'd usually rather run a 14B Q4. ## 13–14B models: the practical sweet spot - Q4_K_M (~9–10 GB) – 12 GB runs it; 16 GB (RTX 4060 Ti 16 GB, RX 7600 XT) runs it with a long context. - Q8_0 (~16 GB) – a 16 GB card runs this with a short context; 24 GB is comfortable. - FP16 (~30 GB) – needs a 24 GB card with offload, or cloud. 14B Q4 at ~40–50 tok/s is the point where a local model stops feeling like a demo and starts feeling like a useful assistant. ## 27–32B models: where 24 GB earns its keep - Q4_K_M (~20 GB + KV) – a 24 GB GPU (RTX 4090, RX 7900 XTX) runs 32B Q4 with an 8–16K context. At 32K context you're right at the edge. - Q8_0 (~32 GB) – beyond consumer cards; needs a 32 GB card or cloud. A 16 GB card can run a 30B Q4 with **partial offload** if you have 16–32 GB of system RAM – it works, but generation drops to a crawl (often 5–15 tok/s). ## Context length is the hidden variable Doubling context roughly doubles the KV cache. At 128K context, KV cache alone can be 3–6 GB – that's a whole quantization tier's worth of memory. If you're planning long documents, buy the bigger card or shorten your context. ## FAQ ### Can I run a model with less VRAM than the weights need? Only by offloading some layers to system RAM. It works, but every offloaded layer slows generation. More RAM helps; it doesn't replace VRAM. ### Does CPU or GPU speed matter for the numbers above? Not for the fit – VRAM is about capacity. Speed matters separately for tokens/sec. ### What's the minimum usable setup? 8 GB VRAM + 16 GB system RAM gets you a decent 7B Q4 local assistant. From there, VRAM is the upgrade that matters. ## Bottom line 8 GB → 7B. 16 GB → 14B. 24 GB → 32B. Add ~1–3 GB for a long context, and never trust a VRAM claim without knowing the quantization. Check your exact model with the calculator below before you spend money on a card. 🔧 Get the exact number for your model LLM VRAM Calculator – weights + KV cache + fit verdict. Then see how fast it'll run with the GPU AI Performance Calculator. [Open calculator →](https://velstech.net/llm-vram-calculator) ## FAQ **What does quantization mean for VRAM?** Quantization lowers the precision of the model's weights (e.g. from 16-bit to 4-bit), which shrinks the memory footprint. A 7B model at 4-bit takes roughly half the VRAM it does at 16-bit. **Does a longer context window need more VRAM?** Yes. The KV cache grows with context length, so a 32K context can add several GB of VRAM on top of the weights. Use a tool like the LLM VRAM calculator to check your exact setup. --- *VelsTech – technology explained for everyone. Original: https://velstech.net/how-much-vram-for-llm*