How much VRAM do you need for 7B, 14B, and 32B models?

The single most common question in local AI is "can my GPU run this model?" The answer comes down to one number: VRAM. Here's how to estimate it yourself in about thirty seconds – and a cheat sheet for the popular model sizes.

The quick answer

VRAM needed for the model alone (weights), plus ~1–3 GB for KV cache + overhead at 8K context:

Model size   Q4_K_M        Q8_0          FP16
──────────────────────────────────────────────
7–8B         ~4–5 GB      ~8–9 GB      ~15–16 GB
13–14B       ~9–10 GB     ~16–17 GB    ~30 GB
27–32B       ~18–21 GB    ~32–35 GB    ~60+ GB
70B          ~40–45 GB    ~75 GB       cloud only

Translate that to GPUs: an 8 GB card runs 7B Q4. A 16 GB card is the sweet spot – 14B Q4 fully, 32B with light offload. A 24 GB card runs 32B Q4 fully. That's why "16 GB" keeps showing up in our best GPU for local LLMs picks.

What actually eats VRAM

Three things, and you need to add all of them:

1. Model weights – the big one. Roughly parameters × bytes per weight. At Q4_K_M that's ~0.6 bytes/weight; at Q8_0 ~1.06; at FP16 exactly 2.
2. KV cache – grows with context length and batch size. Small at 8K, significant at 128K. For a 7B it's ~0.5 GB at 8K and ~3–4 GB at 64K+.
3. Overhead – CUDA context, engine buffers, tokenizer: usually ~0.5–1.5 GB on top.

So a 7B at Q4_K_M with 8K context ≈ 4.5 GB weights + 0.5 GB KV + ~0.5 GB overhead ≈ 5.5 GB. Comfortable on an 8 GB card, tight on a 6 GB one.

7B models: the entry level

• Q4_K_M (~4.5 GB) – runs on an 8 GB GPU (RTX 4060, RX 7600) with room for context.
• Q8_0 (~8 GB) – an 8 GB card is right at the edge; 12 GB is comfortable.
• FP16 (~16 GB) – you need a 16 GB card; at that point you'd usually rather run a 14B Q4.

13–14B models: the practical sweet spot

• Q4_K_M (~9–10 GB) – 12 GB runs it; 16 GB (RTX 4060 Ti 16 GB, RX 7600 XT) runs it with a long context.
• Q8_0 (~16 GB) – a 16 GB card runs this with a short context; 24 GB is comfortable.
• FP16 (~30 GB) – needs a 24 GB card with offload, or cloud.

14B Q4 at ~40–50 tok/s is the point where a local model stops feeling like a demo and starts feeling like a useful assistant.

27–32B models: where 24 GB earns its keep

• Q4_K_M (~20 GB + KV) – a 24 GB GPU (RTX 4090, RX 7900 XTX) runs 32B Q4 with an 8–16K context. At 32K context you're right at the edge.
• Q8_0 (~32 GB) – beyond consumer cards; needs a 32 GB card or cloud.

A 16 GB card can run a 30B Q4 with partial offload if you have 16–32 GB of system RAM – it works, but generation drops to a crawl (often 5–15 tok/s).

Context length is the hidden variable

Doubling context roughly doubles the KV cache. At 128K context, KV cache alone can be 3–6 GB – that's a whole quantization tier's worth of memory. If you're planning long documents, buy the bigger card or shorten your context.

FAQ

Can I run a model with less VRAM than the weights need?

Only by offloading some layers to system RAM. It works, but every offloaded layer slows generation. More RAM helps; it doesn't replace VRAM.

Does CPU or GPU speed matter for the numbers above?

Not for the fit – VRAM is about capacity. Speed matters separately for tokens/sec.

What's the minimum usable setup?

8 GB VRAM + 16 GB system RAM gets you a decent 7B Q4 local assistant. From there, VRAM is the upgrade that matters.

Bottom line

8 GB → 7B. 16 GB → 14B. 24 GB → 32B. Add ~1–3 GB for a long context, and never trust a VRAM claim without knowing the quantization. Check your exact model with the calculator below before you spend money on a card.