The KV cache grows with every token of context. This calculator shows how much memory the cache will use at different context lengths and quantizations, so you know what fits your GPU before you run the model.
Rough estimate: layers × kv_heads × head_dim × 2 × bytes_per_value × context.
For precise numbers including model weights, use the
LLM VRAM Calculator.
See the KV cache guide for how to reduce it.