Wondering how much disk space a model will take at Q4_K_M vs Q8? Enter the parameter count and this shows the estimated file size for every common quantization level – no need to check model pages one by one.
What is GGUF quantization? Quantization reduces model precision – from 16-bit (F16) down to 2.6-bit (Q2_K) – trading a small quality loss for a much smaller file size. A 70B parameter model drops from ~140 GB at F16 to ~40 GB at Q4_K_M, making it feasible to run on a single consumer GPU. The most popular balance is Q4_K_M, which retains most of the original quality at roughly a quarter of the size.
How to choose the right quantization: If you have enough VRAM, start with Q8_0 or Q6_K for the best quality. For 8–12 GB GPUs, Q4_K_M is the sweet spot. For 6 GB or older cards, Q3_K or Q2_K may be the only option, though quality drops noticeably below Q4. This calculator shows you the file size for each level so you can decide whether the model fits your storage before downloading.
Estimates assume ~1.25 bytes per parameter overhead for the tokenizer and metadata. Real sizes vary by model architecture. See the GGUF explained guide for what each quantization level means.