# LLM Hardware Requirements Checklist: VRAM, RAM, CPU & PSU > Will an LLM run on your machine? VRAM by model size, system RAM, CPU, SSD and PSU checklist for running 7B-70B models locally, with calculator links. *Source: https://velstech.net/llm-hardware-requirements · Updated: 2026-09-22 · Category: Hardware · Tags: LLM, VRAM, Hardware, Local AI, Guide* *Markdown version of [LLM Hardware Requirements Checklist: VRAM, RAM, CPU & PSU](https://velstech.net/llm-hardware-requirements). [Read the full guide with interactive tools](https://velstech.net/llm-hardware-requirements).* *Also as Markdown: [Hindi](https://velstech.net/llm-hardware-requirements.hi.md) · [Tamil](https://velstech.net/llm-hardware-requirements.ta.md).* --- "What GPU do I need for AI?" is the wrong first question. The right one is **"will an LLM run on the machine I already have?"** – and the answer is a checklist, not a product pick. VRAM decides *whether* a model loads; system RAM, CPU, storage, and power decide whether it runs *well*. This guide walks the full checklist in order, with the numbers from our own [VRAM guide](https://velstech.net/how-much-vram-for-llm) and [Lab benchmarks](https://velstech.net/benchmarks/index) on a 12 GB RX 6800M. If you finish this and decide you need a new card instead, the [best GPU for local LLMs](https://velstech.net/best-gpu-for-local-llm) picks up where this leaves off. If you want the exact number for your setup, the [LLM VRAM Calculator](https://velstech.net/llm-vram-calculator) does the math with a fit verdict for your GPU. ## The short version - VRAM first: 8 GB runs 7B, 12–16 GB runs 14B, 24 GB runs 32B – all at 4-bit quantization. Everything else is secondary. - System RAM second: 16 GB minimum, 32 GB if you ever want CPU offload as a fallback. - CPU third: 6+ modern cores; it sets prompt speed and carries offloaded layers. - Storage fourth: model files run 5–40 GB each – a 512 GB NVMe SSD is the practical minimum. - PSU and cooling last: a loaded GPU draws its full TDP for hours; a tired PSU causes the weirdest "AI crashes." ## 1. VRAM: the gatekeeper A model either fits in VRAM or it doesn't – there is no "minimum settings" mode that makes a 70B model run on an 8 GB card. At the community-standard `Q4_K_M` quantization, plus ~1–3 GB for KV cache and overhead: | Model size | Q4 weights | Total needed | Fits on | | --- | --- | --- | --- | | 7–8B | ~4–5 GB | ~6 GB | 8 GB card (RTX 4060, RX 7600) | | 13–14B | ~9–10 GB | ~11 GB | 12–16 GB card | | 27–32B | ~18–21 GB | ~22 GB | 24 GB card (RTX 4090, RX 7900 XTX) | | 70B | ~40–45 GB | ~48 GB | Datacenter card or cloud – not home hardware | Two footnotes that bite beginners: **long context eats VRAM** (128K context can add 3–6 GB of KV cache – a whole tier), and **higher quantization costs VRAM** (Q8 roughly doubles the weights column). Our [quantization guide](https://velstech.net/quantization-deep-dive) shows what you gain for it. ## 2. System RAM: the safety net System RAM matters twice: the OS, browser, and engine need their share (8 GB just to breathe), and RAM is where overflow goes when VRAM runs out. **16 GB is the minimum** for any local-AI machine; **32 GB** if you want partial CPU offload as a fallback – a 30B Q4 can limp along on a 16 GB card with 32 GB of system RAM, at 5–15 tok/s. Slow, but it runs, which beats not running. ## 3. CPU: speed, not fit The CPU doesn't decide *whether* a model loads – but it decides prompt-eval speed (how fast long documents get absorbed) and how painful offload is. Rule of thumb: **6+ modern cores** (any Ryzen 5 / Core i5 from the last five years). Below that, even a good GPU feels sluggish on long prompts because the CPU can't feed it. ## 4. Storage: bigger and faster than you think Model files are large and you will collect them: a 7B quant is ~5 GB, a 32B quant ~20 GB, and you will keep three "just to compare." **512 GB NVMe minimum, 1 TB comfortable.** Speed matters at load time (NVMe vs SATA is seconds vs a minute per model load) and for offload swap – never put models on a spinning drive you also boot from. ## 5. PSU, cooling, OS: the unglamorous trio - PSU: a GPU at full inference load draws near its TDP for hours. A 650W quality unit for mid-range cards, 850W+ for 24 GB flagships. Random crashes under load are a dying PSU until proven otherwise. - Cooling: sustained 100% GPU utilization is hotter than gaming bursts. Undervolting a Radeon costs ~5% speed for ~20% less heat – worth it on laptops especially. - OS and drivers: Linux + ROCm for AMD, any OS + CUDA for NVIDIA. Our ROCm vs Vulkan guide and ROCm install guide cover the AMD path step by step. ## Three machines, verdicts - Thin-and-light laptop (integrated graphics, 16 GB RAM): 7B Q4 via CPU at a few tok/s, or small models only. Usable for chat, painful for agents. - 2021 gaming laptop (RTX 3060 / RX 6800M, 12 GB VRAM, 32 GB RAM): the sweet spot – 14B fully, 27–32B with tuned quants (our Lab runs exactly this class daily). - Desktop with a 24 GB card: 32B fully with long context. The point where local stops feeling like a compromise for most work. ## Bottom line Check in order: VRAM → RAM → CPU → storage → PSU. Most "can I run local AI?" questions are answered at step one – 8 GB means 7B, 16 GB means 14B, 24 GB means 32B. Run your exact model through the [VRAM calculator](https://velstech.net/llm-vram-calculator) before spending money, and only then ask which card to buy. ## FAQ **How much VRAM do I need to run an LLM?** At 4-bit quantization: ~6 GB total for 7B (8 GB card), ~11 GB for 14B (12–16 GB card), ~22 GB for 32B (24 GB card). 70B needs ~48 GB – datacenter or cloud, not home hardware. **How much system RAM do I need for local AI?** 16 GB minimum for any local-AI machine. Choose 32 GB if you want CPU offload as a fallback – a 30B model can limp along on a 16 GB card with 32 GB of system RAM. **Does CPU matter for running LLMs locally?** Not for fit – VRAM decides that. It matters for prompt-eval speed and offload pain: 6+ modern cores is the rule of thumb, below which even a good GPU feels sluggish on long prompts. **Can I run a 70B model at home?** Not practically – ~48 GB at 4-bit quantization exceeds every consumer card. Split across multiple GPUs or use the cloud; a 32B model on a 24 GB card is the realistic home ceiling. --- *VelsTech – technology explained for everyone. Original: https://velstech.net/llm-hardware-requirements*