When you use ChatGPT, your prompt is sent to a data center somewhere. But these days, you can also run models entirely on your own computer. Both approaches work — the question is which one fits your use case. Let's break it down honestly.
What "local" actually means
Running AI locally means the model lives on your machine — a laptop, a desktop, or a home server. Tools like Ollama, LM Studio, and llama.cpp make this surprisingly easy. You download a model once, then chat with it offline, forever, for free.
The catch? Your hardware has to be powerful enough. Models are measured in billions of parameters (like 7B, 13B, 70B), and bigger models need more RAM and more compute. A phone or old laptop might only comfortably run smaller models (7B and under).
Cloud: the default choice
Cloud AI (ChatGPT, Claude, Gemini, and developer APIs) runs on massive clusters owned by big companies. You get:
- The best models available — the frontier models are too big to run at home.
- No hardware requirements — any device with a browser works.
- Constant improvements — you always get the latest version.
But you pay — in money (subscriptions or per-token API fees), in privacy (your prompts go to a third party), and in availability (you need an internet connection).
Comparing the four things that matter
Local Cloud
─────────────────────────────────────────────────────
Privacy Best — nothing leaves Prompts sent to provider
your machine
Cost Free after hardware Subscription or per-use fees
Quality 7B–30B models typical Frontier models, best quality
Offline Works fully offline Requires internet
Setup Needs capable hardware Zero setup
Who should go local
- Privacy-conscious users — if you work with personal or sensitive text and don't want it on someone's server.
- Offline needs — travel, remote work, or places with poor connectivity.
- Hobbyists and tinkerers — local AI is fun to experiment with and learn from.
- Budget-conscious — heavy everyday use of a paid chat tool adds up; local is a fixed cost.
Who should stay on cloud
- Power users — for writing, coding, and complex reasoning, cloud models are meaningfully smarter.
- No-capable-hardware users — a typical laptop can run local models but not the really good ones.
- Business/API needs — integrating a hosted API is simpler and more reliable than maintaining your own inference server.
The hybrid approach
You don't have to pick just one. A smart setup many people end up with:
- Local for quick, private, or offline questions — summarising, rewriting, brainstorming.
- Cloud for hard problems — long-form writing, complex code, deep reasoning.
Local tools can even be wired to fall back to cloud for tricky prompts, or vice versa.
Getting started locally
If you want to try it, start small. Install Ollama and grab a 7B model:
# Install Ollama (macOS/Linux/Windows available) curl -fsSL https://ollama.com/install.sh | sh # Pull a good small model ollama pull llama3.2 # Chat with it ollama run llama3.2
Your computer's RAM matters more than its GPU for most models. As a rough rule: 8 GB RAM → 7B models comfortably; 16 GB → 8–13B models; 32 GB+ → 30B class.
Bottom line
Start with cloud — it's the best experience with zero setup. The moment you care about privacy, offline use, or cost for high-volume work, install a local tool and keep the two side by side. It's not either/or; it's a toolkit.