🧪 VelsTech Lab
Practical tests on real hardware – not press releases. Every Lab post lists prerequisites, hardware/software specs, method, raw results and what broke – including failures.
All experiments · VRAM Calculator · GPU Performance · Lab overview
Same 12GB RX 6800M + R9 5900HX – Tiel-Coder 35B-A3B Q4_K_XL with and without the MTP draft head at 262K q8 KV, 28-32 CPU experts, ROCm 10.0. Does speculative decoding pay off on MoE?
MTP 29.09 vs 25.39 tok/s (+14.6%) · acceptance 54.7% · mean len 2.64Same 12GB RX 6800M – Ornith MoE (3B active, 262K) vs Qwen dense (27B, 16K) at q8 KV, ROCm vs Vulkan head-to-head. When MoE wins and when it doesn’t.
ROCm 25.6 vs 19.6 tok/s (Ornith) · Vulkan 21.8 vs 18.1 (Qwen)35B-A3B (3B active) Q5/Q4 mix at 262K q8 KV, 28 CPU experts – ROCm 25.6 tok/s vs Vulkan 19.6, 83 tok/s prompt. Huge context, sparse decode.
12GB · q8 KV · --n-cpu-moe 28 · kv_unified=false27B at 3.7bpw (~12.5GB) at 16K q8 KV – ROCm 18.12 tok/s vs Vulkan 21.84, 217 vs 149 tok/s prompt. First Lab with raw logs.
12GB · q8 KV · -ngl 999 (fit abort) · blk.64 nextnVRAM needed for 7B, 14B, and 32B LLMs – weights, quantization, KV cache, and context length, with a quick-reference table you can bookmark.
Guide · pairs with the VRAM calculatorRe-run both models without forced -ngl 999 (log fitted layers) and with --load-mode none for 28 CPU experts – will the +30% ROCm at 262K hold?
Practical picks for running local AI, based on the benchmarks above:
RTX vs Radeon, how much VRAM you really need, and what actually matters for llama.cpp and Ollama.
Aug 27, 2026Which cards actually fit the budget and can run local LLMs, with VRAM per rupee compared.
Aug 27, 2026A small desktop with a big NPU, made to run LLMs locally – what's inside and whether it's worth buying.
Aug 27, 2026Every Lab follows the same format so you can compare runs:
llama-server command, prompt tokens, slots, KV settingsprint_timing table – prompt eval ms/tok & tok/s, decode tg, graphs reused, total time (no smoothing)failed to fit params / tensor overrides to CPU with mmap / blk.64 unused nextn and the fixFailures are valuable content – we log them. If a run OOMs or offloads, that’s the result.