🧪 VelsTech Lab
Practical tests on real hardware – not press releases. Every Lab post lists prerequisites, hardware/software specs, method, raw results and what broke – including failures.
All experiments · VRAM Calculator · GPU Performance · Lab overview
35B-A3B MoE (Q4_K_M) at full 262K q8 KV on a 12 GB RX 6800M – decode 24.3 → 22.7 → 20.9 tok/s as experts move to RAM, prompt ~89–105 tok/s, and vision needs VRAM headroom: image encode OOMs at 99% but works at 94% and 86%.
24.3 / 22.7 / 20.9 tok/s decode · 262K ctx · 28 experts · 30 experts27B ternary on a 12 GB RX 6800M – stock llama.cpp rejects type 143/142, Prism fork + gfx1031 source build fixes it. PQ2_0 decodes 32.8 tok/s vs PTQ1_0 18.8 tok/s.
PQ2_0 32.8 tok/s · PTQ1_0 18.8 tok/s · 32K ctxThree quants of Spark-X2.5 4B side by side on a 12 GB RX 6800M: Q8_0 wins prompt eval at 2149 tok/s, Q4_K_M wins decode at 98.5 tok/s with 254K context fit, and BF16 loses a quality task to both.
Q8_0 2149 tok/s prompt · Q4_K_M 98.5 tok/s decode · 254K fitTested Qwen3.8 27B GSQ-RCO on a 12 GB RX 6800M – IQ2_XS decodes 20.8 tok/s with 52K context at 90% VRAM, while IQ3_S wins prompt eval but gets stuck at 4K context because --fit aborted on a manual -ngl.
Same 12GB RX 6800M + R9 5900HX – Tiel-Coder 35B-A3B Q4_K_XL with and without the MTP draft head at 262K q8 KV, 28-32 CPU experts, ROCm 10.0. Does speculative decoding pay off on MoE?
MTP 29.09 vs 25.39 tok/s (+14.6%) · acceptance 54.7% · mean len 2.64Same 12GB RX 6800M – Ornith MoE (3B active, 262K) vs Qwen dense (27B, 16K) at q8 KV, ROCm vs Vulkan head-to-head. When MoE wins and when it doesn’t.
ROCm 25.6 vs 19.6 tok/s (Ornith) · Vulkan 21.8 vs 18.1 (Qwen)35B-A3B (3B active) Q5/Q4 mix at 262K q8 KV, 28 CPU experts – ROCm 25.6 tok/s vs Vulkan 19.6, 83 tok/s prompt. Huge context, sparse decode.
12GB · q8 KV · --n-cpu-moe 28 · kv_unified=false27B at 3.7bpw (~12.5GB) at 16K q8 KV – ROCm 18.12 tok/s vs Vulkan 21.84, 217 vs 149 tok/s prompt. First Lab with raw logs.
12GB · q8 KV · -ngl 999 (fit abort) · blk.64 nextnVRAM needed for 7B, 14B, and 32B LLMs – weights, quantization, KV cache, and context length, with a quick-reference table you can bookmark.
Guide · pairs with the VRAM calculatorRe-run both models without forced -ngl 999 (log fitted layers) and with --load-mode none for 28 CPU experts – will the +30% ROCm at 262K hold?
Practical picks for running local AI, based on the benchmarks above:
RTX vs Radeon, how much VRAM you really need, and what actually matters for llama.cpp and Ollama.
Aug 27, 2026Which cards actually fit the budget and can run local LLMs, with VRAM per rupee compared.
Aug 27, 2026A small desktop with a big NPU, made to run LLMs locally – what's inside and whether it's worth buying.
Aug 27, 2026Every Lab follows the same format so you can compare runs:
llama-server command, prompt tokens, slots, KV settingsprint_timing table – prompt eval ms/tok & tok/s, decode tg, graphs reused, total time (no smoothing)failed to fit params / tensor overrides to CPU with mmap / blk.64 unused nextn and the fixFailures are valuable content – we log them. If a run OOMs or offloads, that’s the result.