MoE vs Dense on RX 6800M: 3B Active vs 27B at 16K/262K – VelsTech Lab

🧪 VelsTech Lab Same card, same engines, two opposite ways to scale: Qwen 27B dense at 16K vs Ornith 35B-A3B MoE (3B active) at 262K, both at q8 KV on a 12GB RX 6800M. Which wins depends on what you ask vs how much you generate.

The two runs side-by-side

Same GPU RX 6800M 12GB (~384 GB/s), same q8_0 KV, same n_threads=8, same 335-373 prompt tokens. Only model + context + MoE offload differ.

Model               Params   Context  KV type  Offload          Build   Prompt eval          Decode tg          Total
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Qwen 27B Ridge       27B dense  16K   q8_0     -ngl 999 (+fit abort)  ROCm    1713 ms /373 = 217.6 tok/s   18.12 tok/s   14.23s /601 tok
3.7bpw Ridge                                                Vulkan  2493 ms /373 = 149.6 tok/s   21.84 tok/s   13.57s /616 tok

Ornith 35B-A3B MoE   35B/3B*    262K  q8_0     --n-cpu-moe 28        ROCm    3995 ms /335 = 83.8 tok/s    25.62 tok/s   14.65s /609 tok
Q5_K/Q4_K mix               kv_unified=false              Vulkan  4029 ms /335 = 83.1 tok/s    19.66 tok/s   19.59s /642 tok
* 35B total, 3B active – MoE decodes 3B, not 35B.

First two rows: Qwen 27B 16K full report; last two: Ornith 35B 262K full report. All at -c reserved KV, fa on, 373/335 prompt tokens, ~600 gen tokens.

What actually happens

• Decode is MoE’s win at scale, but not universally. At 16K dense, Vulkan wins +20% (21.84 vs 18.12). At 262K MoE, ROCm wins +30% (25.62 vs 19.66). Same card, opposite engine winner – the bottleneck flips from pure VRAM BW (dense 27B) to CPU-MoE + PCIe (28 experts on CPU) at 262K.
• Prompt eval collapses with context reservation, not prompt size. Qwen 16K prompt 217 tok/s vs Ornith 262K prompt 83 tok/s – same 335 tokens, but Ornith reserves a 262K KV pool (kv_unified=false, n_slots=1) that costs prefill bookkeeping. You pay for -c even when prompt is short.
• MoE decodes far fewer bytes per token. Dense decodes 27B weights per token; MoE decodes ~3B active. That’s why Ornith at 262K can still do 25.6 tok/s on 12GB while Qwen at 16K does 18-21 – active, not total, matters. Our GPU calculator bound decode ≈ BW×0.8 / bytes_per_token predicts ~30 tok/s if resident; the gap to 25.6 is CPU MoE + offload.
• Both forced -ngl 999 → failed to fit params... abort. --fit on was ignored. Next Lab will drop -ngl and log fitted layers, plus add --load-mode none for MoE (fixes tensor overrides to CPU with mmap).

VRAM math – why 12GB still runs

Qwen 27B 3.7bpw: 27B ×0.4625 ≈12.5 GB weights + q8 KV 16K ≈2.2 GB + 6% overhead → ~15.6 GB (from VRAM Calculator) – well over 12GB, so partial offload explains 18-21 tok/s vs 30 tok/s bound.

Ornith 35B MoE 3B active: 35B mix Q5/Q4 ≈13-14 GB if counted as 35B, but KV per token scales with active 3B, so 262K q8 is ~262K×0.5KB ≈131 GB if naively counted as dense. That’s why we force 28 experts to CPU – you’re not resident, you’re sparse. Enter 3B active + custom layers 48 / kv 4 / hd 128 in the calculator, not 35B, or toggle Custom model….

weights = params × bytes_per_weight / GiB
kv_per_token = 2 × layers × kv_heads × head_dim × bytes
kv_cache = kv_per_token × context × batch / GiB
total = weights + kv_cache + (weights+kv)×0.06+0.25

decode_tok/s ≈ (memory_BW ×0.8) / bytes_per_token (weights + KV per token)

So which should you pick on a 12GB card?

• Short prompts + long answers (chat, 16K window): Qwen 27B dense on Vulkan (21.8 tok/s) is the better chat feel.
• Huge context + MoE (262K window, 28 CPU experts): Ornith 35B MoE on ROCm (25.6 tok/s) wins decode despite the window, but needs --load-mode none and patience on first prompt (83 tok/s).
• Next optimization: Both need no -ngl 999, log fitted layers, and test q8 vs f16 KV at each context. That’s Lab #4.