# Qwen3.8 27B GSQ-RCO on RX 6800M: IQ2_XS vs IQ3_S – VelsTech Lab > Tested Qwen3.8 27B GSQ-RCO on a 12 GB RX 6800M – IQ2_XS decodes 20.8 tok/s with 52K context at 90% VRAM, while IQ3_S wins prompt eval but gets stuck at 4K context because --fit aborted on a manual -ngl. *Source: https://velstech.net/qwen38-27b-gsq-rco-rx6800m · Updated: 2026-09-05 · Category: AI · Tags: VelsTech Lab, Qwen, GGUF, RX 6800M, Quantization, Benchmark* *Markdown version of [Qwen3.8 27B GSQ-RCO on RX 6800M: IQ2_XS vs IQ3_S – VelsTech Lab](https://velstech.net/qwen38-27b-gsq-rco-rx6800m). [Read the full guide with interactive tools](https://velstech.net/qwen38-27b-gsq-rco-rx6800m).* *Also as Markdown: [Hindi](https://velstech.net/qwen38-27b-gsq-rco-rx6800m.hi.md) · [Tamil](https://velstech.net/qwen38-27b-gsq-rco-rx6800m.ta.md).* --- 🧪 VelsTech Lab Can a 27B dense model be usable on a 12 GB mobile GPU? I ran two quants of **Qwen3.8 27B GSQ-RCO** – `IQ2_XS` (with an MTP draft head) and `IQ3_S` – side by side on the RX 6800M. The quant you pick decides both your context budget and your speed, and one llama.cpp flag mistake quietly cost the IQ3_S run 92% of its context. 🔧 Check your own fit first [LLM VRAM Calculator](https://velstech.net/llm-vram-calculator) – does 27B IQ2/IQ3 + your context fit your VRAM? · [GPU AI Performance Calculator](https://velstech.net/gpu-ai-calculator) – what tok/s to expect before you run. [Open calculators →](https://velstech.net/llm-vram-calculator) ## The test rig - Machine: ASUS ROG Strix G513QY · Ryzen 9 5900HX (8C/16T) · AMD RX 6800M 12GB (RDNA2, mobile) · 32 GB RAM · 2 TB NVMe - OS: Ubuntu 26.04.1 LTS (Wayland) · ROCm · llama.cpp (current build – see the llama.cpp guide for flags and benchmark reading), n_threads=8 - Model A: Qwen3.8-27B-GSQ-RCO-IQ2_XS-mtp.gguf – 27B dense, i-quants at ~2 bpw, ships a block-64 MTP draft head - Model B: Qwen3.8-27B-GSQ-RCO-IQ3_S.gguf – same model, ~3 bpw, no draft head - Both runs: -ngl 999 --cache-type-k q8_0 --cache-type-v q8_0 -fa on --fit on --reasoning-preserve --jinja ![System details panel showing ASUS ROG Strix G513QY, Ryzen 9 5900HX, 32 GB RAM and AMD Radeon RX 6800M on Ubuntu 26.04](img/qwen38-27b-gsq-rco/rig.webp) The test machine – a 2021 gaming laptop with a 12 GB RDNA2 mobile GPU. ## How we ran it ### Run 1 – IQ2_XS (with MTP tensors) ``` export LD_LIBRARY_PATH=/opt/rocm/lib:$LD_LIBRARY_PATH llama-server \ -m /home/user-name/models/Qwen3.8-27B-GSQ-RCO-IQ2_XS-mtp.gguf \ -ngl 999 \ --cache-type-k q8_0 \ --cache-type-v q8_0 \ -fa on \ --fit on \ --reasoning-preserve \ --jinja ``` ### Run 2 – IQ3_S ``` export LD_LIBRARY_PATH=/opt/rocm/lib:$LD_LIBRARY_PATH llama-server \ -m /home/user-name/models/Qwen3.8-27B-GSQ-RCO-IQ3_S.gguf \ -ngl 999 \ --cache-type-k q8_0 \ --cache-type-v q8_0 \ -fa on \ --fit on \ --reasoning-preserve \ --jinja ``` ## What we measured ``` Metric IQ2_XS-mtp IQ3_S Delta ────────────────────────────────────────────────────────────── Decode (eval) 20.77 tok/s 18.97 tok/s +9.5% Decode (tg) 20.70 tok/s 18.93 tok/s +9.4% Prompt eval 113.51 tok/s 178.32 tok/s -36% (IQ3_S faster) Context fitted 52,736 tok 4,096 tok 12.9× VRAM used 11.57 / 12.87 GB 12.52 / 12.87 GB (90%) (97%) Graphs reused 124 252 Chat UI avg speed 20.9 t/s 19.0 t/s ``` **Takeaway:** IQ2_XS wins decode *while carrying 13× more context* – and a bigger KV cache makes decode slower, not faster, so the win is real. IQ3_S reads more bytes per token (~3 bpw vs ~2 bpw) and the GPU is bandwidth-bound on this card; that gap shows up as ~1.8 tok/s lost. The surprise is prompt eval: IQ3_S processes input **57% faster** (178 vs 114 tok/s) – i-quants at 2 bpw pay a heavier dequant/codebook cost per token during the compute-bound prompt pass, while decode is bandwidth-bound and flips the ranking. ## The proof – chat runs ![llama.cpp chat UI showing the IQ2_XS model introducing itself with 498 of 52.74K context used and average speed 20.9 tokens per second](img/qwen38-27b-gsq-rco/chat-iq2xs.webp) IQ2_XS: 498 / 52.74K context used, 20.9 t/s average in the chat UI. ![llama.cpp chat UI showing the IQ3_S model introducing itself with 711 of 4.10K context used at 17 percent and average speed 19.0 tokens per second](img/qwen38-27b-gsq-rco/chat-iq3s.webp) IQ3_S: a longer answer, but only 4.10K context existed to fill – 19.0 t/s. ## VRAM – the quiet story ![GPU monitor showing AMD Radeon RX 6800M video memory usage at 11.57 of 12.87 GB, 90 percent](img/qwen38-27b-gsq-rco/vram-iq2xs.webp) IQ2_XS at 90% VRAM – weights plus a 52K q8_0 KV cache, with ~1.3 GB spare. ![GPU monitor showing AMD Radeon RX 6800M video memory usage at 12.52 of 12.87 GB, 97 percent](img/qwen38-27b-gsq-rco/vram-iq3s.webp) IQ3_S at 97% VRAM with only the default 4K context – the weights alone nearly fill the card. ## What the logs said – and what broke - Run 2: W common_fit_params: failed to fit params to free device memory: n_gpu_layers already set by user to 999, abort – --fit wants to choose -ngl and context together; forcing -ngl 999 by hand makes it bail, so IQ3_S fell back to the default 4096-token context. (The follow-up below shows that removing -ngl is actually worse on this model – keep it forced.) - Run 1: fifteen W model has unused tensor blk.64.* lines – the whole extra transformer block (attention + FFN + nextn.* MTP head, ~348 MB) is ignored because no --spec-type draft-mtp flag was passed. The MTP head is dead weight in this run; enabling it is the obvious next test (it gave +14.6% on Tiel-Coder 35B-A3B). - Both runs: W srv llama_server: CORS is set to allow all origins ('*') and no API key is set – harmless while the server binds 127.0.0.1 only; never expose this port to a network without an API key. - Both runs: n_ctx_slot = 52736 / 4096, kv_unified = 'true' – unified KV across 4 slots; the 52K figure is what --fit proved could fit alongside the IQ2_XS weights. ## Follow-up: letting --fit choose, and turning MTP on I re-ran both configs to answer the two open questions from the first pass. Both answers were unexpected. ### IQ3_S without -ngl – auto-fit ``` prompt eval: 3329.43 ms / 373 tokens ( 8.93 ms/tok, 112.03 tok/s) eval (decode): 45080.30 ms / 330 tokens ( 137.02 ms/tok, 7.30 tok/s) context: n_ctx_slot = 4096 (fit did NOT grow it) W resolve_fused_ops: layer 0 is assigned to device CPU but fused Gated Delta Net (chunked) is assigned to device ROCm0 W fused Gated Delta Net (chunked) not supported, set to disabled ``` Dropping `-ngl 999` made things dramatically *worse*: 7.30 tok/s, a third of the manual setting. `--fit` decided to put layer 0 on the CPU and never grew the context past the default 4K. The warning also reveals what this model actually is: **Qwen3.8 is a Gated Delta Net (GDN) hybrid** – most layers use linear attention, and ROCm has no fused chunked-GDN kernel, so the hybrid CPU/GPU split collapses throughput. On this card, `-ngl 999` (all layers on GPU, accept the small context) is the correct IQ3_S config – my original "mistake" was the right answer. ### IQ2_XS with --spec-type draft-mtp ``` prompt eval: 3568.06 ms / 373 tokens ( 9.57 ms/tok, 104.54 tok/s) eval (decode): 8148.60 ms / 189 tokens ( 43.34 ms/tok, 23.07 tok/s) generate: 103 tokens @ 26.73 tok/s (tg), 26.99 tok/s (tg_3s) draft: acceptance = 0.48052 (111 accepted / 231 generated), mean len = 2.44 context: n_ctx_slot = 4096 (--fit aborted: -ngl 999 + draft context) W llama_sampler_backend_support: device 'ROCm0' does not have support for op TOP_K needed for sampler 'top-k' (×4) ``` The MTP head is alive: **23.07 tok/s** (peaking at 26.7) versus 20.77 without it. Two caveats keep this from being a clean +11%: the MTP run got only 4K context (the draft context eats the headroom `--fit` would otherwise spend on KV, and with `-ngl 999` set, fit bails), and a smaller KV cache flatters decode – at equal context the net MTP gain is closer to **+5–8%**. The four TOP_K warnings matter too: ROCm can't run the draft sampler's top-k on GPU, which likely caps acceptance at 48% (vs 54.7% on Tiel-Coder). Still the fastest IQ2_XS number on this card so far. ## Bottom line On a 12 GB RX 6800M, **IQ2_XS is the practical choice for this 27B**: ~21 tok/s decode at 52K context, and **~23 tok/s with --spec-type draft-mtp** if you can live with 4K context (or wait for fit/MTP to cooperate). IQ3_S is the better-looking quant this card can't afford: 97% VRAM before any real context, and the "proper" auto-fit path is a trap – it CPU-offloads a GDN layer and decode falls off a cliff at 7.3 tok/s. If you run IQ3_S here, force `-ngl 999` and accept 4K. The open questions now belong to the software stack, not the hardware: a fused GDN kernel for ROCm and TOP_K support for the MTP sampler would unlock both the 3 bpw quant and higher draft acceptance. 🧪 More Lab Related: [How much VRAM does an LLM need?](https://velstech.net/how-much-vram-for-llm) · [Quantization deep dive](https://velstech.net/quantization-deep-dive) · [MoE vs Dense on the same card](https://velstech.net/moe-vs-dense-rx6800m-16k-vs-262k). [Back to VelsTech Lab →](https://velstech.net/lab) --- *VelsTech – technology explained for everyone. Original: https://velstech.net/qwen38-27b-gsq-rco-rx6800m*