BackendROCm (IQ3_S, auto-fit)
Decode7.3 tok/s
Prompt eval112.03 tok/s
VRAM12 GB
Memory bandwidth~384 GB/s
ModelQwen3.8 27B GSQ-RCO
Parameters27B (GDN hybrid)
QuantizationIQ2_XS / IQ3_S
Context52K / 4K
KV quantq8_0
OffloadFully on GPU
About this result
IQ2_XS wins decode (+9%) and fits 52K ctx with q8_0 KV under --fit. Enabling the MTP head (--spec-type draft-mtp) lifts IQ2_XS to 23.07 t/s (48% acceptance, mean len 2.44) but ROCm lacks TOP_K for the draft sampler and the draft context makes --fit bail to 4K. IQ3_S at -ngl 999 wins prompt eval (+57%) yet only fits a 4K ctx at 97% VRAM; letting --fit choose (no -ngl) is worse, not better: it offloads layer 0 to CPU and the fused Gated Delta Net kernel is unsupported on ROCm, collapsing decode to 7.3 t/s. Manual -ngl 999 is the correct config for IQ3_S on this card.
This result was tested in the VelsTech Lab on real hardware. Read the full Lab report →