BackendROCm
Decode18.12 tok/s
Prompt eval217.6 tok/s
VRAM12 GB
Memory bandwidth~384 GB/s
ModelQwen 27B Ridge
Parameters27B (dense)
Quantization3.7bpw
Context16K
KV quantq8_0
OffloadPartial (some layers in RAM)
About this result
Not fully resident on 12 GB (~15.6 GB needed). Partial offload to system RAM explains the ~18-21 tok/s decode. Vulkan wins decode (+20%), ROCm wins prompt eval (+45%).
This result was tested in the VelsTech Lab on real hardware. Read the full Lab report →