# Qwen 27B Ridge 3.7bpw on RX 6800M: ROCm vs Vulkan at 16K – VelsTech Lab

> Tested Qwen 27B Ridge at 3.7bpw on a 12GB RX 6800M – ROCm 18.1 tok/s vs Vulkan 21.8 tok/s decode at 16K, with prompt-eval tradeoffs and VRAM fit.

*Source: https://velstech.net/qwen-27b-ridge-rocm-vs-vulkan · Updated: 2026-08-27 · Category: AI · Tags: VelsTech Lab, Qwen, RX 6800M, ROCm, Vulkan, Benchmark*

*Markdown version of [Qwen 27B Ridge 3.7bpw on RX 6800M: ROCm vs Vulkan at 16K – VelsTech Lab](https://velstech.net/qwen-27b-ridge-rocm-vs-vulkan). [Read the full guide with interactive tools](https://velstech.net/qwen-27b-ridge-rocm-vs-vulkan).*
*Also as Markdown: [Hindi](https://velstech.net/qwen-27b-ridge-rocm-vs-vulkan.hi.md) · [Tamil](https://velstech.net/qwen-27b-ridge-rocm-vs-vulkan.ta.md).*

---

🧪 VelsTech Lab – Coming soon This is the first measured post in the Lab format: exact hardware, software, command, and raw `llama-server` timings – including what broke.

🔧 Check your own fit first

[LLM VRAM Calculator](https://velstech.net/llm-vram-calculator) – does 27B 3.7bpw + 16K fit your VRAM? · [GPU AI Performance Calculator](https://velstech.net/gpu-ai-calculator) – what tok/s to expect before you run.

[Open calculators →](https://velstech.net/llm-vram-calculator)

## What we tested

- GPU: AMD RX 6800M 12GB (mobile, RDNA2, 12 GB VRAM, ~384 GB/s) – the budget 12 GB card we keep recommending for local AI

- Model: Qwen3.8-27B-Ridge-3.7bpw.gguf – Qwen 27B class at 3.7 bits per weight (~12.3 GB weights). Ridge quant, not K-quants.

- Context: -c 16384 with --cache-type-k q8_0 --cache-type-v q8_0 -fa on (q8 KV, flash attention on)

- Offload: -ngl 999 --fit on (forced all layers; see warning below)

- Prompt: 373 tokens (same prompt both runs), n_slots=4, n_ctx_slot=16384, kv_unified=true, n_threads=8

- Builds: Run 1 – /home/user-name/llama.cpp/build/bin/llama-server (ROCm); Run 2 – /home/user-name/llama.cpp_vulkan/build-vulkan/bin/llama-server (Vulkan)

## How we ran it

```
export LD_LIBRARY_PATH=/opt/rocm/lib:$LD_LIBRARY_PATH
llama-server \
 -m /home/user-name/models/Qwen3.8-27B-Ridge-3.7bpw.gguf \
 -ngl 999 \
 --cache-type-k q8_0 --cache-type-v q8_0 \
 -c 16384 -fa on --fit on --reasoning-preserve --jinja
```

Two backends, identical flags. Server reports `listening on http://127.0.0.1:8080` then processes a 373-token prompt and generates ~228-243 tokens (600 token budget, truncated 0).

## What we measured

```
Run            Prompt eval                Decode (tg)                Total
─────────────────────────────────────────────────────────────────────────────
ROCm  (build)  1713 ms / 373 tok = 4.60 ms/tok = 217.6 tok/s   100:18.42  155:18.12  210:18.08 → 18.12 tok/s   14.24s / 601 tok  graphs reused 226
Vulkan        2493 ms / 373 tok = 6.68 ms/tok = 149.6 tok/s   100:22.09  166:21.83  232:21.82 → 21.84 tok/s   13.57s / 616 tok  graphs reused 241
```

**Takeaway:** Vulkan is **~20% faster at decode** (21.84 vs 18.12 tok/s) – the part that matters for chatting. ROCm is ~45% faster at prompt eval (217 vs 149 tok/s) – the time to first token on a long prompt. End-to-end for 615 tokens, Vulkan wins by 0.67s (13.57 vs 14.23) because decode dominates.

## What the logs actually said – and what broke

- W common_fit_params: failed to fit params to free device memory: n_gpu_layers already set by user to 999, abort – You forced -ngl 999 so --fit on is ignored. On a 12 GB card this 27B q8-KV run must partially offload; forcing 999 hides how many layers actually fit. Fix next run: drop -ngl 999 and let --fit on choose, or set -ngl 60 and log n_gpu_layers.

- 14× W model has unused tensor blk.64.* (size ~20KB–73MB) – Model header advertises 65 blocks (0-64) but block 64 is Qwen3 nextn speculative head (eh_proj/enorm/hnorm/shared_head_norm). llama.cpp ignores it when not drafting with --jinja --reasoning-preserve under this build. Not an error, but you load ~300 MB of dead weight.

- kv_unified=true, n_slots=4, n_ctx_slot=16384 – 4 slots share one KV pool. If you hammer parallel requests at 16K each, you’ll OOM faster than the single-slot test shows.

## How this maps to the calculators

Weights: `27B × 0.4625 bytes (3.7bpw/8) ≈ 12.5 GB`. KV q8 at 16K: our [VRAM Calculator](https://velstech.net/llm-vram-calculator) estimates ~1.0 GB at 8K with `2×layers×kv_heads×head_dim×2 bytes×context` – at 16K q8 it's ~2× that, so ~2.2 GB. Plus 6% overhead + 0.25 GB → total ~15.6 GB, well above 12 GB. The fact it ran at all proves partial offload (via system RAM) – which explains the ~18-21 tok/s vs the [GPU AI Performance Calculator](https://velstech.net/gpu-ai-calculator) upper bound of ~30 tok/s if fully resident. That delta is the offload tax.

How we calculate – the formulas behind the numbers

Same logic as the calculators, linked so you can verify.

#### VRAM: LLM VRAM Calculator

```
weights = params × bytes_per_weight / GiB
kv_per_token = 2 × layers × kv_heads × head_dim × 2 bytes
kv_cache = kv_per_token × context × batch / GiB
```

#### Speed: GPU AI Performance Calculator

```
decode_tok/s ≈ (memory_BW × 0.8) / bytes_per_token
```

Your measured 18-21 tok/s is below the 30 tok/s bound because bytes_per_token includes PCIe + RAM offload, not just VRAM BW.

## Bottom line

On a 12 GB RX 6800M at 16K, Qwen 27B Ridge 3.7bpw **does run, but not fully resident**. If you chat (decode-bound), Vulkan’s +20% is real. If you feed long documents, ROCm’s faster prefill matters more. For a Lab verdict: **usable but offloaded** – expect 18-21 tok/s at 16K q8 KV, slower at 32K. Want it comfortably resident? Drop to `c 8192`, use `q8_0→f16` only if you have headroom, or step up to 16/24 GB.

🧪 More Lab

Next: same model with `--fit on` without `-ngl 999` (show fitted layers), and `q8_0 vs f16` KV at 16K. Then 70B range is cloud only.

[Back to VelsTech Lab →](https://velstech.net/index#lab)

---

*VelsTech – technology explained for everyone. Original: https://velstech.net/qwen-27b-ridge-rocm-vs-vulkan*