# Tiel-Coder 35B-A3B on RX 6800M: MTP vs non-MTP at 262K – VelsTech Lab

> Tested Tiel-Coder-35B-A3B Q4_K_XL on RX 6800M – MTP speculative decoding at 29.09 tok/s vs 25.39 tok/s without MTP, at 262K q8 KV, 28-32 CPU experts, ROCm 10.0.

*Source: https://velstech.net/tiel-coder-35b-mtp-rx6800m · Updated: 2026-09-01 · Category: AI · Tags: VelsTech Lab, MoE, MTP, RX 6800M, Speculative Decoding, Benchmark*

*Markdown version of [Tiel-Coder 35B-A3B on RX 6800M: MTP vs non-MTP at 262K – VelsTech Lab](https://velstech.net/tiel-coder-35b-mtp-rx6800m). [Read the full guide with interactive tools](https://velstech.net/tiel-coder-35b-mtp-rx6800m).*
*Also as Markdown: [Hindi](https://velstech.net/tiel-coder-35b-mtp-rx6800m.hi.md) · [Tamil](https://velstech.net/tiel-coder-35b-mtp-rx6800m.ta.md).*

---

🧪 VelsTech Lab Speculative decoding via Multi-Token Prediction (MTP) on a 35B-A3B MoE model – does the draft head actually speed things up on a 12 GB mobile GPU with 28+ CPU expert fallback?

🔧 Check your own fit first

[LLM VRAM Calculator](https://velstech.net/llm-vram-calculator) – does 35B Q4 + 262K fit your VRAM? · [GPU AI Performance Calculator](https://velstech.net/gpu-ai-calculator) – what tok/s to expect before you run.

[Open calculators →](https://velstech.net/llm-vram-calculator)

## Two builds tested

- Machine: R9 5900HX (8C/16T) · AMD RX 6800M 12GB (RDNA2, mobile) · 32 GB RAM · Ubuntu 26.04 · ROCm 10.0 · llama.cpp (current build)

- Model (no MTP): Tiel-Coder-35B-A3B-UD-Q4_K_XL.gguf – 35B total, 3B active MoE, Q4_K_XL quant

- Model (MTP): Tiel-Coder-35B-A3B-MTP-UD-Q4_K_XL.gguf – same model with a multi-token prediction draft head

- Context: -c 262144 with --cache-type-k q8_0 --cache-type-v q8_0 -fa on

- Offload: -ngl 999 (forced – see warning below)

- CPU experts: --n-cpu-moe 28 (non-MTP) vs --n-cpu-moe 32 (MTP) – note the slight difference

- Prompt: ~608-617 tokens (same prompt both runs), n_threads=8, kv_unified=false, n_slots=1, n_ctx_slot=262144

## How we ran it

### Run 1 – no MTP

```
export LD_LIBRARY_PATH=/opt/rocm/lib:$LD_LIBRARY_PATH
llama-server \
 -m /home/user-name/models/Tiel-Coder-35B-A3B-UD-Q4_K_XL.gguf \
 -ngl 999 \
 --n-cpu-moe 28 \
 --cache-type-k q8_0 \
 --cache-type-v q8_0 \
 -c 262144 -fa on \
 --jinja --parallel 1
```

### Run 2 – MTP speculative decoding

```
export LD_LIBRARY_PATH=/opt/rocm/lib:$LD_LIBRARY_PATH
llama-server \
 -m /home/user-name/models/Tiel-Coder-35B-A3B-MTP-UD-Q4_K_XL.gguf \
 --spec-type draft-mtp \
 -ngl 999 \
 --n-cpu-moe 32 \
 --cache-type-k q8_0 \
 --cache-type-v q8_0 \
 -c 262144 -fa on \
 --jinja --reasoning-preserve --parallel 1
```

## What we measured

### Run 1 – no MTP (--n-cpu-moe 28)

```
prompt eval:   5220.06 ms / 621 tokens (  8.41 ms/tok,  118.96 tok/s)
eval (decode): 5155.86 ms / 132 tokens ( 39.36 ms/tok,   25.41 tok/s)
generate:      100 tokens @ 25.39 tok/s (tg), 25.65 tok/s (tg_3s)
total:         10375.93 ms / 753 tokens
graphs reused: 131
```

### Run 2 – MTP (--spec-type draft-mtp, --n-cpu-moe 32)

```
prompt eval:   5574.58 ms / 621 tokens (  8.98 ms/tok,  111.40 tok/s)
eval (decode): 5105.25 ms / 131 tokens ( 39.27 ms/tok,   25.46 tok/s)
generate:      100 tokens @ 29.09 tok/s (tg), 29.38 tok/s (tg_3s)
draft:         acceptance = 0.54667 (82 accepted / 150 generated), mean len = 2.64
total:         10679.83 ms / 752 tokens
graphs reused: 50
```

### Head-to-head

```
Metric               No MTP              MTP           Delta
─────────────────────────────────────────────────────────────
Prompt eval          118.96 tok/s       111.40 tok/s    -6.4%
Decode (slot)        25.41 tok/s         25.46 tok/s    +0.2%
Generate (tg)        25.39 tok/s         29.09 tok/s   +14.6%
Generate (tg_3s)     25.65 tok/s         29.38 tok/s   +14.5%
Total time           10.38 s             10.68 s        +3.0%
Graphs reused         131                  50            -62%
Draft acceptance      –                    54.7%          –
Mean len              –                     2.64          –
```

**akeaway:** MTP speculative decoding delivers a **+14.6% decode speedup** (25.39 → 29.09 tok/s) on a 35B-A3B MoE model. The draft head accepts 55% of its predictions with a mean run of 2.64 tokens. Graphs reused drop 62% (131 → 50) because the draft context reuses fewer cached graph executions. Prompt eval takes a minor hit (-6.4%) due to the extra MTP context. Note that `--n-cpu-moe` also differs (28 vs 32) – the extra 4 CPU expert threads in the MTP run may contribute a small part of the gain.

## What the logs said – and what broke

- Both runs: W common_fit_params: failed to fit params to free device memory: n_gpu_layers already set by user to 999, abort – You forced -ngl 999 on a 12 GB card, so partial CPU offload is inevitable. The model is 35B at Q4 (~17.5 GB weights) + q8 KV at 262K (~12 GB+) – nowhere near fitting in 12 GB. Most MoE expert computations fall back to CPU via --n-cpu-moe.

- Both runs: W tensor overrides to CPU are used with mmap enabled – consider using --load-mode none for better performance – Some tensor overrides force CPU fallback. --load-mode none may improve performance by decoupling mmap.

- MTP run only: W device 'ROCm0' does not have support for op TOP_K needed for sampler 'top-k' – Relevant for the MTP draft sampler; may affect draft quality. The 54.7% acceptance rate might improve with a backend that supports top-k natively.

- MTP run only: I common_speculative_init_result: creating MTP draft context against the target model '...MTP-UD-Q4_K_XL.gguf' – MTP creates its own draft context which adds ~200ms to init time (visible in the slightly higher prompt eval time).

- gent difference: --n-cpu-moe 28 (non-MTP) vs --n-cpu-moe 32 (MTP) – this 4-thread difference is a confound. Re-running both with the same value would isolate the MTP-only gain.

## Bottom line

On an RX 6800M 12GB at 262K q8 KV, the **MTP draft head provides a meaningful +14.6% decode speedup** on this 35B-A3B MoE model, raising decode from 25.4 to 29.1 tok/s. The overhead is minimal (3% more total time, 6% slower prompt eval). Given that `--n-cpu-moe` also differed, the actual MTP-only gain is likely slightly lower – call it **~10-12% net**. If your backend supports top-k for the MTP sampler, acceptance rates may improve further. For chat workloads (decode-bound), MTP is worth enabling on any MoE model that ships a draft head.

🧪 More Lab

Next: re-run both with identical `--n-cpu-moe` and `--load-mode none` to isolate pure MTP gain, then test on Vulkan backend where top-k support differs.

[Back to VelsTech Lab →](https://velstech.net/lab)

---

*VelsTech – technology explained for everyone. Original: https://velstech.net/tiel-coder-35b-mtp-rx6800m*