Gated Delta Net on AMD: why ROCm stumbles on hybrid-attention LLMs

I was benchmarking Qwen3.8 27B on my RX 6800M when a llama.cpp warning explained a 2.6× speed crash I could not otherwise account for: fused Gated Delta Net (chunked) not supported. That one line is a preview of the biggest software gap AMD is going to face as the next generation of language models ships – and most people won't see it coming because the model still runs. It just runs slowly.

The warning that started it all

I ran IQ3_S a second time, this time letting --fit choose the GPU/CPU split instead of forcing -ngl 999. Decode collapsed from 19.0 to 7.3 tok/s. The log said why:

W resolve_fused_ops: layer 0 is assigned to device CPU but fused
  Gated Delta Net (chunked) is assigned to device ROCm0
W fused Gated Delta Net (chunked) not supported, set to disabled

Two facts hide in those lines. First: Qwen3.8 is not a plain transformer – it mixes standard attention layers with Gated Delta Net (GDN) layers. Second: llama.cpp has a fast fused kernel for GDN on CUDA, but nothing equivalent on ROCm. When the memory-fit algorithm touched the GDN layer at all, it had to fall back – and the fallback is catastrophically slow.

What Gated Delta Net is, in plain language

Standard attention compares every new token against every previous token. That history is the KV cache – the thing that grows with context length and eats your VRAM. Linear attention says: forget the exact history, keep a fixed-size running summary instead. Cost per token stops growing with context; memory becomes constant.

GDN is the current best-known recipe for that summary. Two ideas stacked:

• Delta rule – when new information arrives, don't just add it to memory; overwrite the part of the summary it contradicts, like a diff instead of an append log.
• Gating – a learned per-channel volume knob deciding how much of the update to apply, so the layer can also just… hold state when nothing changes.

Used alone, linear attention forgets details. So every serious new model is a hybrid: roughly three GDN layers for every one full-attention layer. The attention layers keep exact recall; the GDN layers do the cheap bulk of sequence mixing. Qwen3-Next proved the design; Qwen3.8 brings it to the 27B size class; Kimi Linear, MiniMax's lightning attention, Falcon-H1 and Nemotron-H (Mamba-2, same family of idea) are all racing in the same direction.

Why this bites AMD users specifically

A GDN layer is a recurrent state update. Run naively, token-by-token, it is sequential – and GPUs hate sequential. The whole point of the "chunked" fused kernel is to reformulate the recurrence so it runs in parallel blocks. NVIDIA users got that kernel early. On ROCm, llama.cpp's fused-op table simply doesn't have it yet – so the layer either runs as an unfused fallback or, as in my run, gets pushed to CPU.

What that cost me, same model, same card, one flag different:

Config                 Layers on GPU    Decode        Context
──────────────────────────────────────────────────────────────
-ngl 999 (forced)      all              18.97 tok/s   4K
--fit (auto)           all but layer 0  7.30 tok/s    4K

A 62% penalty for moving one layer's worth of hybrid machinery to the CPU – the GPU then stalls waiting on it every single token. The lesson generalises badly: on ROCm today, a hybrid model is only as fast as its worst-placed linear-attention layer.

Is your model affected? Three checks

1. Read the startup log. Any resolve_fused_ops or "not supported, set to disabled" line mentioning GDN, Mamba, SSM or "recurrent" means you have a hybrid and ROCm is missing its fast path.
2. Check the architecture. llama-gguf --metadata model.gguf (or any GGUF reader) shows general.architecture – qwen3next-style keys, linear_attention layer types, or ssm* fields are the tell.
3. Benchmark both configs. Force -ngl 999 and compare against auto-fit. If forcing wins, the fit algorithm is placing a hybrid layer where your backend can't run it fast.

What to do today

• On AMD, force -ngl 999 for hybrid models and accept the smaller context – it beat auto-fit by 2.6× in my test.
• Buy VRAM headroom with quant, not context. IQ2_XS left room for 52K tokens; IQ3_S left room for almost nothing. On a 12 GB card the 2 bpw quant is the hybrid-friendly one.
• Full-attention models are still the safe pick on ROCm. Until the fused kernels land, a dense 27B behaves predictably; a hybrid 27B has a cliff in it.
• Watch llama.cpp's ROCm backend. The fused-op table is growing; this gap is a "when", not an "if". Re-test after big merges.

Bottom line

The architecture layer of the LLM stack is moving from "bigger KV cache" to "smarter fixed-size state", and Gated Delta Net is the current winner. AMD's hardware is fine – its kernel coverage for these new ops is not, and llama.cpp will only tell you with a single quiet warning line at startup. If you run local AI on Radeon, start reading that line now: the models of 2026 are hybrids whether you're ready or not.