# Ternary-Bonsai-2-27B on RX 6800M: PTQ1_0 vs PQ2_0 – VelsTech Lab

> Tested Ternary-Bonsai-2-27B ternary quants on a 12GB RX 6800M – stock llama.cpp rejects type 143/142, Prism fork + gfx1031 source build fixes it, PQ2_0 decodes 32.8 tok/s vs PTQ1_0 18.8 tok/s.

*Source: https://velstech.net/ternary-bonsai-2-27b-rx6800m.ta (Tamil translation of https://velstech.net/ternary-bonsai-2-27b-rx6800m) · Updated: 2026-09-19*

*Markdown version. [Read the interactive guide](https://velstech.net/ternary-bonsai-2-27b-rx6800m.ta). English Markdown: https://velstech.net/ternary-bonsai-2-27b-rx6800m.md.*

---

🧪 VelsTech Lab 12 GB mobile GPU-வில் 27B ternary model இயங்குமா? RX 6800M-ல் **Ternary-Bonsai-2-27B** – `PTQ1_0` (1.75 bits/weight, 5.95 GB) and `PQ2_0` (2.13 bits/weight, 7.21 GB) – இரு packing-களையும் சோதித்தேன். Stock llama.cpp இரு கோப்புகளையும் நிராகரிக்கிறது, Prism prebuilt header-ஐ ஏற்றி ROCm-ல் நிற்கிறது, `gfx1031` க்கான source build மட்டுமே வேலை செய்கிறது. Build செய்த பிறகு, `PQ2_0` இந்த card-ல் கிட்டத்தட்ட 2× வேகமானது.

🔧 Check your own fit first

[LLM VRAM Calculator](https://velstech.net/llm-vram-calculator) – does 27B ternary + 32K context fit your VRAM? · [GPU AI Performance Calculator](https://velstech.net/gpu-ai-calculator) – what tok/s to expect before you run.

[Open calculators →](https://velstech.net/llm-vram-calculator)

## சோதனை அமைப்பு

- Machine: ASUS ROG Strix G513QY · Ryzen 9 5900HX (8C/16T) · AMD RX 6800M 12GB (RDNA2 gfx1031, mobile) · 30 GB RAM · NVMe

- OS / stack: Ubuntu · ROCm driver 7.1.3 / HIP 7.15 · LD_LIBRARY_PATH=/opt/rocm/lib

- Stock build (fails): ggml-org/llama.cpp 5f436dddb, GGML_HIP=ON, Release

- Working build: PrismML-Eng/llama.cpp branch prism, b10709-9a9394a89, built with -DGGML_HIP=ON -DCMAKE_HIP_ARCHITECTURES=gfx1031

- Model A: Ternary-Bonsai-2-27B-PTQ1_0.gguf – 5946648928 bytes, qwen35 arch, ternary g128 {−1,0,+1} + FP16 scales, Hadamard block 1024

- Model B: Ternary-Bonsai-2-27B-PQ2_0.gguf – 7206168928 bytes, same architecture, trit-in-2-bit packing

- Both runs: -ngl 99 -fa on -c 32768 --temp 1.0 --top-p 0.95 --top-k 20 -n 256

## எப்படி இயக்கினோம்

### Run 0 – stock llama.cpp (fails)

```
export LD_LIBRARY_PATH=/opt/rocm/lib:$LD_LIBRARY_PATH
llama-cli \
 -m /home/velsrocky/models/Ternary-Bonsai-2-27B-PTQ1_0.gguf \
 -ngl 99 -fa on -c 32768 \
 --temp 1.0 --top-p 0.95 --top-k 20 \
 -p "Explain quantum computing in simple terms." -n 256
```

```
gguf_init_from_reader: tensor 'output.weight' has invalid ggml type 143.
llama_model_load: error loading model: failed to load model from
  /home/velsrocky/models/Ternary-Bonsai-2-27B-PTQ1_0.gguf
llama_server exited with code 1
```

### Run 1 – Prism prebuilt ROCm 7.2 (loads header, dies on GPU)

```
~/llama-prism/llama-cli -m .../Ternary-Bonsai-2-27B-PTQ1_0.gguf \
 -ngl 99 -fa on -c 32768 ...
```

```
ROCm error: device kernel image is invalid
ggml_cuda_kernel_launch at ggml/src/ggml-cuda/common.cuh:1715
hipGetLastError()
```

Even `-ngl 0` fails on this binary – HIP backend init probes device 0 and aborts.

### Run 2 – Prism fork built for gfx1031 (works)

```
git clone -b prism https://github.com/PrismML-Eng/llama.cpp.git ~/llama-prism-src
cmake -B build -DCMAKE_BUILD_TYPE=Release \
 -DGGML_HIP=ON -DCMAKE_HIP_ARCHITECTURES=gfx1031
cmake --build build -j$(nproc)
```

```
export LD_LIBRARY_PATH=$HOME/llama-prism-src/build/bin:/opt/rocm/lib:$LD_LIBRARY_PATH
~/llama-prism-src/build/bin/llama-cli \
 -m /home/velsrocky/models/Ternary-Bonsai-2-27B-PQ2_0.gguf \
 -ngl 99 -fa on -c 32768 \
 --temp 1.0 --top-p 0.95 --top-k 20 \
 -p "Explain quantum computing in simple terms." -n 256
```

```
build : b10709-9a9394a89
ftype : PQ2_0 - 2.13 bpw (group 128)
[ Prompt: 64.2 t/s | Generation: 32.8 t/s ]
```

```
build : b10709-9a9394a89
ftype : PTQ1_0 - 1.75 bpw ternary (group 128)
[ Prompt: 22.5 t/s | Generation: 18.8 t/s ]
```

## அளவீடுகள்

```
Metric               PQ2_0               PTQ1_0            Delta
──────────────────────────────────────────────────────────────
File size            7.21 GB             5.95 GB           -17%
True bits/weight     2.13                1.75              -18%
Prompt eval          64.2 tok/s          22.5 tok/s        +185% (PQ2_0 faster)
Decode (gen)         32.8 tok/s          18.8 tok/s        +74%
Context              32768               32768             same
Backend              HIP gfx1031         HIP gfx1031       same
```

**Takeaway:** on this RDNA2 card `PQ2_0` wins everywhere – prompt processing is compute-bound and dense-trit unpacking costs arithmetic, so the 2-bit-slot packing is ~2.9× faster at prompt eval and ~1.7× faster at decode. `PTQ1_0` only wins on footprint: 1.3 GB less weight traffic, the pick when 32K+ KV plus projector must squeeze into 12 GB. This matches Prism's own guidance – PQ2_0 is preferred on HIP/CUDA/Metal/CPU.

## Logs சொன்னது – உடைந்தது என்ன

- Type 143 / 142: GGUF dump shows general.architecture = qwen35, general.file_type = 143, prism.hadamard.* rotation, output.weight type=143 (PTQ1_0) / 142 (PQ2_0). Upstream ggml only defines types 0-42 – these are fork packings with a Hadamard activation transform. Stock llama.cpp has no such runtime, so it refuses them. The files are complete, not truncated.

- Kernel image invalid: the prism-b10709 ROCm 7.2 prebuilt targets datacenter gfx (CDNA), not gfx1031. HIP init fails before any token is generated. Fix is a source build with -DCMAKE_HIP_ARCHITECTURES=gfx1031 – same flag pattern as the working upstream HIP build. See the ROCm & Vulkan guide and llama.cpp guide.

- Q2_0 trap: per Prism, never substitute a legacy *-Q2_0.gguf here – stock llama.cpp loads it without warning and outputs garbage because it skips the Hadamard transform. Use PQ2_0 / PTQ1_0 only with fork binaries.

## முடிவு

On a 12 GB RX 6800M, **use the Prism fork built for gfx1031 and run PQ2_0**: ~33 tok/s decode at 32K context. Keep `PTQ1_0` for the tightest VRAM budgets. Do not debug type 143/142 as a download error – it is a backend mismatch by design. The authoritative setup is [Bonsai-demo](https://github.com/PrismML-Eng/Bonsai-demo) (`setup.sh` + `start_llama_server.sh`), which pins the right binary per backend.

🧪 More Lab

Related: [How much VRAM does an LLM need?](https://velstech.net/how-much-vram-for-llm) · [Quantization deep dive](https://velstech.net/quantization-deep-dive) · [Qwen3.8 27B GSQ-RCO on the same card](https://velstech.net/qwen38-27b-gsq-rco-rx6800m) · [ROCm & Vulkan on AMD](https://velstech.net/rocm-vulkan-amd-guide).

[Back to VelsTech Lab →](https://velstech.net/lab)

---

*VelsTech – https://velstech.net/ternary-bonsai-2-27b-rx6800m.ta.md*