ik_llama.cpp Adds Native CUDA and CPU Support for PrismML Ternary Bonsai 2

·2026.11.01 00:01

Key point

Recent commits resolve Hadamard sign tensor bottlenecks, allowing full GPU offload and fast CPU prompt ingestion for PQ2_0 models.

ik_llama.cpp now provides complete native support for PrismML's Ternary Bonsai 2, eliminating previous bottlenecks caused by Hadamard sign tensor handling in other forks.
Native CUDA kernels for the PQ2_0 format are merged via PR #2691 and PR #2747, enabling full layer offload to Nvidia GPUs without token corruption or fallback to heavy k-quants.
A new script (PR #2694) allows lossless repacking of upstream TQ2_0 or TQ1_0 weights to the PQ2_0 format in seconds, preserving exact trits without requantization.
CPU inference utilizes an 8-row interleaved format (PQ2_0_R8) with iqk_mul_mat kernels; passing the -rtr 1 flag drastically accelerates prompt processing.

This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.

Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.