AMD RDNA3-Optimized Inference Engine hipEngine

hipEngine is a new open-source (AGPLv3) local LLM inference engine optimized for the AMD ROCm environment.

While designed based on Python, its core computation (hot-path) is implemented in HIP/C++, leveraging AMD native libraries such as hipBLASLt, hipGraph, and AOTriton to maximize performance.

Benchmarks conducted on gfx1100 (Radeon RX 7900 XTX, etc.) targeting Qwen 3.6 (MoE and Dense) models confirmed superior Prefill (prompt processing) performance compared to llama.cpp.

In particular, when ParoQuant (4.68bpw) was applied, it recorded faster token processing speeds than llama.cpp (HIP and Vulkan) across various context lengths ranging from 512 to 128K.