AI Briefing
KO

llama.cpp adds support for Tencent's Hy3 model

·2026.07.14 11:46

Key point

MTP-based speculative decoding and support for Hy3, Tencent's 299B MoE model, have been added to llama.cpp.

Details

Support for Tencent's Hy3 (hy_v3 / HYV3ForCausalLM) model has been added to llama.cpp. This model has a 299B-parameter MoE (Mixture-of-Experts) architecture.

Key technical features include the following:

  • Consists of 80 layers and 1 MTP (Multi-Token Prediction) layer
  • Support for Speculative Decoding leveraging the MTP head

This update lays the groundwork for efficiently running inference on Hy3, a large-scale MoE model, within the llama.cpp environment.

inferencemoellama.cppspeculative-decodinghy3tencent
Read originalShare on Threads

This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.

Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.

Related

TLDR AITencent Releases 'Hy4 Preview', a 770B Parameter Open-Weight LLMPyTorchKRFreeToken Serves 284B MoE Model on RTX 5090RedditNew Feature Update for ik_llama.cppRedditOpen-Weight CLM Released as Jev Alternative, Agent Decision-Making 4x FasterKakaoKakao Improves Valid Play Rate by 20% with MoE-Based Personalized Ranking Model Using GDCNHacker NewsStep 5 Preview Released: 600B Parameter MoE Model Highlights Agentic Tasks and Financial Analysis Performance