AI Briefing#inference-engine
The latest AI and developer news about #inference-engine, with the original source and a short summary.
The latest AI and developer news about #inference-engine, with the original source and a short summary.
mistral.rs now supports Gemma 4 12B and introduces multimodal and agentic capabilities.
mistral.rs v0.8.2 has been released, delivering up to 2.8x faster CUDA inference performance than llama.cpp on the latest GPUs such as H100 and B200.
Conifer, an open-source local inference engine for Apple Silicon, has been unveiled and is recruiting beta testers.
A template error in the Qwen3.5 Vision model has been fixed in vLLM.
ds4.c introduces a Metal-only inference engine and server built specifically for DeepSeek V4 Flash.
TokenSpeed outperformed TensorRT-LLM in B200-based LLM inference benchmarks for agents.
Atlas, a high-performance inference engine built on Rust and CUDA, has been released as open source.
MTPLX, an engine that leverages MTP heads to boost LLM inference speed by up to 2.24x on Apple Silicon, has been released.
Adding opt-in MMQ to HFQ4 prefill made Strix Halo speed more than 3x faster.
Hipfire, a new inference engine optimized across AMD GPUs, along with the mq4 quantization method, has been released.
They released new benchmarks and a harness tailored to the multi-turn, tool-calling patterns of agentic agents.
This explains how to run Claude Code in a local environment using LM Studio's new CLI and Gemma 4.
Cloudflare Workers AI has added Kimi K2.5, supporting a 256k context window and tool calling.
HuggingFace has announced a roadmap to establish the Transformers library as the model definition standard across the entire AI ecosystem.