AI BriefingGoogle LiteRT-LM - High-Performance LLM Inference Framework for Edge Devices
Key point
Google has released LiteRT-LM, its on-device LLM inference engine.
Details
Google has released LiteRT-LM, a production-grade on-device LLM inference engine.
It's designed to run large language models in edge environments such as Android, iOS, web, desktop, and IoT (Raspberry Pi), and boosts inference performance with GPU/NPU hardware acceleration.
Key features include the following.
- Added Gemma 4 support
- Multimodal input support: vision (images), audio
- Image-attached inference available via the
--attachmentoption in the CLI - Built-in Function Calling (Tool Use) for agentic workflows
- Compatible with various models including Gemma, Llama, Phi-4, Qwen
- Can be run immediately with
litert-lm runafteruv tool install litert-lm
It's also actually being applied in Google products.
- On-device GenAI built into Chrome, Chromebook Plus, and Pixel Watch
- The Google AI Edge Gallery app lets you run models instantly on mobile
Supported languages are Kotlin, Python, and C++, with Swift support in progress.
As for release status, v0.10.2 is the latest, with Gemma 4 and the CLI introduced in v0.10.1. Before that, desktop GPU and multimodal were added in v0.8.0, and NPU acceleration in v0.7.0.
The license is Apache-2.0.
This summary was generated automatically by AI. Check the original for the author's claims and context. Copyright belongs to the original author.
Our guide explains how the AI works. Report summary errors, attribution issues, or removal requests via Contact.