From RTX to Spark: NVIDIA Accelerates Gemma 4 for Local Agentic AI

Gemma 4 is Google's latest open model family, designed for the local AI trend of leveraging real-time context on-device rather than in the cloud. NVIDIA and Google have optimized it for NVIDIA GPUs across the board, enabling it to run broadly from data centers to RTX PCs, DGX Spark, and Jetson Orin Nano.

The model family consists of E2B, E4B, 26B, 31B, with roles divided by use case.

• E2B / E4B: Focused on ultra-low-latency inference at the edge and fully offline execution
• 26B / 31B: Suited for high-performance inference, developer-centric workflows, and agentic AI
• Supported features: reasoning, coding, function calling, vision/video/audio, interleaved multimodal input, support for 35+ languages

Performance comparisons were measured under Q4_K_M quantization, BS=1, ISL=4096, OSL=128 conditions, benchmarked against GeForce RTX 5090 and Mac M3 Ultra desktops. Token generation throughput was measured using the llama-bench tool from llama.cpp b7789.

Local deployment is supported together by Ollama, llama.cpp, and Unsloth. NVIDIA emphasizes that its Tensor Core and CUDA stack improve inference throughput and latency, allowing Gemma 4 to scale from the edge to high-performance PCs without the burden of additional optimization.

The announcement also mentions RTX AI PC ecosystem examples such as always-on local agents leveraging OpenClaw, NemoClaw, and Accomplish FREE, showing that open-model-based local agentic AI is rapidly growing.