Apple Research Proposes Online RL Framework for UI Navigation Agents on Live iOS Apps

Details

Apple researchers propose an online reinforcement learning framework that trains UI navigation agents directly on live iOS applications to overcome the brittleness of supervised fine-tuning and the high costs of frontier vision-language models.
The 8B RL agent achieves nearly a sevenfold improvement over the SFT baseline in out-of-distribution generalization to Android environments.
On real iOS applications, the agent outperforms Gemini-3.5-Flash and ties with Gemini-3.1-Pro.
The framework decouples the agent into a Planning agent and a UI Grounding model, optimized using Proximal Policy Optimization (PPO) with step-level advantage estimation.
An LLM-as-a-Judge provides dense, step-level feedback and terminal rewards, while a novel method called GiGPPO improves critic stability by leveraging reward signals from visually similar states across concurrent trajectories.
The system relies on infrastructure that orchestrates hundreds of concurrent virtual iOS devices to support online training at scale.
Evaluations were conducted on UINavBench, iOSWorld, and AndroidWorld benchmarks.

Key point