iphone-use launches as open-source tool for AI agents to control real iPhones

iphone-use has launched as an open-source (MIT) tool that enables AI agents to drive real iPhones, specifically targeting applications without public APIs such as banking, payment, and health apps. The system provides the screen as text and supports tap, type, and scroll actions through WebDriverAgent.

Core Functionality and Feedback

Unlike standard automation, iphone-use provides agents with actionable feedback for every command, returning statuses such as applied, not sent, or unknown. This allows agents to branch logic based on success rather than assuming silent execution. For apps that block screenshots (e.g., payment interfaces), the tool returns a wireframe representation instead.

Performance and Integration

The tool includes an HTTP API and a 21-tool MCP server compatible with Claude Code, allowing remote control from a browser or iOS app. A key optimization reduced the tap-and-settle time from 8.5 s to 4.2 s by timing every request. Tasks completed once can be recorded as flows that replay without consuming model tokens.

Requirements

Running iphone-use requires a Mac with Xcode and the iPhone connected via USB.