Text-Only Web Control Tool Released

A tool called Web Draw has been released, allowing control of real web browsers using text-only AI models without visual processing capabilities. Existing browser automation tools required vision models or incurred high token costs due to their reliance on screenshot analysis, but this tool removes these constraints by rendering pages as text.

Core Working Principles and Features

• Text-Based Rendering: Assigns stable handles (e.g., e18, e29) to all controls on a web page and converts them into text. The model performs tasks through an 'observe-handle manipulation-reobserve' loop.
• Small Model Support: Designed to be fully usable with text-only models of 7B or 8B size. An Amazon search page is compressed to approximately 750 tokens, and a checkout page to approximately 550 tokens, ensuring efficiency.
• Ambiguity Elimination: To reduce errors in small models, overlapping elements are flagged instead of clicked, ambiguous targets return candidate lists, and actual page responses are reported upon form submission failures.
• Security and Deployment: Uses the user's existing browser and login information as is, with communication restricted to 127.0.0.1. It is free, requires no account, and is available via the Chrome Web Store and MCP (Model Context Protocol) configuration.