tldraw Releases Real-Time Collaborative Canvas Plugin for ChatGPT and Codex

tldraw has released a real-time collaborative canvas plugin that allows humans and AI to jointly modify visual elements within ChatGPT and Codex. Instead of describing screen layouts in text, it passes board drawings, selections, and annotations as work context to enhance AI understanding.

MCP-Based Interaction

Utilizing OpenAI's MCP extension, it maintains an interactive screen in the sidebar. The tldraw MCP server offers various features, including board creation, opening, searching, reading canvas screenshots and shapes, and drawing Mermaid diagrams. Information about open boards, selections, and connected users is also passed to the model, enabling actual board editing based on natural language requests or the generation of visual materials for development tasks.

Use Cases and Connectivity

Tasks such as requesting Codex to implement an app based on a hand-drawn wireframe, or marking modification locations on an app screenshot to request code changes, are possible. It connects with a free tldraw.com account, and changes are saved to the account and synchronized between ChatGPT, Codex, and the web editor. Team members and their respective agents can share boards and exchange work via comments at specific locations.

Limitations

The plugin's target pricing plans, separate pricing, concurrent editing user limits, and quantitative performance metrics have not been confirmed. Connecting a free tldraw account does not imply exemption from ChatGPT or Codex usage fees.