Microsoft Releases 'Skill Recorder' to Convert Screen Recordings into AI Agent Skills

Skill Recorder is a tool that generates structured skills (SKILL.md) and automations executable by AI agents, based on user screen recording data.

Unlike existing RPA, which reproduces coordinate-based clicks and is vulnerable to environmental changes, this tool adopts an interpretation approach to the recorded content. Instead of click coordinates, it records semantic actions such as 'search' or 'information extraction' using the agent's native tools like gh CLI or web_fetch. This enables the creation of generalized skills applicable to various scenarios from just a single example.

Key Features and How It Works:

• Data Composition: Records screen transitions, visited page URLs, clipboard previews, and optional voice descriptions.
• Local-Centric Processing: Recording, storage, frame extraction, and Whisper-based voice transcription are all performed on the user's computer (locally) to enhance security.
• Analysis and Generation: Extracted data is sent to the GitHub cloud only when the user presses 'Analyze', where it is reconstructed into step-by-step procedures via GitHub Copilot.
• User Review: Reconstructed steps can be directly edited or reordered by the user, preventing incorrect procedures from becoming fixed as skills.

Technical Specifications and License:

• License: Released under the MIT license, allowing free personal and commercial use.
• Supported Environments: Primarily supports macOS, with support for Windows 11 (x64/ARM64).
• Prerequisites: Requires a GitHub account with GitHub Copilot enabled.