A screen recording can show what someone clicked, but not necessarily why—or how to repeat the task when the next form, page, or data set differs. The useful shift here is from replaying a demonstration to reconstructing its intent, then turning that intent into an agent procedure that can generalize beyond the original session.
What Sets It Apart
- Grounded in observed work: App switches, window titles, browser URLs, screen images, and optional narration provide evidence for an ordered account of the task. This can make the resulting procedure more faithful than instructions written from memory.
- A skill, not a macro: The generated procedure can favor an agent’s native tools, such as command-line or web-fetch capabilities, instead of repeating screen coordinates. That makes it better suited to variations in pages and inputs, though the agent still needs appropriate tools and access.
- Human approval stays in the loop: Users can review and revise the reconstructed steps and proposed plan before creating a skill. A preview can also expose the generated instructions, but it is not a security guarantee.
- Recording is local; analysis is not: Captured data stays on the device during recording, but choosing Analyze sends selected timeline data, images, narration, and requested terminal excerpts to GitHub’s Copilot cloud. That boundary matters for sensitive workflows.
Who It’s For
Great fit if you repeatedly perform a workflow and want to teach a Copilot-powered agent to handle similar cases without relying on brittle UI playback. Look elsewhere if the work cannot be sent to cloud analysis, requires a different agent ecosystem, or must be fully automated without reviewing the generated procedure. It also depends on GitHub Copilot access, and its primary target is macOS, with Windows 11 support.
