Vercel Labs open-sources agent-desktop for AI control of desktop apps
Vercel Labs has open-sourced agent-desktop, a command-line tool designed to let AI agents control desktop applications across macOS, Windows, and Linux. Instead of relying on screenshots, the tool reads application interfaces through the operating system’s accessibility tree and provides AI agents with JSON snapshots of interactive elements. According to the source, this allows developers to use any coding agent with command-line capabilities to drive native apps without depending on vision models. The tool runs in headless mode by default, meaning it does not move the mouse pointer or steal focus, so users can continue using their computers while an AI agent operates in parallel. MarkTechPost said the tool’s skeleton traversal feature can cut token usage by 78% to 96% in dense Electron applications. Version 1.0.2 was released on Oct. 30, 2026, under the Apache-2.0 license.
