Helix Pilot
Empower AI agents to observe and control your Windows desktop using a local vision LLM.
helix-pilot is an MCP server that enables AI agents to seamlessly interact with your Windows desktop. It captures screenshots, leverages local Ollama Vision models for screen analysis, and executes mouse and keyboard actions, all while running entirely on your machine. This eliminates the need for expensive cloud APIs and ensures data privacy, offering a truly local and robust solution for GUI automation, especially beneficial for users of Claude Code, Codex CLI, and other MCP-compatible clients.
Key Features
01100% local operation with Ollama Vision LLMs for zero cloud API cost
02Windows-native desktop control via Win32 API, not a VM or container
03MCP-native integration for instant compatibility with various AI clients
040 GitHub stars
05Comprehensive built-in safety system including action policies and emergency stop
06Vision LLM-powered screen understanding for robust UI interaction
Use Cases
01Automating complex GUI tasks on Windows via AI agents like Claude Code or Codex CLI
02Enabling conversational AI models (e.g., via Open WebUI) to interact with the desktop
03Developing and deploying AI-driven desktop automation scripts without cloud dependencies