Native Devtools

概要

Native Devtools provides a Model Context Protocol (MCP) server that extends AI agent capabilities to native desktop applications on macOS and Windows. It acts as the 'eyes' and 'hands' for AI agents, enabling them to capture screenshots, perform OCR, simulate mouse clicks and keyboard typing, manage windows, and use template matching for non-textual elements. All operations are processed locally, ensuring privacy and allowing AI models like Claude or Cursor to seamlessly interact with any desktop application, facilitating advanced desktop automation, testing, and RPA workflows.

主な機能

• Computer Vision: Capture screenshots of screens, windows, or regions with built-in OCR for text recognition.
• Template Matching: Find and interact with non-textual UI elements (icons, shapes) using image templates.
• Local & Private: Executes 100% locally with no data or screenshots ever sent to external servers.
• Input Simulation: Simulate mouse clicks, drags, scrolls, and keyboard typing for natural interaction.
• Window Management: List open windows, find applications, and bring them into focus for targeted automation.
• 25 GitHub stars

ユースケース

• Enabling AI agents (e.g., Claude, Cursor) to automate and interact with native desktop applications.
• Performing end-to-end (E2E) and UI test automation for desktop software.
• Implementing Robotic Process Automation (RPA) workflows on macOS and Windows platforms.