How do I find specific UI elements to click?

You can use the 'see' command to generate a JSON map of the UI, providing you with specific element IDs and labels that can be targeted for clicks or moves.

What is the Peekaboo Mac Automation skill?

It is a specialized capability for Claude Code that uses the Peekaboo CLI to automate macOS applications through accessibility APIs and vision-based detection.

Can this skill interact with any Mac application?

Yes, as long as the application is accessible via the macOS Accessibility API or can be identified through vision-based detection, Peekaboo can interact with it.

Does it support screenshot verification?

Yes, the skill includes 'image' and 'see' commands that allow Claude to capture the current state of the screen for visual verification of automation steps.

Which AI model is best for the Peekaboo agent?

The gemini-3-flash model is recommended for the agent feature because it offers the best balance of speed, reliability, and simple credential management.

Peekaboo Mac Automation

Name: Peekaboo Mac Automation
Author: mshuffett

bymshuffett

0•

生产力与工作流

Automates macOS applications and UI interactions using a combination of accessibility APIs and vision-based element detection.

Peekaboo Mac Automation empowers Claude to interact directly with the macOS graphical user interface, bridging the gap between command-line logic and desktop software. By leveraging the Peekaboo CLI, this skill enables Claude to switch between applications, capture screenshots, and perform precise UI manipulations such as clicking labels, typing text, and scrolling. Its most powerful feature is an autonomous AI agent capable of executing complex, multi-step workflows across various Mac apps, making it an essential tool for local system orchestration, automated testing, and streamlining repetitive desktop tasks.

主要功能

01Vision-based and accessibility-driven UI element detection

02Comprehensive input simulation including clicks, typing, and key presses

030 GitHub stars

04Coordinate-based and ID-based element targeting for high accuracy

05Precise application switching with verification

06Autonomous AI agent for executing complex multi-step UI tasks

使用场景

01Orchestrating data transfers and configurations between multiple desktop apps

02Automating repetitive manual workflows across native macOS desktop applications

03Performing end-to-end UI testing for Mac software without formal APIs

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add mshuffett/dotfiles peekaboo-automation

For use in Claude.ai and ChatGPT

Download Skill