Is Linux Computer Use compatible with Wayland?

No, it is strictly X11-only. Wayland sessions do not currently support the necessary capabilities for cross-application window capture or input synthesis via xdotool, which are core to its functionality.

How does it interact with applications on Linux?

It leverages AT-SPI 2 for reading accessibility information and identifying UI elements, combined with xdotool for input simulation, wmctrl for window management, and scrot for screen capturing.

What is Linux Computer Use?

Linux Computer Use is a tool that enables AI agents to interact with and automate tasks on Linux/X11 desktops, mimicking human computer use through a combination of AT-SPI and xdotool.

Which AI agents and platforms does it support?

It serves as an extension for Pi coding agents and an MCP server for Claude Code, OpenCode, and any other MCP-aware client, specifically designed for Linux/X11 environments.

Linux Computer Use

Name: Linux Computer Use
Author: tak-uukti

bytak-uukti

0•

開発者ツール

Enables AI agents to interact with Linux/X11 desktops, mirroring human computer use through AT-SPI and xdotool.

This tool provides a robust set of functionalities, allowing AI agents like Pi, Claude Code, and OpenCode to control and interpret actions within a Linux graphical environment. It acts as a lightweight bridge, utilizing native Linux APIs such as AT-SPI 2, xdotool, wmctrl, and scrot to offer capabilities like window enumeration, screen capturing, clicking, typing, and scrolling. Designed as a drop-in replacement for macOS equivalents, it focuses on efficiency with a minimal codebase and a curated set of essential tools, empowering AI agents to automate tasks and navigate applications on X11-based Linux systems.

主な機能

010 GitHub stars

02Highly efficient with a compact ~1.2k total LOC and a ~470-line Python bridge.

03Verified end-to-end compatibility with various GTK/Qt applications, including specific notes for Chrome/Firefox/Electron.

04Provides 8 core computer-use tools: list_windows, screenshot, click, type_text, set_text, keypress, scroll, and computer_actions.

05Integrates AT-SPI 2, xdotool, wmctrl, and scrot for comprehensive Linux/X11 interaction.

06Serves as an extension for Pi coding agents and an MCP server for Claude Code and OpenCode.

ユースケース

01Integrating advanced computer-use capabilities into Claude Code workflows for Linux application control.

02Enabling OpenCode agents to perform automated actions and navigate graphical user interfaces on Linux machines.

03Empowering Pi coding agents to interact with and automate tasks on Linux desktop environments.