VibeConsole

概要

VibeConsole acts as an MCP server, allowing AI agents to gain deep access and control over Proxmox virtual machines. It bridges the gap between AI and VM environments by providing both graphical (noVNC) and serial console access. AI agents can send keyboard and mouse inputs, capture high-quality screenshots, leverage enhanced OCR with grid overlays for spatial awareness, and even manage VM lifecycle actions like starting, stopping, and rebooting. This tool empowers AI to interact with VMs at a fundamental level, from boot processes to GUI applications, all without requiring any in-VM agents.

主な機能

• 0 GitHub stars
• AI Agent Control: Send keyboard input and capture real JPEG/PNG screenshots
• Enhanced OCR: Get text with precise bounding boxes and grid cell locations
• Grid-Based Clicking: Click at grid references (e.g., "K9") instead of pixel coordinates
• Console Access: Connect to VM consoles via noVNC through Proxmox API
• Serial Console Access: Connect to VM serial console (xterm.js backend) for raw text I/O

ユースケース

• Automating VM provisioning and configuration via AI agents, including early boot stages
• Developing AI systems that require direct, low-level access to VM consoles for testing or operational tasks
• Enabling AI agents to perform complex GUI interactions or debugging within virtual machines