Visually manage AI bot access to your website for optimal SEO control.
Sponsored
robotstxt-ai is a visual robots.txt manager designed for the AI era, empowering website owners to precisely control which AI crawlers access their content. It offers a user-friendly interface to block or allow over 20 specific AI bots, including GPTBot, ClaudeBot, and PerplexityBot, ensuring granular control over how AI models interact with your site. Beyond generation, the tool also allows instant analysis of existing robots.txt files and can be integrated with AI assistants like Claude Desktop or Cursor for enhanced SEO workflows.
主な機能
010 GitHub stars
02Generate custom robots.txt files
03Analyze Existing robots.txt files
04Database of 20+ AI Bots (GPTBot, ClaudeBot, etc.)
05MCP Server integration for AI assistants
06Visual Toggle UI for AI Bots
ユースケース
01Analyzing any website's robots.txt to verify AI bot blocking status.
02Blocking specific AI crawlers from accessing website content.
03Generating a tailored robots.txt file to manage bot interactions.
