Repo Crawler

About

Repo Crawler is an MCP server designed to transform GitHub repositories into structured intelligence for AI agents. It addresses the challenge of AI agents needing deep understanding of code repositories, beyond just file content, by exposing GitHub's entire data surface as structured tools. This includes comprehensive details like repository metadata, file trees, languages, commits, contributors, issues, pull requests, traffic, security alerts (Dependabot, code scanning, secret scanning), and Software Bill of Materials (SBOMs). The tool streamlines data extraction, handling API quotas, context window limitations, and rate limiting through a multi-tiered, section-selective, and gracefully degrading fetching mechanism.

Key Features

• 5 dedicated MCP tools: crawl repos, crawl orgs, summarize, compare, and export data.
• 3-tier data model to fetch repository information from fundamental overview to deep security analysis.
• Section-selective fetching to only retrieve requested data, optimizing API quota usage.
• Graceful degradation and built-in rate limiting ensure robust data collection even with API errors or high load.
• Extensible adapter pattern, currently supporting GitHub, designed for future integration with other platforms like GitLab or Bitbucket.
• 0 GitHub stars

Use Cases

• Perform quick triage of a repository by generating a human-readable summary with minimal API calls.
• Conduct deep security audits of repositories to identify vulnerabilities, dependency alerts, and leaked secrets.
• Compare multiple repositories side-by-side based on aspects like stars, languages, activity, and community health.