Generates and validates YAML configuration files for benchmarking Model Context Protocol (MCP) servers.
The MCP Benchmark Configurator (mcpbr-config) is a specialized skill designed to streamline the setup of benchmarking environments for Model Context Protocol (MCP) servers. It automates the creation of valid YAML configurations, ensuring critical components like the {workdir} placeholder and specific runtime commands are correctly implemented. By providing templates for popular benchmarks like SWE-bench, CyberGym, and MCPToolBench++, this skill helps developers evaluate AI agent performance across diverse datasets while catching common configuration errors before execution.
主要功能
01Automatic verification of critical {workdir} placeholders
028 GitHub stars
03Automated YAML generation for mcpbr configurations
04Simplified model aliasing for Anthropic Claude models
05Pre-configured templates for SWE-bench and CyberGym
06Validation of runtime commands including npx, uvx, and python
使用场景
01Evaluating AI agent coding capabilities on the SWE-bench dataset
02Setting up performance benchmarks for custom MCP servers
03Configuring security and penetration testing scenarios via CyberGym
