Evaluating Skills with Models

关于

This skill automates the comparative testing of agentic AI skills by executing them across the Claude model family, including Sonnet, Opus, and Haiku. It utilizes a sophisticated weighting system that moves beyond binary pass/fail metrics to identify specific model pitfalls and calculate quality-based scores. By running scenarios in parallel via sub-agents, it helps developers determine production readiness, identify the most cost-effective compatible model for specific tasks, and document historical performance directly within the repository's README.

主要功能

• Automated README documentation with historical performance tables
• Difficulty-based scenario prioritization for realistic testing
• 0 GitHub stars
• Parallel execution across Claude Sonnet, Opus, and Haiku models
• Quality-based weighted scoring system (0-100 scale)
• Detection of model-specific pitfalls like over-engineering or shallow reasoning

使用场景

• Regression testing skills during development to ensure consistent behavior across model updates
• Determining if a newly developed AI skill is ready for production deployment
• Comparing model performance to find the cheapest model that meets quality requirements