Agent Performance Evaluation

概要

This skill provides comprehensive methodologies for assessing autonomous agent systems, moving beyond traditional software testing to address non-determinism and complex decision-making. It enables developers to implement LLM-as-judge frameworks, design outcome-focused rubrics across factual and process-oriented dimensions, and validate context engineering choices. By establishing systematic quality gates and performance benchmarks, it ensures agent pipelines remain reliable, catches regressions before deployment, and optimizes the balance between token usage and model performance.

主な機能

• 0 GitHub stars
• LLM-as-judge implementation for scalable automated testing
• Context engineering and degradation impact analysis
• Multi-dimensional rubric design for accuracy, completeness, and tool efficiency
• Complexity-stratified test set generation for diverse scenarios
• Continuous evaluation pipeline integration for production systems

ユースケース

• Validating agent quality improvements after context engineering updates
• Building automated quality gates for production-ready agent pipelines
• Benchmarking different model versions for specific autonomous tasks