What are behavioral contracts in agent testing?

Behavioral contracts define specific invariants that an agent must or must not do, such as 'never provide legal advice' or 'always cite sources.' This skill provides patterns to programmatically enforce these rules.

How does this skill handle agent latency and performance?

It includes statistical analysis patterns that track metrics like p95 latency and token usage, allowing developers to optimize agent speed and cost alongside accuracy.

Why is specialized testing needed for LLM agents?

Agents are stochastic by nature, meaning they can provide different outputs for the same input. Traditional unit tests are often insufficient; agents require statistical evaluation over multiple runs to measure reliability and consistency accurately.

Does this skill support industry-standard benchmarks?

Yes, it is designed to work with primary tools such as AgentBench for multi-environment testing, τ-bench for real-world scenarios, and ToolEmu for detecting risky tool use.

Agent Evaluation & Benchmarking

Name: Agent Evaluation & Benchmarking
Author: sickn33

bysickn33

•

31,721

•

Seguridad y Pruebas

Tests and benchmarks LLM agents through behavioral analysis, capability assessment, and reliability metrics to ensure production readiness.

This skill provides a comprehensive framework for evaluating the performance and safety of LLM agents, addressing the critical challenge where even top-tier agents often achieve low scores on real-world benchmarks. It enables developers to implement statistical test evaluation for stochastic behaviors, define strict behavioral contracts to prevent prohibited actions, and conduct adversarial testing to discover failure modes. By integrating industry-standard tools like AgentBench and Tau-bench, it offers a robust methodology for measuring agentic reliability, latency, and consistency, ensuring agents stay within operational bounds while performing complex tasks.

Características Principales

01Statistical distribution analysis for stochastic agent behaviors

02Production reliability metrics including p95 latency and consistency scores

0331,721 GitHub stars

04Integration with industry benchmarks like AgentBench and Tau-bench

05Adversarial testing patterns to identify prompt injection risks

06Behavioral contract testing with must/must-not assertions

Casos de Uso

01Validating customer service agents against safety and policy constraints

02Benchmarking agentic workflows against real-world task completion datasets

03Regression testing for agent updates to prevent behavioral drift

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add sickn33/antigravity-awesome-skills agent-evaluation

For use in Claude.ai and ChatGPT

Características Principales

01Statistical distribution analysis for stochastic agent behaviors

02Production reliability metrics including p95 latency and consistency scores

0331,721 GitHub stars

04Integration with industry benchmarks like AgentBench and Tau-bench

05Adversarial testing patterns to identify prompt injection risks

06Behavioral contract testing with must/must-not assertions

Casos de Uso

01Validating customer service agents against safety and policy constraints

02Benchmarking agentic workflows against real-world task completion datasets

03Regression testing for agent updates to prevent behavioral drift