What triggers an anti-pattern flag during evaluation?

Anti-pattern flags are triggered by issues such as over-constrained language (excessive MUST/ALWAYS/NEVER), short or missing trigger descriptions, bloated single-file SKILL.md files over 800 lines, and broken reference links.

How does the evaluation methodology calculate composite scores?

It computes a weighted sum across ten dimensions—giving the highest weights to triggering accuracy, orchestration fitness, and output quality—and applies an anti-pattern penalty multiplier.

What are the three evaluation layers in PluginEval?

PluginEval uses Layer 1 for fast, deterministic static analysis, Layer 2 for LLM-judged rubric scoring, and Layer 3 for statistical Monte Carlo simulations measuring activation and failure rates.

How are quality badges awarded to plugins?

Badges (Bronze, Silver, Gold, Platinum) are awarded when a plugin meets composite score thresholds (from 60 to 90+) and corresponding Elo ranking minimums against the gold corpus.

Evaluation Methodology - Claude Code Skill

The Plugin Evaluation Methodology skill provides a comprehensive, three-layer framework for measuring and calibrating the quality of AI agent plugins and skills. By integrating deterministic static analysis, multi-judge LLM evaluations, and Monte Carlo simulations, it scores skills across ten distinct dimensions—including triggering accuracy, orchestration fitness, and token efficiency. Whether you are debugging low performance metrics, avoiding anti-pattern flags, tuning marketplace badge thresholds, or benchmarking against gold-corpus Elo ratings, this skill delivers authoritative guidance and actionable remediation paths for high-reliability agent engineering.

Key Features

01Automated anti-pattern detection for token bloat, missing triggers, and broken links

02Elo ranking system with confidence intervals calibrated against a standard gold corpus

03Three-layer evaluation combining static checks, LLM judges, and Monte Carlo testing

04Comprehensive scoring across 10 distinct quality dimensions with weighted composite formulas

05Quality badge certification tiers ranging from Bronze to Platinum production readiness

0639,277 GitHub stars

Use Cases

01Optimizing skill descriptions and triggers to improve autonomous invocation accuracy

02Auditing and diagnosing low dimension scores in agent skills or marketplace plugins

03Calibrating quality badge thresholds and Elo benchmarks for plugin marketplaces

Tieline — product intent grounded in code

Plugin Evaluation Methodology

Key Features

Use Cases

Security Review

Quarkus Security Review

Repo Scan

Key Features

Use Cases