Which editors support the eval-audit skill?

It is compatible with Claude Code, Cursor, VS Code, Windsurf, and any client that supports the agent skills standard.

Can it fix the issues it identifies automatically?

While it doesn't apply fixes itself, it provides a 'Handoff map' with the exact commands and skills needed to resolve every identified gap.

Do I need Truesight to use this skill?

Yes, this skill is specifically designed to work with the Truesight MCP ecosystem to analyze evidence like datasets, evaluation runs, and review items.

What does the eval-audit skill do?

It analyzes your current LLM evaluation setup to find weaknesses in coverage, error analysis, and review processes, providing a prioritized list of improvements.

How does it prioritize findings?

It uses a standardized severity rubric ranging from 'Critical' for issues causing incorrect go/no-go decisions to 'Low' for documentation or ergonomic optimizations.

LLM Evaluation Audit

Name: LLM Evaluation Audit
Author: Goodeye-Labs

byGoodeye-Labs

•

数据科学与机器学习

Conducts comprehensive audits of LLM evaluation workflows to identify quality regressions and process maturity gaps.

The Eval Audit skill provides a structured framework for assessing the maturity and effectiveness of LLM performance monitoring and evaluation practices. By analyzing existing datasets, live evaluations, and review loops within the Truesight ecosystem, it identifies critical gaps in failure taxonomies, coverage, and operational hygiene. It guides users through an interactive diagnostic process to produce prioritized findings with severity rankings and actionable next steps, ensuring LLM applications remain reliable and high-performing throughout the development lifecycle.

主要功能

01Interactive diagnostic Q&A for scoping audit priorities and goals

023 GitHub stars

03Comprehensive health checks for LLM deployment patterns and promotion loops

04Evidence-based analysis of datasets, live evaluations, and review queues

05Severity-ranked findings categorized from critical to low impact

06Automated routing to specific remediation skills like error-analysis or trace review

使用场景

01Benchmarking the maturity of internal LLM evaluation and error analysis practices

02Inheriting an existing LLM evaluation setup and needing to assess its reliability

03Diagnosing sudden quality regressions in production LLM outputs

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add goodeye-labs/truesight-mcp-skills eval-audit

For use in Claude.ai and ChatGPT

主要功能

01Interactive diagnostic Q&A for scoping audit priorities and goals

023 GitHub stars

03Comprehensive health checks for LLM deployment patterns and promotion loops

04Evidence-based analysis of datasets, live evaluations, and review queues

05Severity-ranked findings categorized from critical to low impact

06Automated routing to specific remediation skills like error-analysis or trace review

使用场景

01Benchmarking the maturity of internal LLM evaluation and error analysis practices

02Inheriting an existing LLM evaluation setup and needing to assess its reliability

03Diagnosing sudden quality regressions in production LLM outputs

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add goodeye-labs/truesight-mcp-skills eval-audit

For use in Claude.ai and ChatGPT