Manages the automated curation and multi-agent validation of high-quality datasets for LLM evaluation.
The Golden Dataset Curator is a sophisticated workflow tool designed to transform raw URLs and documents into high-fidelity evaluation data. It utilizes a 9-phase process featuring four specialized agents—Reviewer, Architect, Engineer, and Generator—to perform deep quality analysis across dimensions like accuracy, coherence, and relevance. The skill includes built-in bias detection, diversity tracking, and a unique 'Silver-to-Gold' promotion system, ensuring your testing benchmarks remain objective, diverse, and production-ready with full semantic versioning support.
主な機能
01Multi-agent parallel quality analysis with detailed score explanations
02Diversity dashboard to track domain and difficulty distribution
03Two-tier Silver and Gold dataset classification system
04Automated bias detection for technology, recency, and vendor neutrality
05Semantic version tracking for dataset additions and schema changes
0669 GitHub stars
ユースケース
01Maintaining version-controlled documentation benchmarks for development teams
02Automating the collection of diverse test cases for LLM fine-tuning
03Building high-quality ground-truth datasets for RAG evaluation
