Why does this skill prefer JSON backups over SQL dumps?

JSON backups are human-readable, portable across different database versions, and allow for clear diffs when stored in Git version control.

How does it handle vector embeddings during a restore?

To ensure accuracy, embeddings are typically excluded from backups and regenerated during the restore process using the current production model.

Can I automate these backups in CI/CD?

Yes, the skill includes patterns for integrating backup and verification scripts into GitHub Actions or other automation pipelines.

What is a URL contract in data management?

It is a requirement that every analysis record stores a real, canonical URL rather than a placeholder, enabling content re-fetching and source validation.

What is a golden dataset in AI development?

A golden dataset is a curated, high-quality collection of examples used as a ground truth to measure model performance, retrieval accuracy, and regression in AI systems.

Golden Dataset Management

Name: Golden Dataset Management
Author: yonatangross

byyonatangross

•

데이터 과학 및 ML

Protects and maintains high-quality AI/ML test datasets through robust backup, restoration, and data integrity validation.

This skill provides production-ready patterns for managing 'golden datasets'—curated collections of high-quality examples used for evaluating RAG systems, benchmarking LLMs, and ensuring retrieval quality. It automates the lifecycle of critical test data by implementing JSON-based backups for version control, enforcing URL contracts to prevent placeholder data, and providing detailed disaster recovery workflows to maintain consistent evaluation environments across your AI development lifecycle.

주요 기능

01Regression detection for retrieval quality metrics

02JSON-based backup for seamless Git version control integration

0373 GitHub stars

04URL contract enforcement to ensure data provenance and auditability

05Automated embedding regeneration during dataset restoration

06Data integrity validation for document chunks and metadata

사용 사례

01Migrating high-quality test data between local, staging, and production environments

02Building and maintaining ground-truth datasets for RAG evaluation

03Implementing automated disaster recovery for AI/ML evaluation pipelines

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add yonatangross/orchestkit golden-dataset-management

For use in Claude.ai and ChatGPT

Download Skill