How does ds-verify confirm reproducibility?

It performs multiple runs of the analysis using fixed random seeds and compares the output hashes to ensure that identical inputs consistently produce identical results.

When should I use the ds-verify skill?

It should be used whenever you need to validate data science output, check reproducibility, or as the final Phase 5 of a standard data science workflow.

What does the User Acceptance Interview involve?

It is a structured step where the AI asks the user about methodology constraints, whether the results answer the core question, and if the output format is acceptable.

What is the 'Iron Law of DS Verification'?

The Iron Law dictates that no completion claim can be made without a fresh, non-cached re-run of the analysis to prove that results are reproducible and accurate.

Can this skill handle Jupyter Notebooks?

Yes, it includes specific protocols for clearing and re-running notebooks via nbconvert or papermill to ensure the execution order is verified from scratch.

Data Science Verification & Reproducibility

Name: Data Science Verification & Reproducibility
Author: edwinhu

byedwinhu

データサイエンスとML

Enforces rigorous technical validation and user acceptance testing for data science workflows to ensure reproducible results.

概要

ds-verify serves as the essential quality gate for data science projects, specifically designed to eliminate 'hallucinated' completion claims and cached result bias. It mandates a strict protocol of fresh code execution, automated reproducibility demonstrations using hashing and random seeds, and structured user acceptance interviews. By generating standardized verification reports that document the environment and methodology constraints, it ensures that every analysis is both technically sound and aligned with stakeholder objectives before a project is marked as finished.

主な機能

0 GitHub stars
Standardized Verification Reporting: Generates detailed reports covering environment, hashes, and audit trails.
Reproducibility Demonstration: Automates hash-based comparisons across multiple runs with fixed seeds.
User Acceptance Interview (UAI): Facilitates a structured dialogue to confirm results meet business needs.
Fresh Execution Protocol: Forces re-running analysis to bypass cached or stale results.
Technical Sanity Checks: Validates output formats, row counts, and data integrity throughout the pipeline.

ユースケース

Verifying that complex data transformations haven't caused silent data loss or corruption.
Finalizing machine learning pipelines where reproducibility is critical for model governance.
Performing a formal sign-off on exploratory data analysis before presenting to stakeholders.

概要

主な機能

0 GitHub stars
Standardized Verification Reporting: Generates detailed reports covering environment, hashes, and audit trails.
Reproducibility Demonstration: Automates hash-based comparisons across multiple runs with fixed seeds.
User Acceptance Interview (UAI): Facilitates a structured dialogue to confirm results meet business needs.
Fresh Execution Protocol: Forces re-running analysis to bypass cached or stale results.
Technical Sanity Checks: Validates output formats, row counts, and data integrity throughout the pipeline.

ユースケース

Verifying that complex data transformations haven't caused silent data loss or corruption.
Finalizing machine learning pipelines where reproducibility is critical for model governance.
Performing a formal sign-off on exploratory data analysis before presenting to stakeholders.