What is the ML Rigor skill for Claude Code?

ML Rigor is a specialized skill that provides domain-specific guidance and code patterns to ensure machine learning projects follow scientific best practices like baseline testing and cross-validation.

Why is baseline comparison required in this skill?

Baseline comparison ensures that a complex model is actually providing value compared to simple strategies like majority-class prediction or basic linear regression.

Can I use this with existing Python ML libraries?

Yes, the skill is designed to work seamlessly with industry-standard libraries including Scikit-Learn, SHAP, Pandas, and SciPy.

What kind of model interpretation does it support?

The skill provides templates and logic for both global and local interpretability, specifically using SHAP values and permutation importance to explain model behavior.

How does this skill help with data leakage?

It implements strict quality gates and code patterns that ensure training and testing data are properly separated, preventing future information from influencing model training.

ML Rigor

Name: ML Rigor
Author: Yeachan-Heo

byYeachan-Heo

•

120

•

Data Science & ML

Enforces machine learning best practices including baseline comparisons, cross-validation, model interpretation, and data leakage prevention.

The ml-rigor skill transforms Claude into a disciplined data scientist by ensuring every machine learning pipeline meets rigorous scientific standards. It enforces critical quality gates such as beating dummy baselines, reporting variance through cross-validation, and providing deep model interpretability through SHAP or permutation importance. This skill is essential for researchers and engineers who need to move beyond exploratory notebooks to production-ready, statistically sound models while preventing common pitfalls like data leakage, overfitting, and poorly calibrated probabilities.

Key Features

01Advanced model interpretability using SHAP values and permutation importance

02Nested cross-validation and Bayesian optimization for unbiased hyperparameter tuning

03120 GitHub stars

04Probability calibration checks with Brier score reporting and isotonic regression

05Standardized stratified K-fold cross-validation with statistical confidence intervals

06Automated baseline comparisons against dummy and simple linear models

Use Cases

01Implementing statistically rigorous cross-validation for small datasets to report reliable metrics

02Validating a new model's performance against baseline heuristics to ensure meaningful learning

03Generating explainable AI reports to justify model predictions to stakeholders using SHAP values

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add yeachan-heo/my-jogyo ml-rigor

For use in Claude.ai and ChatGPT

Download Skill