What is the ML Rigor skill for Claude Code?

ML Rigor is a specialized skill that provides domain-specific guidance and code patterns to ensure machine learning projects follow scientific best practices like baseline testing and cross-validation.

Why is baseline comparison required in this skill?

Baseline comparison ensures that a complex model is actually providing value compared to simple strategies like majority-class prediction or basic linear regression.

Can I use this with existing Python ML libraries?

Yes, the skill is designed to work seamlessly with industry-standard libraries including Scikit-Learn, SHAP, Pandas, and SciPy.

What kind of model interpretation does it support?

The skill provides templates and logic for both global and local interpretability, specifically using SHAP values and permutation importance to explain model behavior.

How does this skill help with data leakage?

It implements strict quality gates and code patterns that ensure training and testing data are properly separated, preventing future information from influencing model training.

ML Rigor

Name: ML Rigor
Author: Yeachan-Heo

byYeachan-Heo

•

120

Data Science & ML

Enforces machine learning best practices including baseline comparisons, cross-validation, model interpretation, and data leakage prevention.

About

The ml-rigor skill transforms Claude into a disciplined data scientist by ensuring every machine learning pipeline meets rigorous scientific standards. It enforces critical quality gates such as beating dummy baselines, reporting variance through cross-validation, and providing deep model interpretability through SHAP or permutation importance. This skill is essential for researchers and engineers who need to move beyond exploratory notebooks to production-ready, statistically sound models while preventing common pitfalls like data leakage, overfitting, and poorly calibrated probabilities.

Key Features

Advanced model interpretability using SHAP values and permutation importance
Nested cross-validation and Bayesian optimization for unbiased hyperparameter tuning
120 GitHub stars
Probability calibration checks with Brier score reporting and isotonic regression
Standardized stratified K-fold cross-validation with statistical confidence intervals
Automated baseline comparisons against dummy and simple linear models

Use Cases

Implementing statistically rigorous cross-validation for small datasets to report reliable metrics
Validating a new model's performance against baseline heuristics to ensure meaningful learning
Generating explainable AI reports to justify model predictions to stakeholders using SHAP values

About

Key Features

Advanced model interpretability using SHAP values and permutation importance
Nested cross-validation and Bayesian optimization for unbiased hyperparameter tuning
120 GitHub stars
Probability calibration checks with Brier score reporting and isotonic regression
Standardized stratified K-fold cross-validation with statistical confidence intervals
Automated baseline comparisons against dummy and simple linear models

Use Cases

Implementing statistically rigorous cross-validation for small datasets to report reliable metrics
Validating a new model's performance against baseline heuristics to ensure meaningful learning
Generating explainable AI reports to justify model predictions to stakeholders using SHAP values