How does it handle hyperparameter tuning?

It utilizes RandomizedSearchCV and Nested Cross-Validation to provide unbiased performance estimates and prevent overfitting to the validation set during the tuning process.

Can this skill help with model explainability?

Yes, it includes built-in templates for calculating permutation importance and SHAP values to help you understand which features drive your model's decisions.

Why is baseline comparison mandatory in this skill?

Baseline comparisons prove your model is learning useful patterns by showing it can beat simple strategies like a majority class classifier or a simple linear model.

What does ML Rigor do for my machine learning projects?

It enforces scientific standards like baseline comparisons and cross-validation to ensure your models are genuinely effective and not just overfitting to noise.

ML Rigor

Name: ML Rigor
Author: Yeachan-Heo

byYeachan-Heo

•

120

Ciencia de Datos y ML

Enforces scientific best practices in machine learning pipelines through baseline comparison, cross-validation, and model interpretation.

Acerca de

The ML Rigor skill provides a comprehensive framework for building robust, scientifically sound machine learning models within the Claude Code environment. It automates the implementation of rigorous testing patterns, including mandatory baseline comparisons against dummy models, stratified cross-validation to assess performance variance, and data leakage prevention. By integrating interpretation tools like SHAP values and permutation importance, it ensures models are not just accurate but also explainable, moving your ML findings from 'exploratory' to production-grade research.

Características Principales

Automated baseline comparisons against dummy and linear models
Stratified and Repeated Cross-Validation with confidence interval reporting
Model interpretation through SHAP values and permutation importance
Unbiased hyperparameter tuning using Nested Cross-Validation
120 GitHub stars
Probability calibration checks and Brier score evaluation

Casos de Uso

Detecting overfitting and data leakage in early-stage model development
Validating that a complex model significantly outperforms simple domain heuristics
Generating standardized, rigorous performance reports for scientific peer review

Acerca de

Características Principales

Automated baseline comparisons against dummy and linear models
Stratified and Repeated Cross-Validation with confidence interval reporting
Model interpretation through SHAP values and permutation importance
Unbiased hyperparameter tuning using Nested Cross-Validation
120 GitHub stars
Probability calibration checks and Brier score evaluation

Casos de Uso

Detecting overfitting and data leakage in early-stage model development
Validating that a complex model significantly outperforms simple domain heuristics
Generating standardized, rigorous performance reports for scientific peer review