Does it support multi-class classification?

Absolutely. It provides configurations for macro, micro, and weighted averaging for multi-class metrics, as well as one-vs-all ROC calculations.

Can I visualize model performance with this skill?

Yes, it provides code patterns for generating high-quality visualizations including ROC curves, Precision-Recall curves, Gain/Lift charts, and Calibration plots using autoplot.

Can this skill handle imbalanced datasets?

Yes, it includes specific patterns for PR AUC (Precision-Recall) and f_meas (F1-score), as well as threshold optimization tools to improve performance on minority classes.

Which R packages does this skill support?

This skill primarily leverages the yardstick, probably, and tidyposterior packages, which are all core components of the Tidymodels ecosystem for R.

How does it compare multiple models?

It uses workflow_set and tidyposterior patterns to perform statistical comparisons across resamples, allowing you to rank models based on mean performance and standard error.

Tidymodels Model Evaluation

Name: Tidymodels Model Evaluation
Author: choxos

bychoxos

データサイエンスとML

Evaluates machine learning model performance using R's yardstick and tidymodels ecosystem for robust classification and regression analysis.

概要

This skill provides a comprehensive toolkit for assessing predictive models within the R Tidyverse framework. It streamlines the generation of performance indicators for classification (ROC AUC, PR AUC, F1-score) and regression (RMSE, MAE, R-squared), while offering advanced techniques like probability calibration via the probably package, threshold optimization for imbalanced data, and statistical model comparison using tidyposterior. It is an essential utility for data scientists looking to implement rigorous validation patterns, quantify uncertainty with bootstrap intervals, and visualize model quality through professional-grade diagnostic plots.

主な機能

Statistical model comparison using resampled data and bootstrap intervals
0 GitHub stars
Comprehensive classification metrics including ROC AUC, PR AUC, and F1-score
Standard and robust regression metrics like RMSE, MAE, and Huber loss
Automated visualization for ROC curves, PR curves, and calibration plots
Advanced probability calibration and cost-sensitive threshold optimization

ユースケース

Validating binary and multi-class classification models for production readiness
Optimizing decision thresholds for imbalanced datasets or rare event detection
Comparing multiple machine learning workflows using statistical significance tests

概要

主な機能

Statistical model comparison using resampled data and bootstrap intervals
0 GitHub stars
Comprehensive classification metrics including ROC AUC, PR AUC, and F1-score
Standard and robust regression metrics like RMSE, MAE, and Huber loss
Automated visualization for ROC curves, PR curves, and calibration plots
Advanced probability calibration and cost-sensitive threshold optimization

ユースケース

Validating binary and multi-class classification models for production readiness
Optimizing decision thresholds for imbalanced datasets or rare event detection
Comparing multiple machine learning workflows using statistical significance tests