What R packages does this skill primarily use?

The skill utilizes the yardstick package for metrics, probably for calibration and thresholding, and the broader tidymodels framework for workflow integration.

How does this skill handle regression model assessment?

It provides code patterns for standard metrics like RMSE and R-squared, as well as robust alternatives like MAE, Huber loss, and Concordance Correlation Coefficients.

How do I visualize performance results?

The skill provides patterns for generating ggplot2-based visualizations using autoplot() for ROC curves, precision-recall curves, and calibration plots.

Does it support probability calibration?

Yes, it includes implementation patterns for Platt scaling (logistic calibration), isotonic regression, and beta calibration to ensure predicted probabilities match actual frequencies.

Can I use this for multi-class classification tasks?

Yes, it includes specific patterns for multi-class evaluation, including macro, micro, and weighted averaging for F1 scores and one-vs-all ROC AUC.

R Tidy Modeling Evaluation

Name: R Tidy Modeling Evaluation
Author: choxos

bychoxos

0•

Data Science & ML

Evaluates machine learning models in R using yardstick and tidymodels patterns for classification and regression tasks.

This skill provides a comprehensive framework for model evaluation within the R tidymodels ecosystem, specifically leveraging the yardstick and probably packages. It offers standardized patterns for calculating classification and regression metrics, generating diagnostic visualizations like ROC and calibration curves, and performing advanced tasks such as probability calibration and threshold optimization. Whether you are comparing multiple resampled models or assessing prediction uncertainty through conformal intervals, this skill streamlines the assessment phase of the data science workflow to ensure robust and reliable results.

Key Features

01Advanced probability calibration and threshold optimization using probably

02Comprehensive classification and regression metric calculation via yardstick

03Statistical model comparison patterns for resampled results

04Uncertainty quantification through bootstrap and conformal prediction intervals

05Automated diagnostic visualizations for ROC, PR, and lift curves

060 GitHub stars

Use Cases

01Assessing binary or multi-class classification performance on imbalanced datasets

02Comparing performance across multiple regression models using cross-validation

03Optimizing classification thresholds for cost-sensitive clinical or business outcomes

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add choxos/biostatagent model-evaluation

For use in Claude.ai and ChatGPT

Download Skill