What visualization outputs are provided?

The skill generates a bar chart (metrics_visualization.png) that provides a side-by-side visual comparison of historical versus current model performance.

Does this skill require a Nixtla API key to run?

No, this specific validation skill operates locally on your provided CSV metrics files and does not require an active Nixtla API key.

What file format is required for input data?

The skill requires CSV files with specific columns: 'model', 'MASE', and 'sMAPE' for both historical and current metrics.

Can I customize the sensitivity of the alerts?

Yes, you can configure custom thresholds for MASE and sMAPE (e.g., 0.1 for 10% or 0.3 for 30%) to control how strictly the skill flags performance drops.

What metrics does this skill use to validate forecasts?

It primarily uses Mean Absolute Scaled Error (MASE) and symmetric Mean Absolute Percentage Error (sMAPE) to assess accuracy and detect degradation.

Nixtla Forecast Validator

Name: Nixtla Forecast Validator
Author: intent-solutions-io

byintent-solutions-io

데이터 과학 및 ML

Validates time series forecast accuracy by comparing current performance against historical benchmarks to detect model degradation.

소개

The Nixtla Forecast Validator is a specialized Claude Code skill designed to monitor and maintain the quality of time series forecasting models. It automates the comparison of current MASE and sMAPE metrics against historical baselines, allowing developers and data scientists to identify performance regressions instantly. By generating detailed validation reports, alert logs, and visual comparison charts, this skill helps teams maintain high-accuracy forecasting pipelines and proactively address concept drift or data quality issues before they impact business decisions.

주요 기능

Automated MASE and sMAPE metric comparison against baselines
0 GitHub stars
Configurable degradation thresholds for proactive alerting
Generation of visual performance charts and bar graphs
Detection of model drift and forecast quality regressions
Comprehensive validation reports and structured alert logs

사용 사례

Monitoring production forecasting models for performance decay
Benchmarking new time series models against historical baselines
Automating quality gates in data science CI/CD pipelines

소개

주요 기능

Automated MASE and sMAPE metric comparison against baselines
0 GitHub stars
Configurable degradation thresholds for proactive alerting
Generation of visual performance charts and bar graphs
Detection of model drift and forecast quality regressions
Comprehensive validation reports and structured alert logs

사용 사례

Monitoring production forecasting models for performance decay
Benchmarking new time series models against historical baselines
Automating quality gates in data science CI/CD pipelines