What visualization outputs are provided?

The skill generates a bar chart (metrics_visualization.png) that provides a side-by-side visual comparison of historical versus current model performance.

Does this skill require a Nixtla API key to run?

No, this specific validation skill operates locally on your provided CSV metrics files and does not require an active Nixtla API key.

What file format is required for input data?

The skill requires CSV files with specific columns: 'model', 'MASE', and 'sMAPE' for both historical and current metrics.

Can I customize the sensitivity of the alerts?

Yes, you can configure custom thresholds for MASE and sMAPE (e.g., 0.1 for 10% or 0.3 for 30%) to control how strictly the skill flags performance drops.

What metrics does this skill use to validate forecasts?

It primarily uses Mean Absolute Scaled Error (MASE) and symmetric Mean Absolute Percentage Error (sMAPE) to assess accuracy and detect degradation.

Nixtla Forecast Validator

Name: Nixtla Forecast Validator
Author: intent-solutions-io

byintent-solutions-io

0•

데이터 과학 및 ML

Validates time series forecast accuracy by comparing current performance against historical benchmarks to detect model degradation.

The Nixtla Forecast Validator is a specialized Claude Code skill designed to monitor and maintain the quality of time series forecasting models. It automates the comparison of current MASE and sMAPE metrics against historical baselines, allowing developers and data scientists to identify performance regressions instantly. By generating detailed validation reports, alert logs, and visual comparison charts, this skill helps teams maintain high-accuracy forecasting pipelines and proactively address concept drift or data quality issues before they impact business decisions.

주요 기능

01Automated MASE and sMAPE metric comparison against baselines

020 GitHub stars

03Configurable degradation thresholds for proactive alerting

04Generation of visual performance charts and bar graphs

05Detection of model drift and forecast quality regressions

06Comprehensive validation reports and structured alert logs

사용 사례

01Monitoring production forecasting models for performance decay

02Benchmarking new time series models against historical baselines

03Automating quality gates in data science CI/CD pipelines

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add intent-solutions-io/plugins-nixtla nixtla-forecast-validator

For use in Claude.ai and ChatGPT

Download Skill