Does this skill require an active Nixtla API key?

No, this specific skill operates on local CSV metrics files and does not require an external API connection for the validation process.

What specific metrics does this skill track?

The skill focuses on MASE (Mean Absolute Scaled Error) and sMAPE (symmetric Mean Absolute Percentage Error) to assess forecast accuracy.

Can I customize the sensitivity of the alerts?

Yes, you can define custom thresholds (e.g., 0.1 for 10% or 0.3 for 30%) to determine what level of metric increase triggers a warning.

What kind of reports does the validator produce?

It generates a summary validation_report.txt, a metrics_comparison.csv for raw data, an alert.log for issues, and a PNG visualization chart.

Nixtla Forecast Validator

Name: Nixtla Forecast Validator
Author: intent-solutions-io

byintent-solutions-io

0•

数据科学与机器学习

Validates time series forecast quality and detects performance degradation by comparing current metrics against historical benchmarks.

The Nixtla Forecast Validator is a specialized Claude Code skill designed to monitor and maintain the reliability of time series models. By analyzing key error metrics like MASE and sMAPE against established baselines, it automatically identifies significant performance drops that could indicate data drift, quality issues, or model decay. This tool is essential for data scientists and ML engineers who need to ensure their automated forecasting pipelines remain accurate and reliable over time without manual oversight.

主要功能

01Automated detection of MASE and sMAPE metric degradation

02Comparison of current performance against historical CSV benchmarks

03Seamless integration with Nixtla-based TimeGPT pipelines

04Configurable validation thresholds for sensitive or lenient alerting

05Generation of visual performance charts and detailed text reports

060 GitHub stars

使用场景

01Automated accuracy reporting for stakeholders and business reviews

02Monitoring production forecasting pipelines for model drift

03Benchmarking experimental model iterations against baseline performance

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add intent-solutions-io/plugins-nixtla nixtla-forecast-validator

For use in Claude.ai and ChatGPT

Download Skill