Does this skill require an active Nixtla API key?

No, this specific skill operates on local CSV metrics files and does not require an external API connection for the validation process.

What specific metrics does this skill track?

The skill focuses on MASE (Mean Absolute Scaled Error) and sMAPE (symmetric Mean Absolute Percentage Error) to assess forecast accuracy.

Can I customize the sensitivity of the alerts?

Yes, you can define custom thresholds (e.g., 0.1 for 10% or 0.3 for 30%) to determine what level of metric increase triggers a warning.

What kind of reports does the validator produce?

It generates a summary validation_report.txt, a metrics_comparison.csv for raw data, an alert.log for issues, and a PNG visualization chart.

Nixtla Forecast Validator

Name: Nixtla Forecast Validator
Author: intent-solutions-io

byintent-solutions-io

データサイエンスとML

Validates time series forecast quality and detects performance degradation by comparing current metrics against historical benchmarks.

概要

The Nixtla Forecast Validator is a specialized Claude Code skill designed to monitor and maintain the reliability of time series models. By analyzing key error metrics like MASE and sMAPE against established baselines, it automatically identifies significant performance drops that could indicate data drift, quality issues, or model decay. This tool is essential for data scientists and ML engineers who need to ensure their automated forecasting pipelines remain accurate and reliable over time without manual oversight.

主な機能

Automated detection of MASE and sMAPE metric degradation
Comparison of current performance against historical CSV benchmarks
Seamless integration with Nixtla-based TimeGPT pipelines
Configurable validation thresholds for sensitive or lenient alerting
Generation of visual performance charts and detailed text reports
0 GitHub stars

ユースケース

Automated accuracy reporting for stakeholders and business reviews
Monitoring production forecasting pipelines for model drift
Benchmarking experimental model iterations against baseline performance

概要

主な機能

Automated detection of MASE and sMAPE metric degradation
Comparison of current performance against historical CSV benchmarks
Seamless integration with Nixtla-based TimeGPT pipelines
Configurable validation thresholds for sensitive or lenient alerting
Generation of visual performance charts and detailed text reports
0 GitHub stars

ユースケース

Automated accuracy reporting for stakeholders and business reviews
Monitoring production forecasting pipelines for model drift
Benchmarking experimental model iterations against baseline performance