Which forecasting models can this skill benchmark?

It compares four major Nixtla libraries: TimeGPT (Foundation Model), StatsForecast (Statistical), MLForecast (Machine Learning), and NeuralForecast (Deep Learning).

What metrics does the benchmark output provide?

The generated scripts provide a CSV and plots ranking models by MAE, RMSE, MAPE, sMAPE, and execution time (speed).

What data format is needed for benchmarking?

The skill requires a standard time series format, typically a CSV containing a timestamp column and a value column (and an optional unique ID for multi-series data).

Is a TimeGPT API key required to use this skill?

A key is only required if you want to include TimeGPT in the benchmark; the skill can generate scripts that exclusively test open-source models like StatsForecast and MLForecast if no key is provided.

Nixtla Model Benchmarker

Name: Nixtla Model Benchmarker
Author: intent-solutions-io

byintent-solutions-io

0•

データサイエンスとML

Generates comprehensive benchmarking pipelines to compare Nixtla's TimeGPT, StatsForecast, MLForecast, and NeuralForecast models on custom datasets.

The Nixtla Model Benchmarker automates the evaluation of time series forecasting models by creating a standardized benchmarking environment. It allows developers to compare the performance of foundation models like TimeGPT against classical statistical methods, machine learning, and deep learning architectures on their specific datasets. By providing automated metric calculation—including RMSE, MAE, and sMAPE—alongside visualizations of accuracy versus execution speed, it simplifies the complex process of model selection for production-ready forecasting services.

主な機能

01Automated comparison of TimeGPT, StatsForecast, MLForecast, and NeuralForecast.

02Calculates multiple accuracy metrics including MAE, RMSE, MAPE, and sMAPE.

03Customizable configuration for forecast horizons, data frequency, and seasonality.

04Provides automated visualizations and CSV reports for model ranking.

05Generates performance insights comparing training and inference execution times.

060 GitHub stars

ユースケース

01Evaluating whether TimeGPT's zero-shot accuracy justifies API costs compared to local models.

02Selecting the optimal forecasting model for large-scale production sales data.

03Benchmarking the performance gains of deep learning models over traditional statistical baselines.

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add intent-solutions-io/plugins-nixtla nixtla-model-benchmarker

For use in Claude.ai and ChatGPT

Download Skill