Which forecasting models can this skill benchmark?

It compares four major Nixtla libraries: TimeGPT (Foundation Model), StatsForecast (Statistical), MLForecast (Machine Learning), and NeuralForecast (Deep Learning).

What metrics does the benchmark output provide?

The generated scripts provide a CSV and plots ranking models by MAE, RMSE, MAPE, sMAPE, and execution time (speed).

What data format is needed for benchmarking?

The skill requires a standard time series format, typically a CSV containing a timestamp column and a value column (and an optional unique ID for multi-series data).

Is a TimeGPT API key required to use this skill?

A key is only required if you want to include TimeGPT in the benchmark; the skill can generate scripts that exclusively test open-source models like StatsForecast and MLForecast if no key is provided.

Nixtla Model Benchmarker

Name: Nixtla Model Benchmarker
Author: intent-solutions-io

byintent-solutions-io

数据科学与机器学习

Generates comprehensive benchmarking pipelines to compare Nixtla's TimeGPT, StatsForecast, MLForecast, and NeuralForecast models on custom datasets.

关于

The Nixtla Model Benchmarker automates the evaluation of time series forecasting models by creating a standardized benchmarking environment. It allows developers to compare the performance of foundation models like TimeGPT against classical statistical methods, machine learning, and deep learning architectures on their specific datasets. By providing automated metric calculation—including RMSE, MAE, and sMAPE—alongside visualizations of accuracy versus execution speed, it simplifies the complex process of model selection for production-ready forecasting services.

主要功能

Automated comparison of TimeGPT, StatsForecast, MLForecast, and NeuralForecast.
Calculates multiple accuracy metrics including MAE, RMSE, MAPE, and sMAPE.
Customizable configuration for forecast horizons, data frequency, and seasonality.
Provides automated visualizations and CSV reports for model ranking.
Generates performance insights comparing training and inference execution times.
0 GitHub stars

使用场景

Evaluating whether TimeGPT's zero-shot accuracy justifies API costs compared to local models.
Selecting the optimal forecasting model for large-scale production sales data.
Benchmarking the performance gains of deep learning models over traditional statistical baselines.

关于

主要功能

Automated comparison of TimeGPT, StatsForecast, MLForecast, and NeuralForecast.
Calculates multiple accuracy metrics including MAE, RMSE, MAPE, and sMAPE.
Customizable configuration for forecast horizons, data frequency, and seasonality.
Provides automated visualizations and CSV reports for model ranking.
Generates performance insights comparing training and inference execution times.
0 GitHub stars

使用场景

Evaluating whether TimeGPT's zero-shot accuracy justifies API costs compared to local models.
Selecting the optimal forecasting model for large-scale production sales data.
Benchmarking the performance gains of deep learning models over traditional statistical baselines.