How do I generate an evaluation report?

The skill includes specialized Python scripts that process prediction and actual result files to generate comprehensive Markdown reports with tiered metric analysis.

What ML quality metrics are included?

It supports Spearman Rank Information Coefficient (IC), prediction autocorrelation, and specific diagnostics to detect if a model has collapsed to a mean prediction.

Why do range bars need special metrics?

Range bars are price-based rather than time-based, meaning they violate standard IID assumptions and require daily aggregation to avoid overstating metrics like the Sharpe ratio due to autocorrelation.

Does this skill handle crypto market calculations?

Yes, it includes specific logic for 24/7 crypto markets, using sqrt(7) for weekly and sqrt(365) for annual scaling, while providing separate rules for session-filtered data.

Range Bar Evaluation Metrics

Name: Range Bar Evaluation Metrics
Author: terrylica

byterrylica

•

数据科学与机器学习

Evaluates financial models using state-of-the-art metrics specifically designed for range bar (price-based sampling) data.

关于

This skill provides specialized tools and references for calculating performance and risk metrics in quantitative finance, specifically tailored for range bars which violate standard time-series assumptions. It automates the computation of daily-aggregated Sharpe ratios, Information Coefficients (IC), and Probabilistic Sharpe Ratios (PSR), ensuring accurate annualization for crypto and equity markets while diagnosing common pitfalls like model collapse in BiLSTM architectures. It is essential for quantitative researchers who need to validate strategy performance on non-temporal data sampling methods.

主要功能

Probabilistic and Deflated Sharpe Ratio (PSR/DSR) statistical validation
Crypto-specific annualization factors (sqrt(7) and sqrt(365))
Model collapse detection and BiLSTM architecture remediation
8 GitHub stars
Daily-aggregated Sharpe ratio calculation for non-IID price data
ML prediction quality diagnostics including Information Coefficient (IC)

使用场景

Validating quantitative research results to prevent statistical over-fitting
Evaluating deep learning model performance in 24/7 cryptocurrency markets
Backtesting algorithmic trading strategies on price-based sampling data

关于

主要功能

Probabilistic and Deflated Sharpe Ratio (PSR/DSR) statistical validation
Crypto-specific annualization factors (sqrt(7) and sqrt(365))
Model collapse detection and BiLSTM architecture remediation
8 GitHub stars
Daily-aggregated Sharpe ratio calculation for non-IID price data
ML prediction quality diagnostics including Information Coefficient (IC)

使用场景

Validating quantitative research results to prevent statistical over-fitting
Evaluating deep learning model performance in 24/7 cryptocurrency markets
Backtesting algorithmic trading strategies on price-based sampling data