What is the recommended reward scale for PPO trading?

A reward scale of 0.001 is recommended to ensure that reward volatility reflects realistic proxy equity changes, typically resulting in a 5-15% MaxDD range.

Why is my MaxDD showing 80% when my model is profitable?

This is often a 'phantom' metric caused by an aggressive reward scale (like 0.1) treating raw rewards as massive equity shifts; reducing the scale to 0.001 aligns metrics with realistic volatility.

Does this skill help with double-penalization in rewards?

Yes, it includes guidance on reducing drawdown penalty weights when drawdown is already present in the model's observation space, preventing overly conservative behavior.

How does this skill change model gating thresholds?

It provides recalibrated 'Approved' and 'Review' thresholds for Profit Factor, Consistency, and Fitness to match the more conservative metrics produced by the new scaling.

Reward Scaling Calibration

Name: Reward Scaling Calibration
Author: smith6jt-cop

bysmith6jt-cop

データサイエンスとML

Calibrates reinforcement learning reward scales to eliminate unrealistic drawdown metrics in algorithmic trading models.

概要

This skill provides a specialized workflow for fine-tuning reward scaling in GPU-native PPO training environments, specifically addressing the 'phantom MaxDD' issue. It helps developers correct scenarios where aggressive scaling causes simulated equity to fluctuate wildly, leading to misleading validation results despite high model performance. By recalibrating the reward scale and adjusting model gating thresholds, this skill ensures that metrics like Maximum Drawdown accurately reflect training stability and reward volatility, allowing for more reliable model selection and evaluation in financial contexts.

主な機能

Updates ModelGatingConfig thresholds for fitness, PF, and consistency
Optimizes per-step equity clamping to prevent extreme mathematical swings
Recalibrates reward scaling to 0.001 for realistic equity simulation
Fixes unrealistic 35-80% Maximum Drawdown (MaxDD) validation values
Adjusts reward weights to prevent double-penalizing drawdown observations
0 GitHub stars

ユースケース

Correcting misleading performance metrics in PPO-based trading bot training
Optimizing GPU-native reinforcement learning environments for finance
Fine-tuning model gating configurations for automated strategy selection

概要

主な機能

Updates ModelGatingConfig thresholds for fitness, PF, and consistency
Optimizes per-step equity clamping to prevent extreme mathematical swings
Recalibrates reward scaling to 0.001 for realistic equity simulation
Fixes unrealistic 35-80% Maximum Drawdown (MaxDD) validation values
Adjusts reward weights to prevent double-penalizing drawdown observations
0 GitHub stars

ユースケース

Correcting misleading performance metrics in PPO-based trading bot training
Optimizing GPU-native reinforcement learning environments for finance
Fine-tuning model gating configurations for automated strategy selection