Which training environments is this compatible with?

While designed for GPU-native PPO training in vectorized environments, the reward logic can be applied to any RL framework where trading frequency is lower than desired.

What are the recommended slippage settings for crypto?

The skill suggests reducing slippage costs from 0.5% to 0.2% and lowering the slippage weight to 0.02 to prevent the penalty from overwhelming the learning signal.

What is HOLD bias in RL trading?

HOLD bias occurs when an RL model learns that 'doing nothing' is the most rational action because the costs of trading (like slippage) consistently outweigh the expected rewards, resulting in a model that never enters positions.

How does this skill fix low trade rates?

It implements a multi-step fix: reducing excessive slippage penalties, adding a small 'trading incentive' bonus for non-HOLD actions, and increasing the exploration bonus to encourage the model to find profitable signals.

RL Trading Reward Function Optimizer

Name: RL Trading Reward Function Optimizer
Author: smith6jt-cop

bysmith6jt-cop

Data Science & ML

Eliminates HOLD bias in reinforcement learning trading models by calibrating reward functions and slippage penalties.

About

This skill provides specialized logic and parameter configurations to fix the common 'HOLD bias' in Reinforcement Learning (RL) trading agents. It specifically addresses scenarios where models learn that staying out of the market is the only rational choice due to asymmetric payoffs, excessive slippage penalties, or insufficient exploration incentives. By adjusting trading incentives, direction thresholds, and rebalancing reward weights, it helps models identify profitable entry points and achieve realistic trade rates (30-60%) in volatile environments like BTCUSD.

Key Features

Multi-component reward weight rebalancing for PPO training
Automated calibration of slippage and transaction cost penalties
0 GitHub stars
Introduction of trading incentives to balance asymmetric payoffs
Directional threshold optimization to ensure profit exceeds costs
Dynamic exploration bonus scaling based on model uncertainty

Use Cases

Correcting models with low trade rates (<10%) in volatile markets
Fine-tuning reward functions to prevent 'rational' inactivity in RL agents
Optimizing GPU-native PPO training environments for crypto trading

About

Key Features

Multi-component reward weight rebalancing for PPO training
Automated calibration of slippage and transaction cost penalties
0 GitHub stars
Introduction of trading incentives to balance asymmetric payoffs
Directional threshold optimization to ensure profit exceeds costs
Dynamic exploration bonus scaling based on model uncertainty

Use Cases

Correcting models with low trade rates (<10%) in volatile markets
Fine-tuning reward functions to prevent 'rational' inactivity in RL agents
Optimizing GPU-native PPO training environments for crypto trading