RLOO Policy Optimization

About

RLOO is a specialized reinforcement learning skill for Claude Code that focuses on variance reduction during policy optimization. By using a leave-one-out baseline estimation—where multiple completions are generated per prompt and compared against each other—it provides significantly more stable gradients than traditional single-sample RL methods. This skill is particularly effective for training reasoning-capable models (such as Qwen-Thinking) and includes pre-defined patterns for reward function integration, thinking-aware token boundaries, and memory-efficient training configurations using Unsloth and TRL.

Key Features

• Optimized hyperparameters for training reasoning and chain-of-thought models
• Thinking-aware reward functions with token-based boundary detection
• Leave-One-Out baseline estimation for superior variance reduction
• Seamless integration with Unsloth for memory-efficient 4-bit and bf16 training
• Standardized RLOOTrainer patterns for stable policy optimization
• 0 GitHub stars

Use Cases

• Training LLMs to follow complex logical reasoning and chain-of-thought patterns
• Stabilizing RL training sessions where standard methods like GRPO exhibit high variance
• Optimizing model behavior through multi-completion reward-based scoring