How does this skill handle CUDA Out of Memory (OOM) errors?

It provides systematic approaches including reducing batch sizes, implementing gradient checkpointing, using mixed precision training (AMP), and clearing the CUDA cache to optimize memory footprint.

How does it assist with tensor shape mismatches in complex layers?

The skill suggests debugging wrappers that print shapes at every layer, using einops for clearer transformations, and programmatic dimension calculations for convolutional and linear layers.

Does it support debugging multi-GPU and device-mismatch issues?

Absolutely. It includes patterns for explicit device movement, checking parameter locations, and using map_location during model state loading to prevent device conflict errors.

Can it help me find exactly where a NaN value is entering my model?

Yes, it guides the setup of torch.autograd.set_detect_anomaly(True) and provides custom check_nan functions to monitor tensors during the forward and backward passes.

PyTorch Debugger

Name: PyTorch Debugger
Author: SnakeO

bySnakeO

Ciencia de Datos y ML

Troubleshoots and resolves complex PyTorch issues including CUDA memory errors, tensor shape mismatches, and gradient instabilities.

Acerca de

The PyTorch Debugger skill equips Claude with systematic strategies and implementation patterns to diagnose and fix deep learning workflow bottlenecks. It provides specialized guidance for handling high-pressure scenarios like CUDA Out of Memory (OOM) errors, numerical instability (NaN losses), and complex autograd graph issues. By leveraging this skill, developers can quickly implement memory profiling, gradient clipping, and shape assertions, significantly reducing the iteration time required to build and train production-ready neural networks.

Características Principales

0 GitHub stars
CUDA Out of Memory (OOM) mitigation via gradient checkpointing and mixed precision.
Automated tensor shape validation and layer dimension mismatch troubleshooting.
Cross-device management for seamless CPU/GPU data transfers and model loading.
Detection and resolution of numerical instabilities like NaN losses or exploding gradients.
DataLoader optimization and autograd graph debugging for complex architectures.

Casos de Uso

Resolving training crashes caused by GPU memory limitations or leaks.
Refactoring model code to be device-agnostic and compatible with multiple GPU configurations.
Isolating the specific layer or operation causing numerical instability during backpropagation.

Acerca de

Características Principales

0 GitHub stars
CUDA Out of Memory (OOM) mitigation via gradient checkpointing and mixed precision.
Automated tensor shape validation and layer dimension mismatch troubleshooting.
Cross-device management for seamless CPU/GPU data transfers and model loading.
Detection and resolution of numerical instabilities like NaN losses or exploding gradients.
DataLoader optimization and autograd graph debugging for complex architectures.

Casos de Uso

Resolving training crashes caused by GPU memory limitations or leaks.
Refactoring model code to be device-agnostic and compatible with multiple GPU configurations.
Isolating the specific layer or operation causing numerical instability during backpropagation.