How do I run experiments on large models like Llama-3 405B?

You can use the remote=True flag within the model.trace context to execute your interpretability code on NDIF's infrastructure without needing local GPUs.

Does nnsight support gradient analysis?

Yes, nnsight allows you to access and analyze gradients during the backward pass for local models, which is essential for attribution and sensitivity analysis.

Can I use nnsight with any PyTorch model?

Yes, nnsight is architecture-agnostic and works with any PyTorch model, including custom architectures, though it has built-in support for HuggingFace models.

nnsight is a specialized library for interpreting and manipulating PyTorch model internals, supporting both local execution and remote execution on massive models via NDIF.

What is the difference between nnsight and TransformerLens?

While TransformerLens provides a highly consistent API for a specific set of transformers, nnsight is more flexible across diverse architectures and uniquely supports massive-scale remote execution.

nnsight Model Interpretability

Name: nnsight Model Interpretability
Author: zechenzhangAGI

byzechenzhangAGI

•

384

데이터 과학 및 ML

Interprets and manipulates neural network internals for any PyTorch model, including massive foundation models via remote execution.

소개

nnsight (pronounced 'insight') provides a unified interface to probe and modify the inner workings of PyTorch models, bridging the gap between small local experiments and massive scale inference. It allows researchers to trace computation graphs, save activations, and perform interventions like activation patching on models ranging from GPT-2 to Llama-3-405B. By leveraging the National Deep Inference Facility (NDIF), users can run complex mechanistic interpretability experiments on industrial-scale models without requiring high-end local GPU hardware, making advanced AI research accessible to everyone.

주요 기능

Cross-prompt activation sharing and multi-token generation tracing
Activation patching and hidden state intervention capabilities
Unified API for any PyTorch architecture including Transformers and Mamba
Remote execution on massive models (70B, 405B) via NDIF
Deferred execution model for efficient computation graph manipulation
384 GitHub stars

사용 사례

Activation patching to identify circuit components within neural networks
Steering model behavior by intervening in hidden states during generation
Mechanistic interpretability research on foundation models too large for local memory

소개

주요 기능

Cross-prompt activation sharing and multi-token generation tracing
Activation patching and hidden state intervention capabilities
Unified API for any PyTorch architecture including Transformers and Mamba
Remote execution on massive models (70B, 405B) via NDIF
Deferred execution model for efficient computation graph manipulation
384 GitHub stars

사용 사례

Activation patching to identify circuit components within neural networks
Steering model behavior by intervening in hidden states during generation
Mechanistic interpretability research on foundation models too large for local memory