How do I run experiments on large models like Llama-3 405B?

You can use the remote=True flag within the model.trace context to execute your interpretability code on NDIF's infrastructure without needing local GPUs.

Does nnsight support gradient analysis?

Yes, nnsight allows you to access and analyze gradients during the backward pass for local models, which is essential for attribution and sensitivity analysis.

Can I use nnsight with any PyTorch model?

Yes, nnsight is architecture-agnostic and works with any PyTorch model, including custom architectures, though it has built-in support for HuggingFace models.

nnsight is a specialized library for interpreting and manipulating PyTorch model internals, supporting both local execution and remote execution on massive models via NDIF.

What is the difference between nnsight and TransformerLens?

While TransformerLens provides a highly consistent API for a specific set of transformers, nnsight is more flexible across diverse architectures and uniquely supports massive-scale remote execution.

nnsight Model Interpretability

Name: nnsight Model Interpretability
Author: zechenzhangAGI

byzechenzhangAGI

•

384

•

Data Science & ML

Interprets and manipulates neural network internals for any PyTorch model, including massive foundation models via remote execution.

nnsight (pronounced 'insight') provides a unified interface to probe and modify the inner workings of PyTorch models, bridging the gap between small local experiments and massive scale inference. It allows researchers to trace computation graphs, save activations, and perform interventions like activation patching on models ranging from GPT-2 to Llama-3-405B. By leveraging the National Deep Inference Facility (NDIF), users can run complex mechanistic interpretability experiments on industrial-scale models without requiring high-end local GPU hardware, making advanced AI research accessible to everyone.

Key Features

01Cross-prompt activation sharing and multi-token generation tracing

02Activation patching and hidden state intervention capabilities

03Unified API for any PyTorch architecture including Transformers and Mamba

04Remote execution on massive models (70B, 405B) via NDIF

05Deferred execution model for efficient computation graph manipulation

06384 GitHub stars

Use Cases

01Activation patching to identify circuit components within neural networks

02Steering model behavior by intervening in hidden states during generation

03Mechanistic interpretability research on foundation models too large for local memory

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add zechenzhangagi/ai-research-skills nnsight

For use in Claude.ai and ChatGPT

Download Skill