Which models are compatible with these SAE workflows?

While you can train SAEs for most models, the skill highlights pre-trained support for popular models like GPT-2 and Gemma through the TransformerLens integration.

What is the primary purpose of SAELens?

SAELens is used to train and analyze Sparse Autoencoders (SAEs), which help researchers decompose complex, polysemantic neural network activations into sparse, human-interpretable features.

What are the key metrics for evaluating an SAE?

The most important metrics are L0 (the number of active features), CE Loss Recovery (how much predictive performance is maintained), and the percentage of dead features.

Can I use this skill to steer model behavior?

Yes, this skill includes workflows for feature steering, allowing you to add specific feature directions back into the residual stream to influence how the model generates text.

Is a GPU required for using the SAELens skill?

A CUDA-enabled GPU is highly recommended for training SAEs and running the underlying language models efficiently, though some small-scale analysis can be done on CPU.

SAELens Mechanistic Interpretability

Name: SAELens Mechanistic Interpretability
Author: Orchestra-Research

byOrchestra-Research

•

3,983

•

数据科学与机器学习

Decomposes neural network activations into interpretable, sparse features using SAELens for deep mechanistic interpretability research.

SAELens addresses the challenge of polysemanticity in large language models where individual neurons represent multiple unrelated concepts. This skill provides a comprehensive framework for training, loading, and analyzing Sparse Autoencoders (SAEs) to map dense activations into sparse, monosemantic features. By using these workflows, researchers can perform feature discovery, causal steering, and logit attribution, making it possible to understand exactly what a model has learned and how to surgically influence its behavior for safety or performance.

主要功能

01Training custom SAEs with configurable architectures including Standard, Gated, and TopK variants

02Feature attribution tools to identify specific concepts driving model predictions and outputs

033,983 GitHub stars

04Loading and analyzing pre-trained SAEs for models like GPT-2 and Gemma via integrated repositories

05Activation steering capabilities to modify model behavior using discovered feature directions

06Automated metric tracking for L0 sparsity, cross-entropy recovery, and dead feature detection

使用场景

01Discovering human-interpretable features to understand model internal representations and concepts

02Identifying and mitigating safety-relevant features such as deception, bias, or harmful content

03Performing surgical model interventions via feature-based steering and ablation studies

What are Skills?·How to Install

Install with 🐟 Skill.Fish

npx skillfish add orchestra-research/ai-research-skills saelens

For use in Claude.ai and ChatGPT

Download Skill