SAELens: Mechanistic Interpretability

소개

SAELens provides a specialized framework for training and analyzing Sparse Autoencoders (SAEs), based on Anthropic’s groundbreaking research into monosemanticity. It allows researchers and engineers to solve the problem of polysemanticity—where single neurons represent multiple concepts—by extracting distinct, interpretable features from dense model activations. This skill enables deep visibility into what models have learned, supporting advanced tasks such as feature-based steering, safety analysis, and the discovery of specific semantic concepts within language models.

주요 기능

• Feature steering and logit attribution analysis for model behavior control
• 384 GitHub stars
• Deep integration with TransformerLens and Neuronpedia for feature visualization
• Sparsity and reconstruction metric monitoring (L0, CE loss recovery)
• Pre-trained SAE loading for popular models including GPT-2 and Gemma
• Training of custom Sparse Autoencoders (SAEs) with configurable architectures

사용 사례

• Analyzing safety-critical features like deception or bias in model weights
• Discovering interpretable features within language model hidden layers
• Steering model output by manipulating specific semantic feature directions