Sparse Autoencoders Find Highly Interpretable Features in Language Models

By Hoagy Cunningham, Aidan Ewart, Logan Riggs Smith, Robert Huben, Lee Sharkey. International Conference on Learning Representations, 2023

1,374 citations recorded by the source corpus.

One of the roadblocks to a better understanding of neural networks' internals is \textit{polysemanticity}, where neurons appear to activate in multiple, semantically distinct contexts. Polysemanticity prevents us from identifying concise, human-understandable explanations for what neural networks are doing internally. One hypothesised cause of polysemanticity is \textit{superposition}, where neural networks represent more features than they have neurons by assigning features to an overcomplete set of directions in activation space, rather than to individual neurons. Here, we attempt to identify those directions, using sparse autoencoders to reconstruct the internal activations of a language model. These autoencoders learn sets of sparsely activating features that are more interpretable and monosemantic than directions identified by alternative approaches, where interpretability is measured by automated methods. Moreover, we show that with our learned set of features, we can pinpoint the features that are causally responsible for counterfactual behaviour on the indirect object identification task \citep{wang2022interpretability} to a finer degree than previous decompositions. This work indicates that it is possible to resolve superposition in language models using a scalable, unsupervised method. Our method may serve as a foundation for future mechanistic interpretability work, which we hope will enable greater model transparency and steerability.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/edb548fe7574d99454b352ffdb61bca93c3072ba

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

Citations in the corpus

The Geometry of Truth: Emergent Linear Structure in Large Language Model Representations of True/False Datasets (2023, background): “…neurons represent features (Wang et al., 2022; Sajjad et al., 2022; Bau et al., 2020), but features may more generally be represented by directions in a LLM’s latent space (i.e. linear combinations of neurons) (Dalvi et al., 2018; Gurnee et al., 2023; Cunningham et al., 2023; Elhage et al., 2022).”
Mechanistic Interpretability for AI Safety - A Review (2024, methodology): “Under review as submission to TMLR Sparse Dictionary Learning Sparse autoencoders (Cunningham et al., 2024) represent a solution attempt to the challenge of polysemantic neurons.”
Refusal in Language Models Is Mediated by a Single Direction (2024, positive): “Recent works use sparse autoencoders to discover feature directions in an unsupervised manner (Bricken et al., 2023; Cunningham et al., 2023; Templeton et al., 2024).”
Language Models Represent Space and Time (2023, background): “This suggests future work on extracting representations in the model’s coordinate system rather than trying to reconstruct human interpretable coordinates, perhaps with sparse autoencoders (Cunningham et al., 2023).”
Scaling and evaluating sparse autoencoders (2024, positive): “To measure whether we model features relevant to language modeling, we follow prior work [Bills et al., 2023, Cunningham et al., 2023, Bricken et al., 2023, Braun et al., 2024] and consider downstream Kullback-Leibler (KL) divergence and cross-entropy loss.”
Sparse Feature Circuits: Discovering and Editing Interpretable Causal Graphs in Language Models (2024, neutral): “The backbone of our approach is to, given an activation x ∈ R d model of some submodule for which we have an SAE, use the SAE to compute the quantities f i ( x ) and ϵ ( x ) in (1), and then intervene in our model’s forward pass to set Even though x was already numerically equal to the right-hand…”
Gemma Scope: Open Sparse Autoencoders Everywhere All At Once on Gemma 2 (2024, neutral): “18 we evaluate the change in loss (delta loss) on the user prompts, and surprisingly find that splicing in the base model SAE can reduce the loss in expectation in some cases.”
Sparse Prototype Code Underlies Classification and Prediction Across Modalities (2026, unknown): “Sparse autoencoders and dictionary-learning methods seek low-dimensional, semantically meaningful directions in activations [Makhzani and Frey, 2013, Huben et al., 2024, Templeton et al., 2026, Gao et al., 2025]; our results identify a complementary sparse class-supervised set of directions whose…”
Latent Terms: Dense Retrievers Contain Trivially Extractable BM25-ready Zipfian Vocabularies (2026, unknown): “…Autoencoders (SAEs), models trained to map a model activation into a higher-dimensional sparse code and then reconstruct the original activation from that code, have become a widely used tool for analyzing the internal representations of neural networks (Cunningham et al., 2023; Gao et al., 2024).”
Navigating the Concept Space of Language Models (2026, unknown): “On the other hand, single-level interaction embeddings like neuropedia.org or the one utilized by Cunningham et al. (2023) make feature discovery difficult due to visualization clutter.”
The Future is Sparse: Embedding Compression for Scalable Retrieval in Recommender Systems (2025, unknown): “52% (Figure 3, right).”
Beyond Matryoshka: Revisiting Sparse Coding for Adaptive Representation (2025, unknown): “To achieve this goal, we adopt sparse autoencoders due to their ability to scale with large data and restore feature semantics (Cunningham et al., 2023; Yan et al., 2024a).”