Thread: Circuits
By Nick Cammarata, Shan Carter, Gabriel Goh, Chris Olah, Michael Petrov, Ludwig Schubert. 2020
Subfields: Model Architectures
How researchers see this paper
The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.
In Model Architectures
- Jacob Steinhardt
- Mor Geva
- Stella Biderman
- Ankit Gupta
- Dale Schuurmans
- Jonathan Berant
- Tengyu Ma
- Ekin Akyürek
Citations in the corpus
- GPT-NeoX-20B: An Open-Source Autoregressive Language Model (2022, background): “Mechanistic interpretability research (Cammarata et al., 2020) hopes to gain an understanding into how models accomplish the tasks they do, in part in the hopes of detecting problematic or deceptive algorithms implemented by models before these failures manifest in the real world.”
- Scaling Laws and Interpretability of Learning from Repeated Data (2022, background): “There is a relatively detailed understanding of mechanistic interpretability for convolutional image models [Cammarata et al., 2020], some understanding for multimodal models [Goh et al., 2021, Radford et al., 2021], and such an understanding is starting to be built up for Transformers trained on…”
- Analyzing Transformers in Embedding Space (2022, background): “[Cammarata et al., 2020] made observations about the interpretability of weights of neural networks.”
- What learning algorithm is in-context learning? Investigations with linear models (2022, background): “Research on extracting precise algorithmic descriptions of learned models is still in its infancy (Cammarata et al., 2020; Mu & Andreas, 2020).”
- Explainable Artificial Intelligence (XAI) 2.0: A Manifesto of Open Challenges and Interdisciplinary Research Directions (2023, background): “These problems and the related challenges are often interconnected; thus, they may, in principle, belong to multiple categories.”
- Progress measures for grokking via mechanistic interpretability (2023, methodology): “…interpretability suggests that models form circuits (Cammarata et al., 2020) – clean interpretable algorithms formed by subnetworks of the model, such as curve detectors (Cammarata et al., 2020) in image classification networks and induction heads (Elhage et al., 2021; Olsson et al., 2022) in LLMs.”