Steering Language Models With Activation Engineering

By A. M. Turner, Lisa Thiergart, Gavin Leech, David S. Udell, Juan J. Vazquez, Ulisse Mini, M. MacDiarmid. 2023

663 citations recorded by the source corpus.

Prompt engineering and finetuning aim to maximize language model performance on a given metric (like toxicity reduction). However, these methods do not fully elicit a model's capabilities. To reduce this gap, we introduce activation engineering: the inference-time modification of activations in order to control (or steer) model outputs. Specifically, we introduce the Activation Addition (ActAdd) technique, which contrasts the intermediate activations on prompt pairs (such as"Love"versus"Hate") to compute a steering vector (Subramani et al. 2022). By tactically adding in e.g. the"Love"-"Hate"steering vector during the forward pass, we achieve SOTA on negative-to-positive sentiment shift and detoxification using models including LLaMA-3 and OPT. ActAdd yields inference-time control over high-level output properties (like topic and sentiment) while preserving performance on off-target tasks. ActAdd is lightweight: it does not require any machine optimization and works with a single pair of data points, which enables rapid iteration over steering. ActAdd demonstrates the power of activation engineering.

Subfields: Artificial Intelligence

View Paper: https://www.semanticscholar.org/paper/fe303bbaae47b1b08d0641b41d3288fcd74a3a80

How researchers see this paper

The researchers whose trusted networks rank it highest, strongest advocate first. Every researcher gets a different order.

In Artificial Intelligence

1. Meg Tong
2. Victor Veitch
3. Kiho Park
4. Nina Rimsky
5. Efstratios Gavves
6. Yo Joong Choe
7. Leonard Bereska

Citations in the corpus

• Steering Llama 2 via Contrastive Activation Addition (2023, background): “The set of alignment techniques known as “ac-tivation engineering” or “representation engineer-ing” work by making targeted perturbations to a model’s activations (Subramani et al., 2022; Her-nandez et al., 2023; Zou et al., 2023; Turner et al., 2023; Li et al., 2023; Liu et al., 2023).”
• The Linear Representation Hypothesis and the Geometry of Large Language Models (2023, background): “This is particularly prominent in mechanistic interpretability (Elhage et al., 2021; Meng et al., 2022; Hernandez et al., 2023; Turner et al., 2023; Zou et al., 2023; Todd et al., 2023; Hendel et al., 2023).”
• Mechanistic Interpretability for AI Safety - A Review (2024, background): “Moreover, linear addition techniques for model steering (Turner et al., 2023; Sakarvadia et al., 2023a; Li et al., 2023b) and representation engineering (Zou et al., 2023) highlight the practical implications of linear feature representations.”