Tracing Attention Computation Through Feature Interactions

Transformer-based language models involve two main kinds of computations: multi-layer perceptron (MLP) layers that process information within a context position, and attention layers that conditionally move and process information between context positions. In our recent papers we made significant progress in breaking down MLP computation into interpretable steps. In this update, we fill in a major missing piece in our methodology, by introducing a way to decompose attentional computations as well.

Tracing Attention Computation Through Feature Interactions Transformer Circuits Thread Tracing Attention Computation Through Feature Interactions We describe and apply a method to explain attention patterns in terms of feature interactions, and integrate this information into attribution graphs. Authors Harish Kamath * , Emmanuel Ameisen * , Isaac Kauvar, Rodrigo Luger, Wes Gurnee, Adam Pearce, Sam Zimmerman, Joshua Batson, Thomas Conerly, Chris Olah, Jack Lindsey ‡ Affiliations Anthropic Published July 31st, 2025 * Core Research Contributor; ‡ Correspondence to jacklindsey@anthropic.com Trans

related reading

• A Mathematical Framework for Transformer Circuitstransformer-circuits.pub
• Circuit Tracing: Revealing Computational Graphs in Language Modelstransformer-circuits.pub
• The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io
• Bridging the Attention Gap: Complete Replacement Models for Complete Circuit Tracinginterp.open-moss.com
• Transformer Circuits Threadtransformer-circuits.pub
• On the Biology of a Large Language Modeltransformer-circuits.pub
• transformer_attention.pdfarxiv.org
• Thinking like Transformersrush.github.io
• Transformer Explainer: LLM Transformer Model Visually Explainedpoloclub.github.io
• The Illustrated Transformer – Jay Alammar – Visualizing machine learning one concept at a time.jalammar.github.io
• Transformers from Scratche2eml.school
• 1706.03762arxiv.org