Asked about Working out what the network is doing
What is mechanistic interpretability?
An answer that works
Reverse-engineering the computation a network performs, identifying circuits of components that implement a specific behaviour. More ambitious than attribution or probing, which say where the model looked or what information is present, not what it computed.
Reading an answer is not being able to give one. The section that teaches this is Working out what the network is doing, 4 hours of Interpretability.
Learn this properlyAlso asked about Interpretability
What is superposition and why does it make interpretability hard?What can interpretability actually establish today?What is the difference between probing and activation patching?What is superposition and what is the response to it?Why is attribution not an explanation?What is the residual stream, and why is it a useful way to think about a transformer?Walk me through an activation patching experiment.How would you check whether an SAE feature really means what its label says?
All 10 questions on Interpretability