Asked about Sparse autoencoders, hands on
How would you check whether an SAE feature really means what its label says?
An answer that works
Read activations across the whole range, not only the top examples, test predictions on new text where the label says it should and should not fire, and intervene by adding or removing the feature direction to see whether the model's behaviour changes as the label predicts.
Reading an answer is not being able to give one. The section that teaches this is Sparse autoencoders, hands on, 4 hours of Interpretability.
Learn this properlyAlso asked about Interpretability
What is mechanistic interpretability?What is superposition and why does it make interpretability hard?What can interpretability actually establish today?What is the difference between probing and activation patching?What is superposition and what is the response to it?Why is attribution not an explanation?What is the residual stream, and why is it a useful way to think about a transformer?Walk me through an activation patching experiment.
All 10 questions on Interpretability