Residual Thoughts

Notes on mechanistic interpretability, geometry, and model behavior.

How Far Can You Read a Model From Its Weights?

mechanistic-interpretability
mlp-neurons
continual-learning
weight-geometry
activation-footprints

Weight crowding contains a real intervention signal, but it is not a substitute for usage or gradients. A Paper 1 field report across co-use, sequential updates, regime mixing, and quantization.

Jul 14, 2026
11 min

Polysemanticity Is Not the Price of Packing

mechanistic-interpretability
superposition
polysemanticity
mlp-neurons
negative-results

Superposition has a folk corollary: the neurons that mix meanings should be the ones whose weight vectors crowd together. Tested dictionary-free on both faces of the MLP across four model families, it is false at neuron granularity – and a planted-law control shows the pipeline would have caught it if it were true.

Jul 5, 2026
18 min

Correcting the Weight-Space Map: Three Bugs and a Cleaner Picture

mechanistic-interpretability
gemma-2
sae
attention-heads
errata

A short follow-up: three pipeline bugs (gamma folding, RoPE pairing, SAE encoding) invalidated the late-layer ‘anti-predictive regime,’ and what the corrected map plus two new studies actually show.

Jun 14, 2026
10 min

A Weight-Space Map of Attention Heads in Gemma-2 (with SAEs)

mechanistic-interpretability
gemma-2
sae
attention-heads

Can we learn something useful about attention heads without sampling new activations? This post uses Gemma-2 2B and Gemma Scope residual-stream SAEs to build a weight-space map of routing and writing behavior.

Mar 22, 2026
24 min
No matching items