How random.seed(42) discovered a 15-head circuit in GPT-2 small

Sometimes you don’t actually need an ML model at all. In my case, the model I used was numpy.random.seed(42). And it discovered a circuit in GPT-2 that existing methods can’t detect.

The background: I decided, for posterity’s sake, to manually investigate the randomly selected attention head I was using as a baseline. L9H3, it turned out, had a very distinct diagonal line positive OV matrix, just like the copier heads I was studying in the IOI circuit. But it didn’t seem to perform any copying role. It didn’t seem to do anything at all. It ignored names 99.4% of the time, had hardly any effect on downstream tasks. It was almost too unremarkable.

But then I saw another head in the same layer that looked almost exactly the same, zero effect on anything, no reason to exist as far as I could tell. But I wasn’t satisfied, they were clones of each other in every metric I could find. Long story short, after 12 hours straight of obsessive hacking, and 2 different totally separate sets of nodes, I converged on a set of 13 other heads which work synergistically with these two, but they have almost no effect individually - and existing methods like ACDC and EAP didn’t work to identify circuits with them at all.

How is that even possible, these methods literally maximize causal importance per head? The heads I found are synergistic and are not the most causally active heads per layer, making them sort of a “hidden circuit” that only clearly appears when you look at the weights. I wanted it to make more sense and be validated by existing methods, but literally everything I, claude code, claude web, perplexity, and all the prior works I found, failed to identify the circuit. But the causal intervention (IIA) numbers didn’t lie, it’s a real circuit, just not the type I (or anyone) was looking for.

Let’s zoom out for a second: To do circuit discoveries (IOI, GT, SVA, Gendered pronoun, etc.) the primary methods (automated like EAP/ACDC or manual) seem to boil down to “give the model some inputs for some task, and measure how it lights up”. The problem is, what do you do for components which don’t seem to light up much? And the other problem is, how do you choose which task?

The latter is a question of creativity, although in this case I wasn’t the one to come up with the idea of repeated tokens, I only figured it out afterwards once the numbers were clear. The former I intend to answer by what I’m calling “weight-space circuit discovery”. There’s a lot of names for the different types of analyses (spectral, compositional, structural) but essentially it just means analyzing the weights in order to discover circuits. My intuition was that if we can talk about functional heads like “name mover”, surely there’s some way to tell what a “name mover” looks like, as compared to other heads, or in a new model. With a lot of trial and error, I was able to distill each type of head that we currently know about, into a “fingerprint” of a few weight-space features which uniquely identify it, no examples or activations necessary. This allows for us to look for the same fingerprint in other model families and sizes.

This was actually what I was originally working on, testing each of the existing circuits and seeing why they were able to be classified by these weight-space feature fingerprints, before I got side tracked and accidentally discovered a new circuit and what I believe to be new types of functional heads. I believe these same techniques and ways of investigating things should be used more broadly and can yield many more circuit discoveries and a better mechanistic understanding of transformer networks.

Some more color commentary: arguably the main reason I even cared about weights to begin with is because my computer quite literally could not run forward passes without visually freezing so much I had to restart it afterwards. Torch stopped supporting intel mac’s 9 versions ago (2.2.0) and therefore nnsight, transformerlens, huggingface, pretty much any modern library ended up entertaining me with many red squiggles and required extensive hacking to get fully working. Suffice to say I’m happy that you can do so much with only looking at weights.

“The unlikely story began with a shitty laptop.”

Preprint: Elliot Tower. Weight-Space Discovery of an Epistatic Circuit Invisible to Activation-Based Methods. 2026. [Preprint]

Follow-up work (2 months later): Elliot Tower. Epistatic Circuits: Exhaustive Interaction Decomposition of Transformer Circuits. 2026. [Preprint]