In one sentence
Most neural networks are not arbitrary knots of unrelated features. Their apparent polysemanticity is usually structured generalization viewed through inadequate tools, and interpretability may be tractable by studying the network’s own mechanisms directly.
Overview
The essay is a personal working hypothesis, not a settled survey. Cammarata revisits canonical evidence that helped motivate sparse autoencoders and feature decomposition, especially InceptionV1’s cat-and-car neuron and its apparent car-in-dog superposition circuit. He argues these cases were misread: the cat-and-car neuron detects a shared visual pattern, such as a V-shape, while the car-and-dog circuit reflects shared pose and visualization failures rather than hidden arbitrary storage. He then extends the argument to modern vision and language models, claiming that better architectures, better microscopes, and iterative experimentation reveal mostly coherent neurons and circuits. The practical conclusion is optimistic: mechanistic interpretability should prioritize ordinary empirical science on the original network, potentially accelerated by automated agents.
Core ideas
Superposition may have been overdiagnosed
Superposition was proposed to explain why later InceptionV1 layers looked incoherent. Cammarata argues that the canonical evidence does not support that interpretation: architectural limitations caused deep-layer failure, while weak interpretability tools made ordinary structure look arbitrary.
Recursive generalization is the central alternative
A neuron can begin with a feature and grow related offshoots across training. An “iy” detector may extend to “diy,” then to “do it yourself”; a broad car feature may differentiate into specific car types as capacity increases. The meanings are not unrelated collisions but a branching, fractal form of generalization.
Describe patterns, not just nouns
Human interpreters tend to label neurons with lists of objects—cats and cars—when the neuron may detect a shared relation, shape, pose, or transition. A more useful question is: what distinction does this neuron make for downstream readers? Neurons often function more like verbs than nouns.
The canonical cat-and-car neuron is probably a shape detector
The famous InceptionV1 neuron 4e:55 appears to respond to cats and car fronts, but Cammarata finds evidence for a shared pointed or V-like pattern, with round forms on its negative side. Its connected readers respond to related pointed structures, supporting one coherent feature rather than an arbitrary mixture.
Feature visualization can manufacture false polysemanticity
Gradient-optimized visualizations may fall into common, easy-to-reach categories such as dogs and fail to reveal the neuron’s natural activation paths. Visualizing through upstream or downstream circuit components, and comparing natural examples, can expose the real shared structure.
Architecture and tools matter as much as model size
Residual streams, nonlinearities, normalization, and later architectural improvements can make networks easier to read. Cammarata repeatedly finds that apparent disorder disappears after fixing visualization settings, accounting for hidden dimensions, or using tools that test causal effects rather than merely reconstructing activations.
Modern language-model neurons look mostly structured
The essay reports preliminary audits in which roughly 90% of Qwen3-32B neuron poles appeared to have one main job, another roughly 7% had two related jobs, and only 2–4% remained unexplained. These are approximate, tool-dependent judgments, not a standardized result.
Weak activations are usually related, not hidden arbitrary features
Strong activations often reveal the neuron’s main role; weak activations tend to add analogies, neighboring concepts, or circuit spillover. Ablating large amounts of weak activation reportedly has little effect compared with removing a small amount of strong activation. Weak behavior should still be traced when it matters, but it is not automatically evidence of superposition.
Practical takeaways
- Study the original network before replacing it with a decomposition. Decompositions may be useful, but they can hide architectural quirks, unusual regimes, and errors in the interpretability tool.
- When a neuron appears polysemantic, search for a shared pattern, relation, pose, syntactic role, or positive/negative contrast before concluding that it contains unrelated features.
- Use multiple tests: natural activating examples, upstream inputs, downstream readers, negative poles, ablations, steering, and targeted counterexamples. A plausible label is not enough unless causal behavior supports it.
- Treat interpretability as a cumulative map-building process. Once neighboring neurons, motifs, and circuits are understood, unexplained components become easier to interpret and explanations constrain one another.
- Prefer explanations that reflect the network’s native organization. The goal is not necessarily a universal dictionary of all features, but becoming another reader of the mechanisms the network already uses.
- Use automated agents to propose, test, revise, and connect explanations across scales. The essay’s optimistic scenario is a laboratory of automated scientists operating over training and inference.
Caveats and counterpoints
- The headline claim is explicitly a working hypothesis. The essay offers extensive case studies and preliminary audits, not a definitive demonstration that superposition is absent in modern models.
- The reported cleanliness percentages depend heavily on thresholds, activation percentiles, depth, model architecture, evaluator judgment, and what counts as one thing. Cammarata cautions against treating a single score as an optimization target.
- The strongest evidence comes from models and tools the author has examined, especially Goodfire’s analyses. The selection may not represent all architectures, training regimes, or behaviors.
- The essay is much more confident about neurons and some circuits than about attention heads or the full residual stream. Cammarata says attention remains less understood and offers only a low-confidence prediction about its organization.
- Interpretability of local mechanisms does not automatically yield a complete understanding of cognition, agency, deception, or alignment-relevant behavior. The proposed route may make these questions more approachable, but that implication remains speculative.
- A neuron can be coherent without being simple. Recursive generalization may still produce broad, nested, or context-dependent mechanisms that require substantial investigation.
Questions worth revisiting
- How often do independent researchers reproduce the claimed reinterpretations of the classic superposition examples?
- Do the reported cleanliness results hold across models trained with different objectives, modalities, sparsity patterns, and scaling regimes?
- Can direct neuron-and-circuit analysis reliably discover rare but safety-critical mechanisms, or does it mainly reveal common structure?
- What empirical test would distinguish recursive generalization from genuine superposition when both produce related-looking activation patterns?
- How much can automated interpretability agents improve reliability without importing systematic errors from their training or evaluation procedures?
- Does understanding a network’s native mechanisms provide enough leverage for robust alignment interventions, or only better descriptions?
Return to this when…
Return to this essay when deciding whether to use sparse autoencoders or other decompositions, when a neuron looks arbitrarily polysemantic, or when evaluating optimistic claims about mechanistic interpretability. Its most useful reminder is methodological: apparent messiness may reflect a bad microscope, a bad model organism, or a human labeling bias rather than the network’s actual organization.