Baseten urges domain-specific interpretability tools
Baseten argues that mechanistic interpretability should move away from broad, internet-scale sparse autoencoders and toward tools confined to a single domain, because general-purpose SAEs miss the specialized features that matter.
Inference company Baseten has published an essay calling for a "paradigm inversion" in mechanistic interpretability. It argues that the field has copied foundation-model scaling logic, training sparse autoencoders (SAEs) on broad, internet-wide data, and that this approach works against the goal of understanding how models reason.
The authors say plainly that the piece "will not be an empirical or evidence-based paper." They offer it as a position statement, backed by what they call "a very small taste" of evidence. Their research focuses on healthcare.
The main quantitative claim is that up to 70–80% of the residual error in broad-domain SAEs is linearly predictable from the input activations. Engels termed this unlearned structure "dark matter." Baseten says its experiments consistently show this result but does not give setups or models in the excerpt available.
The essay also cites earlier work on Claude 3 Sonnet. There, an SAE with 34 million features captured features for only about 60% of London boroughs, even though the model knew all of them and could name individual streets.
Baseten traces these gaps to the reconstruction objective. An SAE that has to cover every kind of input spreads a fixed feature budget thin, which favors generic features. The essay names two failure modes:
- Feature splitting: a concept like "diabetes" breaks into partial features, with no single feature for the whole concept.
- Feature absorption: specific clinical signs such as "orthopnoea" get folded into vague features like "breathlessness," or into surface patterns such as words beginning with "p."
The available text cuts off before Baseten describes its domain-confined method or shows results.