Measuring dead directions
Measuring Dead Directions: Decomposing and Classifying Singular Structure off Canonical Alignment · arXiv:2607.006031
The last spoke removes the last preconditions. Every earlier reading either watched training happen or asked the optimizer to cooperate; this paper reads the order of each dead direction at a single frozen checkpoint, with no descent and no alignment, in whatever basis the run ended in, asking only that the structure has formed. Three mechanisms make it work: the direction is constructed from the layer's K-FAC factors rather than searched for in a spectrum whose bottom is all gauge; a purity-matched window isolates the trustworthy part of the scan and refuses to report when nothing clean exists; and a slope-plus-magnitude verdict classifies each direction, genuine death, gauge, or the curved orbit that imitates one. On top of the read sit the paper's larger claims: the order is an invariant the architecture fixes while optimizers only choose the basis and sharpness; a fine-tuned ViT's dead structure is entirely gauge while a from-scratch one forms a genuine rotated node-death; and where the structure can be enumerated, the per-direction orders assemble into the global $(\lambda, m)$ matching the closed forms to machine precision, with $\nu$ universal per direction and measurably absorbed by the live network.
Where it is explained here
- The precondition every read shares: nothing to read yet#20
- The frozen read and its three mechanisms: reading at a standstill#21
- The taxonomy and the invariance: every way a direction dies#22
- The assembly, and ν: adding up the orders#23
The signature demo
The read itself, with the structure rotated off the axes:
And this one closes the loop, turning four instruments into one methodology, applicable to any trained checkpoint anyone hands you.
- Tejas Pradeep Shirodkar, Measuring Dead Directions: Decomposing and Classifying Singular Structure off Canonical Alignment, arXiv:2607.00603 (2026). ↩︎