Ordered Fisher Metric Degeneration Along the Parameter-Scale Axis: A Fisher-Degeneration Perspective on Neural Scaling Laws

We study whether Fisher metric degeneration---a principle previously studied in supervised, reinforcement, and generative learning settings---extends to the parameter-scale axis. Specifically, we investigate the effective Fisher rank ratio r_eff/N, which measures what fraction of parameters actively receive gradient signal from the data. Using small-scale MLP experiments (N = 177 to 92,289) on a harmonic regression task, we find an overall 45-fold net decrease in r_eff/N across this range: as model size grows, an increasing fraction of parameters becomes informationally inert. These results are consistent with the hypothesis that Fisher information dilutes systematically with scale---a pattern consistent with the Universal Degeneration Principle of the UCD series. We propose a Fisher-degeneration perspective on neural scaling laws, and conjecture that the power-law loss exponent alpha_N reflects the rate of this dilution. Quantitative verification at LLM scale remains an open problem; the present work provides small-scale evidence and theoretical motivation for such investigation. This is Paper X of the UCD series.