Rethinking Dispersion: Entropy, Robustness, and the Limits of Variance

The quantification of statistical dispersion is a cornerstone of data analysis, yet its foundational measure, the variance, possesses critical limitations that are often overlooked. This paper re-examines the concept of dispersion by challenging the primacy of variance and proposing a more holistic framework grounded in information theory and robust statistics. We argue that variance, despite its mathematical convenience, is a fragile and often misleading measure of spread due to its quadratic nature, which renders it exquisitely sensitive to outliers and ill-suited for heavy-tailed or skewed distributions. Its conceptual link to the mean ties it to a measure of centrality that is itself not robust. In contrast, this paper explores two alternative paradigms. First, we investigate robust statistical measures like the Median Absolute Deviation (MAD) and the Interquartile Range (IQR), which are designed to resist the influence of extreme observations and provide a more stable characterization of dispersion for real-world data. Second, and more fundamentally, we posit Shannon entropy as a superior, non-parametric measure of dispersion, understood as uncertainty. Unlike variance, entropy is defined directly from the probability distribution without reference to a central moment, making no assumptions about the metric properties of the sample space. It quantifies the true uncertainty or 'surprise' inherent in a distribution. We analyze the theoretical properties, axiomatic foundations, and practical implications of these different approaches, demonstrating through conceptual examples—including distributions where variance is undefined or uninformative—that a shift in perspective is necessary. This paper advocates for a decision-theoretic approach to selecting dispersion measures, urging practitioners to move beyond the default use of variance towards more robust and information-theoretically sound alternatives that better reflect the underlying structure and uncertainty of their data.