Task Optimization Drives Statistical Decorrelation: An Empirical Study of Integration Dynamics in Feed-forward and Recurrent Neural Networks

Integrated Information Theory (IIT) posits that consciousness is related to a system's capacity for information integration (Φ). In contrast, modern deep learning systems are optimized for task performance, often through mechanisms that promote feature disentanglement and redundancy reduction. In this study, we investigate the relationship between task optimization and internal statistical integration by analyzing the training dynamics of Feed-forward (MLP) and Recurrent Neural Networks (RNN) on non-linearly separable tasks (e.g., XOR).

Using Gaussian Total Correlation (TC) as a tractable statistical proxy for multivariate dependency (not a direct measure of Φ), we observe a consistent inverse relationship: as model accuracy improves, internal statistical dependency significantly decreases. Furthermore, we find that recurrent architectures maintain higher baseline levels of statistical integration compared to feed-forward models, likely due to temporal feedback mechanisms, yet still exhibit a similar decay trend during optimization.

These results suggest a potential trade-off between task performance and statistical integration in standard neural architectures. While not directly measuring integrated information in the IIT sense, our findings highlight a systematic tendency toward decorrelation under gradient-based optimization, which may have implications for integration-based theories of consciousness. This work underscores the importance of temporal feedback and architectural design in sustaining integrated representations during learning.