The φ-Compression Law: Golden Ratio Optimization in Neural Architecture
Description
We report a conservation law governing neural network compression: when layer widths follow golden ratio (φ ≈ 1.618) scaling via Lucas numbers, the product of efficiency gain and parameter retention equals unity:
efficiency × retention = 1.0004 ≈ 1
Systematic experiments across 216 training runs and four neural network architectures reveal that φ-based (Lucas) scaling achieves 2.81 ± 0.07× parameter efficiency gains with 64.4 ± 0.9% parameter reduction in convolutional networks (SimpleCNN, ResNet-18, ConvNeXt-Tiny), while showing exactly no effect in Vision Transformers. The efficiency gain approaches Euler's number (e ≈ 2.718, 3.4% deviation) and the retention fraction approaches 1/e (≈ 0.368, 3.3% deviation), suggesting a deep connection between golden ratio architecture and information-theoretic optimality.
The transformer null result is not a failure but a prediction: φ-scaling benefits hierarchical/cascaded processing, which CNNs possess inherently through width progression but standard transformers lack due to uniform layer structure. We propose and preliminarily validate the φ-Transformer architecture—with Lucas-scaled layer capacities—demonstrating that introducing hierarchy to transformers recovers the conservation law (efficiency = 2.76× ≈ e, product ≈ 1.15).
Files
Files
(25.0 kB)
| Name | Size | |
|---|---|---|
|
md5:37e41db9512357b305de155fc622d559
|
25.0 kB | Download |
Additional details
Software
- Repository URL
- https://github.com/ExeqTer91/phi-compression-law