Published November 9, 2026 | Version 1.0

Neural Computation: Optimal Pairwise-Max ReLU Representations

Authors/Creators

Description

The maximum function offers a concrete benchmark for the cost of exact neural computation. For five and six inputs in the pairwise-max model of side size two (PM2), we prove the exact directional widths W*(5,2)=6 and W*(6,2)=33. The features are all pairwise maxima, including the input coordinates; each hinge applies a ReLU to the difference between two sums of two features, with repetitions allowed. We count distinct nonparallel hinge directions and allow a free feature-linear term. Explicit rational identities attain both bounds on all real inputs. Optimality holds for arbitrary coefficient signs and magnitudes, without symmetry assumptions. The lower bounds exploit cancellation: every kink introduced where the maximum is linear must disappear from the sum. This condition yields exact linear equations. Small dependencies among binary kink profiles settle the five-input case; a canonical reduction that never increases support and a complete analysis of local support costs settle the six-input case. Both rational optimality theorems are fully formalized in Lean. A separate rationalization argument gives the same minima for real coefficients in the fixed dictionary.

Files

main-paper.pdf

Files (24.4 MB)

Name Size
md5:1d84223bcb4a5693a71677f4f81bff21
196.8 kB Preview Download
md5:b02c796cf10a037c1e31769d6bd138d7
23.3 MB Preview Download
md5:fea467d18b19152f9ca94430f7630359
969.7 kB Preview Download