Metric Entropy and Generalization Upper Bounds for Low-Rank Symmetric-Score Polynomial Self-Attentio

We study a single-token polynomial surrogate f_{S,V} (x) = (x^T Sx)(V x) motivated by softmax-free self-attention, where S ∈ Sym_d(R) represents the symmetric score matrix and V ∈ R^{d×d} is the value matrix. By exploiting quadratic form invariance x^T Ax = x^T sym(A)x, we eliminate non-functional skew-symmetric components and establish that the rank-at-most-r symmetric score parameter space forms a bounded determinantal variety Sr with regular stratum dimension d_{S}(r) = r(2d−r+1)/2. Under bounded inputs ∥x∥_2 ≤ B, we prove an unconditional universal L_2(Q) metric entropy bound log N(ϵ, F_r, L_{2}(Q)) ≤ D_r log(C_{0}M/ϵ) across all probability measures Q,governed by the rank-dependent parameter-space entropy exponent D_r =r(2d−r+1)/2 + d^2. Via Dudley chaining on the centered scalar loss class, we derive an explicit O(C_R L_ℓ B^3 R_A R_V sqrt{D_r/n})
uniform generalization upper bound. In addition, we present a conditional local singular
complexity analysis: near the rank-collapse locus, we formulate a local determinantal normal
form model K_{NF}(x, y, z, w, v) = (xy − zw)^2 + ∥v∥_2^2 with v ∈ R^{d^2} and prove an exact Real Log
Canonical Threshold (RLCT) λ_{NF} = 0.5 + (d^2)/2 =(1+d^2)/2 (yielding λ_{NF} = 8.5 for d = 4). Numerical simulations across 50 random seeds with rejection sampling show strong quantitative alignment with our theoretical predictions, confirming exact convergence to the noise floor 0.5d σ^2 = 0.000200
and validating the rank-1 functional approximation bias.