Cross-View Variance Correlation in Path-Traced Stereo: A Hidden Shortcut in Synthetic Training Data

Path-traced synthetic stereo data underlie a large fraction of modern
disparity-estimation training pipelines. We report a previously
unrecognised property of such data: while the Monte~Carlo (MC) noise
streams of the two cameras are statistically independent, the underlying
\emph{variance fields}---deterministic per-pixel functions of the
rendering integrand---are highly correlated once aligned by the
ground-truth disparity warp. Across 20 scenes rendered with Mitsuba~3,
the warped Pearson correlation reaches $\rho{=}0.754{\pm}0.016$ across
20 scenes at $\mathrm{SPP}{=}512$, and on a representative scene
remains essentially invariant ($\rho{=}0.778{\pm}0.001$) over a
$16\times$ range of samples per pixel. The effect is strongest in
Lambertian regions ($\rho{\approx}0.78$) and substantially weaker in
glass ($\rho{\approx}0.30$), as predicted by an integrand decomposition
into view-independent and view-dependent components. A residual-shuffle intervention that breaks
the cross-view alignment while preserving the clean image collapses
the GT cost margin by $33\%$ on non-glass and degrades variance-based
winner-take-all accuracy on glass by $4.3\times$, confirming the
structure functions as a matching cue. This signal is unique to
MC-rendered data and constitutes a candidate sim-to-real shortcut whose
impact on trained networks remains to be quantified.