Solution Manifold Incompatibility in Multi-Fidelity PINNs: Why Linear Blending Fundamentally Fails for Nonlinear PDEs
Description
Description (v2.3.2):
This version presents a refined and expanded analysis of Solution Manifold Incompatibility (SMI) in multi-fidelity physics-informed neural networks for nonlinear partial differential equations.
This work identifies a structural limitation of linear multi-fidelity fusion for nonlinear partial differential equations. We introduce Solution Manifold Incompatibility (SMI), a geometric and algebraic obstruction arising from operator non-closure under linear solution blending. Through analysis and experiments on Allen–Cahn, Burgers, and incompressible Navier–Stokes systems, we demonstrate localized, residual-cancelled, and global failure regimes. The results show that residual minimization alone does not certify operator consistency in nonlinear multi-fidelity settings.
Relative to earlier versions, this release improves exposition, organization, and clarity, and consolidates the theoretical framework and experimental evidence supporting the central claims. The core mathematical results, diagnostics, and conclusions remain unchanged.
This version corresponds to a pre-submission draft intended for archival and reproducibility purposes.
Abstract (English)
Multi-fidelity physics-informed neural networks (MF-PINNs) aim to accelerate the solution of complex partial differential equations by combining inexpensive low-fidelity models with more costly high-fidelity ones. While this paradigm is effective for linear systems, it has proven far less reliable for strongly nonlinear problems, where multi-fidelity models often degrade in ways that are not captured by residual-based diagnostics. Although the breakdown of superposition under nonlinearity is classical, its consequences for learning-based solution fusion have not been formally articulated or systematically diagnosed.
In this work, we show that these failures arise from a structural incompatibility between nonlinear operators and linear solution-level fusion, including its affine extensions via additive correction terms. To make this precise, we introduce the notion of Solution Manifold Incompatibility (SMI). Because solutions of nonlinear partial differential equations lie on manifolds that are not closed under linear combination at the operator level, blending exact solutions generically activates cross-nonlinear terms that cannot be represented by either constituent operator. As a result, the fused solution can be physically inconsistent despite the accuracy of its constituents.
We illustrate the consequences of this incompatibility using three canonical examples: the Allen–Cahn equation, the viscous Burgers equation, and the incompressible Navier–Stokes equations. In Allen–Cahn, incompatibility remains localized near sharp interfaces; in Burgers, it is more deceptive, as nonlinear residuals may partially cancel while concealing substantial pointwise errors; and in Navier–Stokes, incompatibility is global, leaving no region where linear fusion is admissible.
We further show that in nonlinear regimes, practical multi-fidelity settings—such as coarse- and fine-grid solutions—can implicitly correspond to different operators, rendering linear fusion structurally incompatible with the target operator.
To expose these effects, we introduce diagnostic measures that reveal structural incompatibility masked by residual minimization. Taken together, our results identify a fundamental limitation of linear multi-fidelity learning for nonlinear physics.
Files
gangwar_SMI_MF_PINNs_v2.3.2.pdf
Files
(8.0 MB)
| Name | Size | Download all |
|---|---|---|
|
md5:2652ebf9b64c02b297c4056c339f629e
|
8.0 MB | Preview Download |
Additional details
Software
- Repository URL
- https://github.com/KartikeyGangwar/MF-PINN-SMI-Failure-Modes
- Programming language
- Python
References
- Olivier Botella and Roger Peyret. Benchmark spectral results on the lid-driven cavity flow. Computers & Fluids, 27(4):421–433, 1998. doi:10.1016/S0045-7930(98)00002-4.
- Lawrence C Evans. Partial Differential Equations. American Mathematical Society, 2010.
- U. K. N. G. Ghia, Kirti N. Ghia, and C. T. Shin. High-Re solutions for incompressible flow using the Navier-Stokes equations and a multigrid method. Journal of Computational Physics, 48(3):387–411, 1982. doi:10.1016/0021-9991(82)90058-4.
- George Em Karniadakis, Ioannis G Kevrekidis, Lu Lu, Paris Perdikaris, Sifan Wang, and Liu Yang. Physics-informed machine learning. Nature Reviews Physics, 3(6):422–440, 2021. doi:10.1038/ s42254-021-00314-5.
- Nikola Kovachki, Zongyi Li, Burigede Liu, Kamyar Azizzadenesheli, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Neural operator: Learning maps between function spaces with applications to pdes. Journal of Machine Learning Research, 24(89):1–97, 2023. URL: http: //jmlr.org/papers/v24/21-1524.html.
- Aditi Krishnapriyan, Amir Gholami, Shandian Zhe, Robert Kirby, and Michael W Mahoney. Charac- terizing possible failure modes in physics-informed neural networks. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, Advances in Neural Information Processing Systems, volume 34, pages 26548–26560. Curran Associates, Inc., 2021.
- Xuhui Meng and George Em Karniadakis. A composite neural network that learns from multi-fidelity data: Application to function approximation and inverse PDE problems. Journal of Computational Physics, 401:109020, 2020. doi:10.1016/j.jcp.2019.109020.
- Benjamin Peherstorfer, Karen Willcox, and Max Gunzburger. Survey of multifidelity methods in uncertainty propagation, inference, and optimization. SIAM Review, 60(3):550–591, 2018. doi: 10.1137/16M1082469.
- Maziar Raissi, Paris Perdikaris, and George Em Karniadakis. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. Journal of Computational Physics, 378:686–707, 2019. doi:10.1016/j.jcp.2018.10. 045.
- Kartikey Singh. A reference finite-difference solver for the 2d lid-driven cavity flow, 2026. doi: 10.5281/zenodo.18312938.
- Sifan Wang, Yujun Teng, and Paris Perdikaris. Understanding and mitigating gradient flow pathologies in physics-informed neural networks. SIAM Journal on Scientific Computing, 43(5):A3055–A3081, 2021. doi:10.1137/20M1318043.
- Randall J LeVeque. Finite Difference Methods for Ordinary and Partial Differential Equations. SIAM, 2007.
- Zongyi Li, Nikola Kovachki, Kamyar Azizzadenesheli, Burigede Liu, Kaushik Bhattacharya, Andrew Stuart, and Anima Anandkumar. Fourier neural operator for parametric partial differential equations, 2021. URL: https://arxiv.org/abs/2010.08895, arXiv:2010.08895.