Optimal covariance selection for estimation using graphical models

We consider a problem encountered when trying to estimate a Gaussian random field using a distributed estimation approach based on Gaussian graphical models. Because of constraints imposed by estimation tools used in Gaussian graphical models, the a priori covariance of the random field is constrained to embed conditional independence constraints among a significant number of variables. The problem is, then: given the (unconstrained) a priori covariance of the random field, and the conditional independence constraints, how should one select the constrained covariance, optimally representing the (given) a priori covariance, but also satisfying the constraints? In 1972, Dempster provided a solution, optimal in the maximum likelihood sense, to the above problem. Since then, many works have used Dempster's optimal covariance, but none has addressed the issue of suitability of this covariance for Bayesian estimation problems. We prove that Dempster's covariance is not optimal in most minimum mean squared error (MMSE) estimation problems. We also propose a method for finding the MMSE optimal covariance, and study its properties. We then illustrate the analytical results via a numerical example, that demonstrates the estimation performance advantage gained by using the optimal covariance vs Dempster's covariance. The numerical example also shows that, for the particular estimation scenario examined, Dempster's covariance violates the necessary conditions for optimality.