Covariance, Subspace, and Intrinsic Cramér-Rao Bounds

Cramér-Rao bounds on estimation accuracy are established for estimation problems on arbitrary manifolds in which no set of intrinsic coordinates exists. The frequently encountered examples of estimating either an unknown subspace or a covariance matrix are examined in detail. The set of subspaces, called the Grassmann manifold, and the set of covariance (positive-definite Hermitian) matrices have no fixed coordinate system associated with them and do not possess a vector space structure, both of which are required for deriving classical Cramér-Rao bounds. Intrinsic versions of the Cramér-Rao bound on manifolds utilizing an arbitrary affine connection with arbitrary geodesics are derived for both biased and unbiased estimators. In the example of covariance matrix estimation, closed-form expressions for both the intrinsic and flat bounds are derived and compared with the root-mean-square error (RMSE) of the sample covariance matrix (SCM) estimator for varying sample support K. The accuracy bound on unbiased covariance matrix estimators is shown to be about (10/log 10)n/√K dB, where n is the matrix order. Remarkably, it is shown that from an intrinsic perspective, the SCM is a biased and inefficient estimator and that the bias term reveals the dependency of estimation accuracy on sample support observed in theory and practice. The RMSE of the standard method of estimating subspaces using the singular value decomposition (SVD) is compared with the intrinsic subspace Cramér-Rao bound derived in closed form by varying both the signal-to-noise ratio (SNR) of the unknown p-dimensional subspace and the sample support. In the simplest case, the Cramér-Rao bound on subspace estimation accuracy is shown to be about √(p(n-p))/√K/√SNR rad for p-dimensional subspaces. It is seen that the SVD-based method yields accuracies very close to the Cramér-Rao bound, establishing that the principal invariant subspace of a random sample provides an excellent estimator of an unknown subspace. The analysis approach developed is directly applicable to many other estimation problems on manifolds encountered in signal processing and elsewhere, such as estimating rotation matrices in computer vision and estimating subspace basis vectors in blind source separation.