Asymptotic Stochastic Analysis of Partially Relaxed DML

The Partial Relaxation approach has recently been proposed to solve the Direction-of-Arrival estimation problem [1], [2]. In this paper, we investigate the outlier production mechanism of the Partially Relaxed Deterministic Maximum Likelihood (PR-DML) Direction-of-Arrival estimator using tools from Random Matrix Theory. An accurate description of the probability of resolution for the PR-DML estimator is provided by analyzing the asymptotic stochastic behavior of the PR-DML cost function, assuming that both the number of antennas and the number of snapshots increase without bound at the same rate. The finite dimensional distribution of the PR-DML cost function is shown to be Gaussian in this asymptotic regime and this result is used to compute the probability of resolution.


INTRODUCTION
Direction-of-Arrival (DoA) estimation is a major area of research mainly due to its wide spread applications in radar, sonar, seismology, electronic surveillance and mobile communication [3][4][5][6]. Several high resolution algorithms, such as Multiple Signal Classification (MUSIC) [7], the minimum variance method of Capon [8], Estimation of Signal Parameters via Rotational Invariance Technique (ESPRIT) [9] have been proposed [10,11]. However, the performance of conventional "low-cost" methods strongly degrades when two or multiple sources are closely spaced [12,13]. This is due to the fact that conventional spectral search based approaches ignore the dependence between the sources, hence the interference, and therefore treat multi-source scenarios as single source scenarios.
The Partial Relaxation (PR) framework was introduced to overcome the aforementioned disadvantages of the conventional spectral-based DoA methods [1,2]. Instead of ignoring the presence of multiple sources, the PR approach considers both the signal impinging from the current direction of interest as well as the interfering ones. To reduce the computational demand, the manifold structure of the undesired signal components is relaxed, whereas the manifold structure of the desired signal component is kept unchanged. The multi-dimensional optimization problem reduces to a one-dimensional problem that admits a simple spectral based grid search applicable to any array geometry.
The main objective of this paper is the performance characterization of the recently introduced PR-DML technique in the threshold region, whereby both the number of samples per antenna and the Signal-to-Noise Ratio (SNR) take moderate values. This region is typically characterized by a systematic appearance of outliers in the DoA estimates, which are mainly caused by the incapability of resolving closely spaced sources. In [14], the probability of resolution of the PR-DML method was investigated by studying the stochastic behavior of the corresponding cost function for Gaussian distributed observations. The analysis in [14] is asymptotic in both the number of antennas and the number of snapshots. In this paper, we follow a similar approach as in [14] and utilize tools from Random Matrix Theory (RMT) to compute the asymptotic stochastic behavior of the PR-DML cost function whereby both the number of snapshots and the number of antennas are large quantities but their quotient converges to a fixed finite value. In contrast to [14] where the second order asymptotic behavior of the PR-DML cost function is computed numerically, we provide a closed-form expression for the variance and a suitable approximation for the closed-form expression of the covariance.
The paper is organized as follows. The signal model is introduced in Section 2 followed by the description of the PR-DML DoA estimation technique in Section 3. The asymptotic stochastic behavior of the PR-DML cost function is given in Section 4 followed by an expression for the probability of resolution in Section 5. Simulation results are presented in Section 6. Finally, Section 7 concludes this paper.

SIGNAL MODEL
Consider an antenna array equipped with M sensors and K impinging narrowband signals that satisfy M >K. The source signal at time instant n is denoted by s(n)=[s1(n),...,sK (n)] T ∈C K . The corresponding DoAs of the signals are denoted by θ=[θ1,...,θK ] T . Furthermore, the full-rank steering matrix is given by A(θ)= [a(θ1),...,a(θK )]∈C M ×K where a(θi)∈C M denotes the sensor array response for the i-th impinging signal. The number of sources K is assumed to be known. The received baseband signal x(n)∈C M at the n-th time instant is given by where N denotes the number of snapshots and n(n)∈C M the sensor noise. Assuming that signal and noise variables are statistically independent zero-mean circularly symmetric Gaussian distributed, the covariance matrix of the received signal R∈C M ×M is given by where Rs=E s(n)s H (n) denotes the covariance of the transmitted signal and σ 2 IM is the noise covariance matrix. Since the true covariance matrix is unavailable in practice, the sample covariance matrixR= 1 N N n=1 x(n)x H (n) is used instead.

PARTIALLY RELAXED DETERMINISTIC MAXIMUM LIKELIHOOD
In the framework of PR, not only the signals from the desired directions but also the dependence between the sources, hence the interference is considered [1]. However, the structure of the interfering signals is relaxed and consequently the computational complexity of the multi-source criteria is greatly reduced. Unlike in conventional Deterministic Maximum Likelihood (DML) and Stochastic Maximum Likelihood (SML) DoA estimation criteria, the steering matrix A is not described by a fully structured array manifold. Instead, A describes the partially relaxed array manifold which still retains some geometric structure of the sensor array [2]. Applying the PR approach to the conventional DML method and optimizing with respect to the relaxed signal part B yields the concentrated PR-DML cost function [1,2] where P ⊥ a (θ) denotes the orthogonal projection matrix and the eigenvalues of the modified sample covariance matrixR(θ)= P ⊥ a (θ)RP ⊥ a (θ) are sorted in non-descending order 0=λ1(θ)≤···≤ λM (θ). The K DoA estimates are given by the K arguments that correspond to the K deepest local minima of the concentrated objective function in (2). An efficient implementation of the PR-DML method is provided in [1].
The distinct true eigenvalues of the modified true covariance and their corresponding multiplicities are given by Km(θ), for m=0,...,M (θ). The number of distinct true eigenvalues of the modified true covariance matrix R(θ) amounts toM (θ)+1 and the sum of the multiplicities satisfies The non-necessarily Hermitian M ×M positive definite square root of the modified true covariance matrix R(θ) can also be expressed using the singular value decomposition where Ur(θ)∈C M ×Kr (θ) and Vr(θ)∈C M ×Kr (θ) generate the left and right orthonormal basis of R(θ) 1/2 .

ASYMPTOTIC BEHAVIOR OF THE PARTIALLY RELAXED DETERMINISTIC MAXIMUM LIKELIHOOD COST FUNCTION
In the following, the asymptotic behavior of the cost function in (2) is derived for the case where M,N →∞, M/N →c, 0<c<∞.
The covariance matrixR(θ) is assumed to have uniformly bounded spectral norm for all M . Furthermore, the covariance matrix can be In RMT it is well known that under all the previously mentioned assumptions, the empirical eigenvalue distribution ofR(θ) is almost surely close to an asymptotic non-random distribution which is absolutely continuous with density qM (x,θ) [15]. With increasing number of snapshots and therefore decreasing c=M/N , qM (x,θ) tends to concentrate around the true eigenvalues forming different eigenvalue clusters. The number of eigenvalue clusters increases with decreasing c as clusters begin to split [16]. Assuming there are S distinct clusters, the support of the clusters is given by the set of S disjoint compact intervals Furthermore, it can be observed that each modified true and distinct eigenvalue γm(θ) is associated to only one cluster. However, one cluster may be associated to multiple true eigenvalues which results in a non-bijective correspondence [16]. For sufficiently small c, there exist exactly as many clusters as distinct true eigenvaluesM (θ) [16]. In order to distinguish between the eigenvalues that are considered by the PR-DML cost function in (2) and the remaining ones it is crucial that the (M −K+1)-th modified sample eigenvalue asymptotically splits from the (M −K+2)-th one, which can be formalized as follows. We assume that there exists an integer m(θ) such that M −K+1= m(θ) r=0 Kr(θ), and the cluster associated to the eigenvalue γ m(θ) (θ) separates from the one associated to γ m(θ)+1 (θ) in the asymptotic eigenvalue distribution ofR(θ).
Let us consider the asymptotic stochastic behavior of the random real-valued L×1 vector whereθ= θ 1,...,θL T denotes a set of L points within the Field of View (FoV). Under the previously mentioned assumptions and as M,N →∞, M/N →c and 0<c<∞, the random vectorη(θ) in (3) converges in distribution to a multivariate standardized Gaussian distribution In the following we provide expressions for the asymptotic mean η(θ) of the random vectorη(θ) in (3) (first oder asymptotic behavior) and the corresponding L×L asymptotic covariance matrix Γ θ /M 2 (second order asymptotic behavior).

First Order Asymptotic Behavior
In [14] it was shown shown that the PR-DML cost functionη(θ) in (2) becomes asymptotically close to its deterministic counterpart Kr(θ)γr(θ) in the sense that |η(θ)−η(θ)|→0 almost surely pointwise in θ as M,N →∞ at the same rate. Furthermore, we define the asymptotic mean (first order asymptotic behavior) of the random vectorη θ in
Proof. The proof can be obtained by using the approach in [17]. Also see [18,Theorem 2].
In Section 4.2.1 we provide a closed-form expression for the variance of the PR-DML cost function followed by an approximation of the closed-form expression for the covariance that is valid for sufficient small c=M/N in Section 4.2.2.

Closed-Form Solution for the Variance
Motivated by the fact that the complex double contour integral in (7) simplifies for the special case where p=q we compute a closed-form expression for the variance according to the following Theorem. Corollary 1.1. For the special case whereθp=θq, the computation of the asymptotic variance of the PR-DML cost function simplifies to

Approximated Closed-Form Solution for the Covariance
Let us consider the more general case of the covariance in (7) for p,q=1,...,L. Instead of numerically computing the complex double contour integrals in (7) as it was done in [14], we express the logarithm log(1−Ω(ω1,ω2)) of the integrand in (7) as a Taylor expansion and only consider the terms of the integrand that involve 1/N or 1/N 2 . This is motivated by the closed-form solution of the asymptotic variance obtained in (8) which only involves expressions that vanish with N at rates 1/N and 1/N 2 . Using the Taylor expansion of the logarithm, the integrand in (7) is written as a rational function which allows to approximate the closed-form solution of the covariance by using conventional residue calculus [19]. In order to solve the integral in (7), we define A(ω,θ)= 1 which allows to write [Γ(θ)]p,q in (7) as Ω(ω1,ω2))dω1dω2. (9) The logarithm in the integrand of (9) can be expressed as a Taylor expansion according to the following Lemma. Lemma 1. The logarithm in the integrand of (9) can be expressed as Taylor expansion of Ω(ω1,ω2) around zero which converges since |Ω(ω1,ω2)|<1 for ω1∈Cω(θp)=ω(C(θp),θp) and ω2∈Cω(θq)=ω(C(θq),θq).
Furthermore, we separate between the terms of the integrand in (9) that involve 1/N r , r=1,2 and the terms that involve higher order factors 1/N r , r>2. The approximated closed-form solution of the asymptotic covariance in (7) is given by which can be solved in closed-form by integrating term by term using conventional residue calculus [19].
Remark It can be observed, that the approximated closed-form solution in (15) is equivalent to the closed-form expression of the asymptotic variance in (8) forθp=θq. The approximated closedform solution of the asymptotic covariance is only valid for sufficient small quotient c=M/N and if there exists separation between γ m(θp) θ p , respectively γ m(θq ) θ q , and adjacent eigenvalues. However, in contrast to the numerical solution in [14], the approximated closed-form solution of [Γ(θ)]p,q in (11) still provides a valid estimate on the asymptotic covariance in case of no separation as illustrated in the simulations.

PROBABILITY OF RESOLUTION
To analyze the outlier production mechanism of the PR-DML method we examine the probability of resolution as one possible application of the results derived in the previous section [21]. Let us study a scenario with two sources K=2 located at θ1 and θ2.
The predicted probability of resolution is therefore obtained by computing the cumulative distribution function (cdf) [24] Pres=Pr where f u Hη (θ) (x) denotes the asymptotic pdf of the test quantity.

SIMULATION RESULTS
In this Section, the predicted probability of resolution in (16) is compared to the simulated one. A Uniform Linear Array (ULA) equipped with M =10 sensors and two uncorrelated and closely spaced sources at θ=[45°,50°] T are considered. The transmitted signals are zero-mean and statistically independent with unit power and the SNR is given by SNR=1/σ 2 n . The separation boundary is defined as the smallest SNR that provides separation between the eigenvalue clusters associated to the m θ l -th true eigenvalue and larger adjacent true eigenvalues for l=1,2,3 and θ=[45°,50°,47.5°] T . For SNR values smaller than the separation boundary it is not possible to distinguish between the eigenvalues that are considered by the PR-DML cost-function and the remaining ones. Figures 1 and 2 depict the probability of resolution versus the SNR for N =10 and N =100 snapshots. As expected, the probability of declaring resolution increases with increasing number of snapshots. We observe that from our expressions it is possible to predict the probability of resolution remarkably well in both scenarios. Even in case of no separation our prediction of the probability of resolution in (16) that utilizes the approximated closed-form solution of the covariance in (11) is close to the actual one.

CONCLUSION
In this paper we have investigated the asymptotic behavior of the PR-DML DoA estimator under the setting of RMT where both the number of snapshots and the number of sensors go to infinity at the same rate. The finite dimensional distribution of the PR-DML cost function has been derived with closed-form expression for the variance and approximated closed-form expression for the covariance. Furthermore, the asymptotic probability distribution of the PR-DML method was used to characterize the probability of resolution in the threshold region, where the generation of outliers causes a performance breakdown.