Constrained ML estimation of structured covariance matrices with applications in radar STAP

The disturbance covariance matrix in radar space time adaptive processing (STAP) must be estimated from training sample observations. Traditional maximum likelihood (ML) estimators are effective when training is generous but lead to degraded false alarm rates and detection performance in the realistic regime of limited training. We exploit physically motivated constraints such as 1.) rank of the clutter subspace which can be inferred using existing physics based models such as the Brennan rule, and 2.) the Toeplitz constraint that applies to covariance matrices obtained from stationary random processes. We first provide a closed form solution of the rank constrained maximum likelihood (RCML) estimator and then subsequently develop an efficient approximation under joint Toeplitz and rank constraints (EASTR). Experimental results confirm that the proposed EASTR estimators outperform state-of-the-art alternatives in the sense of widely used measures such as the signal to interference and noise ratio (SINR) and probability of detection - particularly when training support is limited.