Orthogonal Sparse PCA and Covariance Estimation via Procrustes Reformulation

Konstantinos Benidis, Ying Sun, Prabhu Babu, Daniel P. Palomar

Research output: Contribution to journalArticlepeer-review

35 Scopus citations


The problem of estimating sparse eigenvectors of a symmetric matrix has attracted a lot of attention in many applications, especially those with a high dimensional dataset. While classical eigenvectors can be obtained as the solution of a maximization problem, existing approaches formulate this problem by adding a penalty term into the objective function that encourages a sparse solution. However, the vast majority of the resulting methods achieve sparsity at the expense of sacrificing the orthogonality property. In this paper, we develop a new method to estimate dominant sparse eigenvectors without trading off their orthogonality. The problem is highly nonconvex and hard to handle. We apply the minorization-maximization framework, wherein we iteratively maximize a tight lower bound (surrogate function) of the objective function over the Stiefel manifold. The inner maximization problem turns out to be a rectangular Procrustes problem, which has a closed-form solution. In addition, we propose a method to improve the covariance estimation problem when its underlying eigenvectors are known to be sparse. We use the eigenvalue decomposition of the covariance matrix to formulate an optimization problem wherein we impose sparsity on the corresponding eigenvectors. Numerical experiments show that the proposed eigenvector extraction algorithm outperforms existing algorithms in terms of support recovery and explained variance, whereas the covariance estimation algorithms improve the sample covariance estimator significantly.

Original languageEnglish (US)
Pages (from-to)6211-6226
Number of pages16
JournalIEEE Transactions on Signal Processing
Issue number23
StatePublished - Dec 1 2016

All Science Journal Classification (ASJC) codes

  • Signal Processing
  • Electrical and Electronic Engineering


Dive into the research topics of 'Orthogonal Sparse PCA and Covariance Estimation via Procrustes Reformulation'. Together they form a unique fingerprint.

Cite this