A first-order smoothed penalty method for compressed sensing

We propose a first-order smoothed penalty algorithm (SPA) to solve the sparse recovery problem min{||x||₁: Ax = b}. SPA is efficient as long as the matrix-vector product Ax and A^T y can be computed efficiently; in particular, A need not have orthogonal rows. SPA converges to the target signal by solving a sequence of penalized optimization subproblems, and each subproblem is solved using Nesterov's optimal algorithm for simple sets [Yu. Nesterov, Introductory Lectures on Convex Optimization: A Basic Course, Kluwer Academic Publishers, Norwell, MA, 2004] and [Yu. Nesterov, Math. Program., 103 (2005), pp. 127-152]. We show that the SPA iterates xk are ∈-feasible; i.e. || Ax_k - b ||₂ ≤ e and e-optimal; i.e. | ||x_k ||1 - ||x^*||1| ≤ ∈ after Õ (∈^-2/3) iterations. SPA is able to work with l ₁, l₂, or l_∞ penalty on the infeasibility, and SPA can be easily extended to solve the relaxed recovery problem min{||x||₁: ||Ax - b||2 ≤θ}.