TY - JOUR
T1 - A universally optimal multistage accelerated stochastic gradient method
AU - Aybat, Necdet Serhat
AU - Fallah, Alireza
AU - Gürbüzbalaban, Mert
AU - Ozdaglar, Asuman
N1 - Funding Information:
The work of Necdet Serhat Aybat is partially supported by NSF Grant CMMI-1635106. Alireza Fallah is partially supported by Siebel Scholarship. Mert Gürbüzbalaban acknowledges support from the grants NSF DMS-1723085 and NSF CCF-1814888.
Publisher Copyright:
© 2019 Neural information processing systems foundation. All rights reserved.
PY - 2019
Y1 - 2019
N2 - We study the problem of minimizing a strongly convex, smooth function when we have noisy estimates of its gradient. We propose a novel multistage accelerated algorithm that is universally optimal in the sense that it achieves the optimal rate both in the deterministic and stochastic case and operates without knowledge of noise characteristics. The algorithm consists of stages that use a stochastic version of Nesterov's method with a specific restart and parameters selected to achieve the fastest reduction in the bias-variance terms in the convergence rate bounds.
AB - We study the problem of minimizing a strongly convex, smooth function when we have noisy estimates of its gradient. We propose a novel multistage accelerated algorithm that is universally optimal in the sense that it achieves the optimal rate both in the deterministic and stochastic case and operates without knowledge of noise characteristics. The algorithm consists of stages that use a stochastic version of Nesterov's method with a specific restart and parameters selected to achieve the fastest reduction in the bias-variance terms in the convergence rate bounds.
UR - http://www.scopus.com/inward/record.url?scp=85090170275&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85090170275&partnerID=8YFLogxK
M3 - Conference article
AN - SCOPUS:85090170275
SN - 1049-5258
VL - 32
JO - Advances in Neural Information Processing Systems
JF - Advances in Neural Information Processing Systems
T2 - 33rd Annual Conference on Neural Information Processing Systems, NeurIPS 2019
Y2 - 8 December 2019 through 14 December 2019
ER -