TY - GEN
T1 - Distributed Gradient Descent
T2 - 57th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2019
AU - Swenson, Brian
AU - Murray, Ryan
AU - Poor, H. Vincent
AU - Kar, Soummya
N1 - Publisher Copyright:
© 2019 IEEE.
PY - 2019/9
Y1 - 2019/9
N2 - The paper studies continuous-time distributed gradient descent (DGD) and considers the problem of showing that in nonconvex optimization problems, DGD typically converges to local minima rather than saddle points. In centralized settings, the problem of demonstrating nonconvergence to saddle points is typically handled by way of the stable-manifold theorem from classical dynamical systems theory. However, the classical stable-manifold theorem is not applicable in the distributed setting. The paper develops an appropriate stable-manifold theorem for DGD. This shows that convergence to saddle points may only occur from a low-dimensional stable manifold. Under appropriate assumptions (e.g., coercivity), the result implies that DGD almost always converges to local minima.
AB - The paper studies continuous-time distributed gradient descent (DGD) and considers the problem of showing that in nonconvex optimization problems, DGD typically converges to local minima rather than saddle points. In centralized settings, the problem of demonstrating nonconvergence to saddle points is typically handled by way of the stable-manifold theorem from classical dynamical systems theory. However, the classical stable-manifold theorem is not applicable in the distributed setting. The paper develops an appropriate stable-manifold theorem for DGD. This shows that convergence to saddle points may only occur from a low-dimensional stable manifold. Under appropriate assumptions (e.g., coercivity), the result implies that DGD almost always converges to local minima.
UR - http://www.scopus.com/inward/record.url?scp=85077803431&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85077803431&partnerID=8YFLogxK
U2 - 10.1109/ALLERTON.2019.8919658
DO - 10.1109/ALLERTON.2019.8919658
M3 - Conference contribution
AN - SCOPUS:85077803431
T3 - 2019 57th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2019
SP - 595
EP - 601
BT - 2019 57th Annual Allerton Conference on Communication, Control, and Computing, Allerton 2019
PB - Institute of Electrical and Electronics Engineers Inc.
Y2 - 24 September 2019 through 27 September 2019
ER -