TY - GEN
T1 - A simulated annealing based inexact oracle for wasserstein loss minimization
AU - Ye, Jianbo
AU - Wang, James Z.
AU - Li, Jia
N1 - Funding Information:
This material is based upon work supported by the National Science Foundation under Grant Nos. ECCS-1462230 and DMS-1521092. The authors would also like to thank anonymous reviewers for their valuable comments.
Publisher Copyright:
© Copyright 2017 by the authors(s).
PY - 2017
Y1 - 2017
N2 - Learning under a Wasserstein loss, a.k.a. Wasserstein loss minimization (WLM), is an emerging research topic for gaining insights from a large set of structured objects. Despite being conceptually simple, WLM problems are computationally challenging because they involve minimizing over functions of quantities (i.e. Wasserstein distances) that themselves require numerical algorithms to compute. In this paper, we introduce a stochastic approach based on simulated annealing for solving WLMs. Particularly, we have developed a Gibbs sampler to approximate effectively and efficiently the partial gradients of a sequence of Wasserstein losses. Our new approach has the advantages of numerical stability and readiness for warm starts. These characteristics are valuable for WLM problems that often require multiple levels of iterations in which the oracle for computing the value and gradient of a loss function is embedded. We applied the method to optimal transport with Coulomb cost and the Wasserstein non-negative matrix factorization problem, and made comparisons with the existing method of entropy regularization.
AB - Learning under a Wasserstein loss, a.k.a. Wasserstein loss minimization (WLM), is an emerging research topic for gaining insights from a large set of structured objects. Despite being conceptually simple, WLM problems are computationally challenging because they involve minimizing over functions of quantities (i.e. Wasserstein distances) that themselves require numerical algorithms to compute. In this paper, we introduce a stochastic approach based on simulated annealing for solving WLMs. Particularly, we have developed a Gibbs sampler to approximate effectively and efficiently the partial gradients of a sequence of Wasserstein losses. Our new approach has the advantages of numerical stability and readiness for warm starts. These characteristics are valuable for WLM problems that often require multiple levels of iterations in which the oracle for computing the value and gradient of a loss function is embedded. We applied the method to optimal transport with Coulomb cost and the Wasserstein non-negative matrix factorization problem, and made comparisons with the existing method of entropy regularization.
UR - http://www.scopus.com/inward/record.url?scp=85048536367&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85048536367&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:85048536367
T3 - 34th International Conference on Machine Learning, ICML 2017
SP - 6005
EP - 6017
BT - 34th International Conference on Machine Learning, ICML 2017
PB - International Machine Learning Society (IMLS)
T2 - 34th International Conference on Machine Learning, ICML 2017
Y2 - 6 August 2017 through 11 August 2017
ER -