TY - GEN
T1 - STRADS-AP
T2 - 2019 USENIX Annual Technical Conference, USENIX ATC 2019
AU - Kim, Jin Kyu
AU - Aghayev, Abutalib
AU - Gibson, Garth A.
AU - Xing, Eric P.
N1 - Publisher Copyright:
© Proceedings of the 2019 USENIX Annual Technical Conference, USENIX ATC 2019. All rights reserved.
PY - 2019
Y1 - 2019
N2 - It is a daunting task for a data scientist to convert sequential code for a Machine Learning (ML) model, published by an ML researcher, to a distributed framework that runs on a cluster and operates on massive datasets. The process of fitting the sequential code to an appropriate programming model and data abstractions determined by the framework of choice requires significant engineering and cognitive effort. Furthermore, inherent constraints of frameworks sometimes lead to inefficient implementations, delivering suboptimal performance. We show that it is possible to achieve automatic and efficient distributed parallelization of familiar sequential ML code by making a few mechanical changes to it while hiding the details of concurrency control, data partitioning, task parallelization, and fault-tolerance. To this end, we design and implement a new distributed ML framework, STRADS-Automatic Parallelization (AP), and demonstrate that it simplifies distributed ML programming significantly, while outperforming a popular data-parallel framework with a non-familiar programming model, and achieving performance comparable to an ML-specialized framework.
AB - It is a daunting task for a data scientist to convert sequential code for a Machine Learning (ML) model, published by an ML researcher, to a distributed framework that runs on a cluster and operates on massive datasets. The process of fitting the sequential code to an appropriate programming model and data abstractions determined by the framework of choice requires significant engineering and cognitive effort. Furthermore, inherent constraints of frameworks sometimes lead to inefficient implementations, delivering suboptimal performance. We show that it is possible to achieve automatic and efficient distributed parallelization of familiar sequential ML code by making a few mechanical changes to it while hiding the details of concurrency control, data partitioning, task parallelization, and fault-tolerance. To this end, we design and implement a new distributed ML framework, STRADS-Automatic Parallelization (AP), and demonstrate that it simplifies distributed ML programming significantly, while outperforming a popular data-parallel framework with a non-familiar programming model, and achieving performance comparable to an ML-specialized framework.
UR - http://www.scopus.com/inward/record.url?scp=85077018135&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=85077018135&partnerID=8YFLogxK
M3 - Conference contribution
AN - SCOPUS:85077018135
T3 - Proceedings of the 2019 USENIX Annual Technical Conference, USENIX ATC 2019
SP - 207
EP - 221
BT - Proceedings of the 2019 USENIX Annual Technical Conference, USENIX ATC 2019
PB - USENIX Association
Y2 - 10 July 2019 through 12 July 2019
ER -