TY - GEN
T1 - Multi-Dimensional ML-Pipeline Optimization in Cost-Effective Disaggregated Datacenter
AU - Huo, Pingyi
AU - Devulapally, Anusha
AU - Al Maruf, Hasan
AU - Chandramoorthy, Nandhini
AU - Arunachalam, Meena
AU - Akbulut, Gulsum Gudukbay
AU - Kandemir, Mahmut T.
AU - Narayanan, Vijaykrishnan
N1 - Publisher Copyright:
© 2025 Copyright held by the owner/author(s).
PY - 2025/10/17
Y1 - 2025/10/17
N2 - Machine learning (ML) pipelines deployed in datacenters are becoming increasingly complex and resource intensive, requiring careful optimizations to meet performance and latency requirements. Deployment in NUMA architectures with heterogeneous memory types such as CXL introduces a vast configuration space for memory and thread management. However, existing methods often need help to efficiently navigate this space, relying on significant manual tuning and needing more adaptability across diverse hardware platforms and extensive user-side code modification. This paper presents and experimentally evaluates an adaptive auto-tuning framework that optimizes memory configurations to maximize system throughput while adhering to Service Level Agreements (SLAs) on latency. Our optimization is carried out in two phases, addressing both performance and power efficiency. The proposed framework integrates an extended Berkeley Packet Filter (eBPF)-based kernel module for real-time performance monitoring and a user-space optimization core leveraging Bayesian Optimization and Pareto Optimality to explore high-dimensional configuration spaces. Extensive experimental evaluations on various workloads demonstrate that our framework achieves up to a 48% increase in throughput compared to default NUMA, TPP, Caption, and GPU while reducing search costs by as much as 77%. Furthermore, our two-phase optimization approach incorporates power efficiency, achieving up to 14.3% power savings without compromising throughput.
AB - Machine learning (ML) pipelines deployed in datacenters are becoming increasingly complex and resource intensive, requiring careful optimizations to meet performance and latency requirements. Deployment in NUMA architectures with heterogeneous memory types such as CXL introduces a vast configuration space for memory and thread management. However, existing methods often need help to efficiently navigate this space, relying on significant manual tuning and needing more adaptability across diverse hardware platforms and extensive user-side code modification. This paper presents and experimentally evaluates an adaptive auto-tuning framework that optimizes memory configurations to maximize system throughput while adhering to Service Level Agreements (SLAs) on latency. Our optimization is carried out in two phases, addressing both performance and power efficiency. The proposed framework integrates an extended Berkeley Packet Filter (eBPF)-based kernel module for real-time performance monitoring and a user-space optimization core leveraging Bayesian Optimization and Pareto Optimality to explore high-dimensional configuration spaces. Extensive experimental evaluations on various workloads demonstrate that our framework achieves up to a 48% increase in throughput compared to default NUMA, TPP, Caption, and GPU while reducing search costs by as much as 77%. Furthermore, our two-phase optimization approach incorporates power efficiency, achieving up to 14.3% power savings without compromising throughput.
UR - https://www.scopus.com/pages/publications/105021350177
UR - https://www.scopus.com/pages/publications/105021350177#tab=citedBy
U2 - 10.1145/3725843.3756088
DO - 10.1145/3725843.3756088
M3 - Conference contribution
AN - SCOPUS:105021350177
T3 - Proceedings of the Annual International Symposium on Microarchitecture, MICRO
SP - 1178
EP - 1192
BT - MICRO 2025 - 58th IEEE/ACM International Symposium on Microarchitecture
PB - IEEE Computer Society
T2 - 58th IEEE/ACM International Symposium on Microarchitecture , MICRO 2025
Y2 - 18 October 2025 through 22 October 2025
ER -