DRackSim: Simulating CXL-enabled Large-Scale Disaggregated Memory Systems

Memory disaggregation has emerged as an alternative to traditional server architecture in data centers to target better memory utilization and higher scalability. It involves multiple independent compute nodes and remote memory pools that get hardware support through high-speed cache-coherent interconnects such as CXL. This paper introduces DRackSim, a simulation infrastructure for scalable disaggregated memory systems. DRackSim primarily models multiple compute nodes, memory pools, local/global memory managers, and a network interconnect for coherent memory access. An application-level simulation approach simulates an out-of-order x86 multi-core processor and a multi-level cache hierarchy at compute nodes. The network interface is simulated through a queue-based approach to handle remote memory access at multiple granularity. It also models a global memory manager for remote address space at the memory pools. Finally, we integrate a modified DRAMSim2 to perform local/remote memory simulation by declaring multiple instances of DRAMSim2. We rigorously validate DRackSim subsystems against Gem5 and a hardware prototype. Finally, we explore the design space by modeling various use-case scenarios for disaggregated memory systems and evaluate their performance over various HPC and cloud benchmarks.