Challenges in getting flash drives closer to CPU

The PCI Express Solid State Disks (PCIe SSDs) blur the difference between block and memory access semantic devices. Since these SSDs leverage PCIe bus as storage interface, their interfaces are different from conventional memory system interconnects as well as thin storage interfaces. This leads to a new SSD architecture and storage software stack design. Unfortunately, there are not many studies focusing on the system characteristics of these emerging PCIe SSD platforms. In this paper, we quantitatively analyze the challenges faced by PCIe SSDs in getting flash memory closer to CPU and study two representative PCIe SSD architectures (from-scratch SSD and bridge-based SSD) using state-of-the-art real SSDs from two different vendors. Our experimental analysis reveals that 1) while the from-scratch SSD approach offers remarkable performance improvements, it requires enormous host-side memory and computation resources which may not be acceptable in many computing systems; 2) the performance of the from-scratch SSD significantly degrades in a multi-core system; 3) redundant flash software and controllers should be eliminated from the bridge-based SSD architecture; and 4) latency of PCIe SSDs significantly degrade with their storage-level queueing mechanism. Finally, we discuss system implications including potential PCIe SSD applications such as all-flash array.