Machine learning techniques for improved data prefetching

With the advent of teraflop-scale computing on both a single coprocessor and many-core designs, there is tremendous need for techniques to fully utilize the compute power by keeping cores fed with data. Data prefetching has been used as a popular method to hide memory latencies by fetching data proactively before the processor needs the data. Fetching data ahead of time from the memory subsystem into faster caches reduces observable latencies or wait times on the processor end and this improves overall program execution times. We study two types of prefetching techniques that are available on a 61-core Intel Xeon Phi co-processor, namely software (compiler-guided) prefetching and hardware prefetching on a variety of workloads. Using machine learning techniques, we synthesize workload phases and the sequence of phase patterns using raw performance data from hardware counters such as memory bandwidth, miss ratios, prefetches issued, etc. Furthermore, we use performance data from workloads with different impacts and behaviors under various prefetcher settings. Our contribution can help in future prefetching design in the following ways: (1) to identify phases within workloads that have different characteristics and behaviors and help dynamically modify prefetch types and intensities to suit the phase; (2) to manage auto setting of prefetcher knobs without great effort from the user; (3) to influence software and hardware prefetching interaction designs in future processors; and (4) to use valuable insights and performance data in many areas such as power provisioning for the nodes in a large cluster to maximize both energy and performance efficiencies.