CNS Core: Small: A Multi-Stakeholder Integrated Approach to Reduce Tail Latency Using Heterogeneity

Project: Research project

Project Details


There are many different types of computer hardware. Examples include mobile cell phone processors, machine learning accelerators, traditional server processors, gaming platforms, and so forth, but these resources are specialized and primarily utilized within a single domain. As cloud computing providers broaden their markets and start offering more types of hardware, it opens an opportunity to design heterogeneity-aware computing systems, which combine different types of resources that are synergistic. This research will investigate techniques for intentionally utilizing heterogeneity as a controllable parameter for improving performance. Specifically, the research will focus on the tail latency performance metric, which has been identified by industry as an important performance metric affecting the responsiveness of user-facing Internet services. To deploy and harness the right heterogeneity at the right time, this research will consider the unique problems faced by three key stakeholders (Application Deployer, Resource Manager, and Infrastructure Provider). From the Application Deployer's perspective, the research will consider how to decompose an application into phases/sub-components that can benefit from different types of resources. From the Resource Manager's perspective, the research will determine the right quantity and mixture of resource types as well as how to schedule across these resources to minimize tail latency. From the Infrastructure Provider's perspective, the research will consider issues arising from sharing a mixture of different resources between multiple applications. Additionally, the research will leverage cross-stakeholder information towards a unified strategy for deploying and harnessing heterogeneity.

This research will be applicable to both providers of data centers, such as cloud providers, as well as businesses that use that infrastructure. For the provider, the research can improve the performance and lower the cost of data centers, which are critical components of the national infrastructure and economy. For the user, the research can enable emerging interactive applications, such as complex data analytics and real-time machine learning, to achieve high performance at low cost. This research promotes using diverse mixtures of resources, which could spur the development of new types of hardware and software. In addition to the broader research impacts, there is a plan to enhance the undergraduate and graduate courses with ideas and software generated by this research.

This award reflects NSF's statutory mission and has been deemed worthy of support through evaluation using the Foundation's intellectual merit and broader impacts review criteria.

Effective start/end date10/1/199/30/23


  • National Science Foundation: $500,000.00


Explore the research topics touched on by this project. These labels are generated based on the underlying awards/grants. Together they form a unique fingerprint.