CSR: Small: PROM in Clouds: Exploiting Scheduling for PeRformance OptiMization in Clouds

Cloud computing is emerging as an attractive computing paradigm, where both hardware and software resources can be leased from a cloud provider to achieve better economy of scale, elasticity, availability, and flexibility for a plethora of services. A major challenge in designing such cloud platforms is in providing high and predictable performance along with Quality of Service (QoS) guarantees. The performance unpredictability is attributed to a myriad of factors including system scale, workload dynamism, virtualization overheads, resource sharing and contention. The objective of this proposal is to investigate performance enhancement and performance quantification techniques in a cloud environment through three intertwined tasks. First, a comprehensive scheduling framework consisting of a high-level job scheduler and a fine-grained resource manager will be developed for efficient management of cloud resources for performance enhancement. Second, performance prediction models will be developed for estimating job completion time (JCT) in a cloud platform. Third, for providing QoS guarantees, a control-theoretic model incorporating the proposed scheduler and analytical models will be developed. Measurements on real platforms with MapReduce and other representative cloud workloads will be used to validate the proposed ideas. This research addresses one of the main concerns of cloud computing - What are the performance implications if my application is moved to a cloud? If successful in answering this question, it would have a tremendous impact on the cloud-enabled application domains. The cross-cutting nature of this work can foster new research avenues. In addition to undergraduate and graduate student training in areas such as architecture, distributed systems, and performance modeling, special attention will be given to involving women and minority students in this project. The modeling tools and techniques developed in this research will be made publicly available.