Performance implications of architectural and software techniques on I/O-intensive applications

Meenakshi A. Kandaswamy, Mahmut Kandemir, Alok Choudhary, David E. Bernholdt

Research output: Chapter in Book/Report/Conference proceedingConference contribution

6 Scopus citations

Abstract

Many large scale applications, have significant I/O requirements as well as computational and memory requirements. Unfortunately, limited number of I/O nodes provided by the contemporary message-passing distributed-memory architectures such as Intel Paragon and IBM SP-2 limits the I/O performance of these applications severely. In this paper, we examine some software optimization techniques and architectural scalability and evaluate the effect of them in five I/O intensive applications from both small and large application domains. Our goals in this study are twofold: First, we want to understand the behavior of large-scale data intensive applications and the impact of I/O subsystem on their performance and vice-versa. Second, and more importantly, we strive to determine the solutions for improving the applications' performance by a mix of architectural and software solutions. Our results reveal that the different applications can benefit from different optimizations. For example, we found that some applications benefit from file layout optimizations whereas some others benefit from collective I/O. A combination of architectural and software solutions is normally needed to obtain good I/O performance. For example, we show that with limited number of I/O resources, it is possible to obtain good performance by using appropriate software optimizations. We also show that beyond a certain level, imbalance in the architecture results in performance degradation even when using optimized software, thereby indicating the necessity of increase in I/O resources.

Original languageEnglish (US)
Title of host publicationProceedings - 1998 International Conference on Parallel Processing, ICPP 1998
EditorsTen H. Lai
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages493-500
Number of pages8
ISBN (Electronic)0818686502
DOIs
StatePublished - 1998
Event1998 International Conference on Parallel Processing, ICPP 1998 - Minneapolis, United States
Duration: Aug 10 1998Aug 14 1998

Publication series

NameProceedings of the International Conference on Parallel Processing
ISSN (Print)0190-3918

Other

Other1998 International Conference on Parallel Processing, ICPP 1998
Country/TerritoryUnited States
CityMinneapolis
Period8/10/988/14/98

All Science Journal Classification (ASJC) codes

  • Software
  • General Mathematics
  • Hardware and Architecture

Fingerprint

Dive into the research topics of 'Performance implications of architectural and software techniques on I/O-intensive applications'. Together they form a unique fingerprint.

Cite this