I/O programming paradigms: Past and future

Large scale parallel scientific applications in general tend to be computationally-intensive as well as data-intensive. The advances in I/O systems, both in hardware and software, lag behind those in processors and interconnection networks, resulting in poor performance for data-intensive applications [17]. In this paper, we make two contributions. First, we present in detail the current I/O programming (optimization) techniques at different software levels and discuss their deficiencies. Second, we present a new I/O optimization approach that demands contributions from application writers, optimizing compilers, and run-time systems. We also summarize the experimental results obtained through the application of the proposed technique to five different I/O-intensive codes. We conclude that different I/O-intensive applications are amenable to different types of I/O optimizations.