TY - JOUR
T1 - A high-performance application data environment for large-scale scientific computations
AU - Shen, Xiaohui
AU - Liao, Wei Keng
AU - Choudhary, Alok
AU - Memik, Gokhan
AU - Kandemir, Mahmut
N1 - Funding Information:
This research was supported by the Department of Energy under the Accelerated Strategic Computing Initiative (ASCI) Academic Strategic Alliance Program (ASAP) Level 2, under subcontract no. W-7405-ENG-48 from Lawrence Livermore National Laboratories. The authors would like to thank Reagan Moore for discussions and help with SDSC resources. They would also like to thank Mike Wan and Mike Gleicher of SDSC for helping them with the implementation of the volume rendering code and in understanding the SRB and the HPSS. They thank Larry Schoof and Wilbur Johnson for providing the unstructured code used in this paper. They also thank Rick Stevens and Rajeev Thakur of ANL for various discussions on the problem of data management. They also thank Jaechun No for her help with the astrophysics application used in the experiments. Finally, they would like to thank Celeste Matarazzo, John Ambrosiano, and Steve Louis for discussions and their input.
PY - 2003/12
Y1 - 2003/12
N2 - Effective high-level data management is becoming an important issue with more and more scientific applications manipulating huge amounts of secondary-storage and tertiary-storage data using parallel processors. A major problem facing the current solutions to this data management problem is that these solutions either require a deep understanding of specific data storage architectures and file layouts to obtain the best performance (as in high-performance storage management systems and parallel file systems), or they sacrifice significant performance in exchange for ease-of-use and portability (as in traditional database management systems). In this paper, we discuss the design, implementation, and evaluation of a novel application development environment for scientific computations. This environment includes a number of components that make it easy for the programmers to code and run their applications without much programming effort and, at the same time, to harness the available computational and storage power on parallel architectures.
AB - Effective high-level data management is becoming an important issue with more and more scientific applications manipulating huge amounts of secondary-storage and tertiary-storage data using parallel processors. A major problem facing the current solutions to this data management problem is that these solutions either require a deep understanding of specific data storage architectures and file layouts to obtain the best performance (as in high-performance storage management systems and parallel file systems), or they sacrifice significant performance in exchange for ease-of-use and portability (as in traditional database management systems). In this paper, we discuss the design, implementation, and evaluation of a novel application development environment for scientific computations. This environment includes a number of components that make it easy for the programmers to code and run their applications without much programming effort and, at the same time, to harness the available computational and storage power on parallel architectures.
UR - http://www.scopus.com/inward/record.url?scp=0742268947&partnerID=8YFLogxK
UR - http://www.scopus.com/inward/citedby.url?scp=0742268947&partnerID=8YFLogxK
U2 - 10.1109/TPDS.2003.1255638
DO - 10.1109/TPDS.2003.1255638
M3 - Article
AN - SCOPUS:0742268947
SN - 1045-9219
VL - 14
SP - 1262
EP - 1274
JO - IEEE Transactions on Parallel and Distributed Systems
JF - IEEE Transactions on Parallel and Distributed Systems
IS - 12
ER -