A novel application development environment for large-scale scientific computations

ABSTRACT

Our results demonstrate that our novel application development environment provides both ease-of-use and high performance for large-scale, I/O-intensive scientific applications.

REFERENCES

• 1: C. Banl, R. Frost, J. Lopez, R. Marciano, R. Moore, A. Rajasekar, and M. Wan. Meta-data design for a massive data analysis system. In Proc. CASCON'96 Conference, 1996.
• 2: Chaitanya Baru , Reagan Moore , Arcot Rajasekar , Michael Wan, The SDSC storage resource broker, Proceedings of the 1998 conference of the Centre for Advanced Studies on Collaborative research, p.5, November 30-December 03, 1998, Toronto, Ontario, Canada
• 3: R. Bennett, K. Bryant, A. Sussman, R. Das, and J. S. Jovian. A framework for optimizing parallel i/o. In Proc. of the 1994 Scalable Parallel Libraries Conference, 1994.
• 4: Rajesh Bordawekar , Alok Choudhary , Ken Kennedy , Charles Koelbel , Michael Paleczny, A model and compilation strategy for out-of-core data parallel programs, Proceedings of the fifth ACM SIGPLAN symposium on Principles and practice of parallel programming, p.1-10, July 19-21, 1995, Santa Barbara, California, United States
• 5: P. Brown, R. Troy, D. Fisher, S. Louis, J. R. McGraw, and R. Musick. Meta-data sharing for balanced performance. In Proc. the First IEEE Meta-data Conference, Silver Spring. Maryland, 1996.
• 6: P. Cao, E. Felten, and K. Li. Application-controlled file caching policies. In Proc. the 1994 Summer USENIX Technical Conference, pages 171-182, 1994.
• 7: A. Choudhary, R. Bordawekar, M. Harry, R. Krishnalyes, R. Poanusamy, T. Singh, and R. Thaicur. Passion: parallel and scalable software for input-output. In NPAC Technical Report 5CC8-636, 1994.
• 8: P. F. Corbett , D. G. Feltelson , J.-P. Prost , G. S. Almasi , S. J. Baylor , A. S. Bolmarcich , Y. Hsu , J. Satran , M. Snir , R. Colao , B. D. Herr , J. Kavaky , T. R. Morgan , A. Ziotek, Parallel file systems for the IBM SP computers, IBM Systems Journal, v.34 n.2, p.222-248, 1995
• 9: P. Corbott, D. Fietelson, S. Fineberg, Y. Hsu, B. Nitzberg, J. Prost, M. Snir, B. Tmversat, and P. Wong. Overview of the mpi-io parallel i/o interface. In Proc. Third Workshop on I/0 in Parallel and Distributed Systems, IPPS'95, Santa Barbara, CA, 1995.
• 10: D. G. Feitelson , P. F. Corbett , J.-P. Prost , S. J. Baylor, Parallel access to files in the Vesta file system, Proceedings of the 1993 ACM/IEEE conference on Supercomputing, p.472-481, December 1993, Portland, Oregon, United States [doi>10.1145/169627.169786]
• 11: Thomas H. Cormen , David M. Nicol, Out-of-core FFTs with parallel disks, ACM SIGMETRICS Performance Evaluation Review, v.25 n.3, p.3-12, Dec. 1997 [doi>10.1145/270900.270902]
• 12: R.A. Coyne and H. Hulen. An introduction to the mass storage system reference model. In Proc. 12th IEEE Symposium on Mass STorage Systems, Monterey CA, 1993.
• 13: R. A. Coyne , H. Hulen , R. Watson, The high performance storage system, Proceedings of the 1993 ACM/IEEE conference on Supercomputing, p.83-92, December 1993, Portland, Oregon, United States [doi>10.1145/169627.169662]
• 14: J.R. Davis. Datalinks: Managing external data with db2 universal database. In IBM Corporation White Paper, 1997.
• 15: J. del Rosario, R. Bordawekar, and A. Clioudhary. Improved parallel i/o via a two-phase ran-time access strategy. In Proc. the 1993 IPPS Workshop on Input/Output in Parallel Computer Systems, 1993.
• 16: Juan Miguel del Rosario , Alok N. Choudhary, High-performance I/O for massively parallel computers: problems and prospects, Computer, v.27 n.3, p.59-68, March 1994 [doi>10.1109/2.268887 ]
• 17: M. Drewry, H. Conover, S. McCoy, and S. Graves. Meta-data: quality vs. quantity. In Proc. the Second IEEE Meta-data Conference, 1997.
• 18: C.S. Ellis and D. Kotz. Ptefetchiog in file systems for mired multiprocessors. In Proc. the 1989 International Conference on Parallel Processing, pages 306-314, 1989.
• 19: Meenakshi A. Kandaswamy , Mahmut T. Kandemir , Alok N. Choudhary , David E. Bernholdt, Performance Implications of Architectural and Software Techniques on I/O-Intensive Applications, Proceedings of the 1998 International Conference on Parallel Processing, p.493, August 10-14, 1998
• 20: John F. Karpovich , Andrew S. Grimshaw , James C. French, Extensible file system (ELFS): an object-oriented approach to high performance file I/O, Proceedings of the ninth annual conference on Object-oriented programming systems, language, and applications, p.191-204, October 23-28, 1994, Portland, Oregon, United States
• 21: David Kotz, Microprocessor file system interfaces, Proceedings of the second international conference on Parallel and distributed information systems, p.194-201, January 1993, San Diego, California, United States
• 22: D. Kotz. Disk-directed i/o for mired multiprocessors. In Proc. the 1994 Symposium on Operating Systems Design and Implementation, pages 61-74, 1994.
• 23: T. M. Madhyastha , D. A. Reed, Intelligent, adaptive file system policy selection, Proceedings of the 6th Symposium on the Frontiers of Massively Parallel Computation, p.172, March 27-31, 1996
• 24: Meat. In http://www.npaci, edu/DICE/SRB/mcat.html.
• 25: G. Memik, M. Kandemir, A. Choudhary, and V. E. Taylor. April: A run-time library for tape resident data. In the 17th IEEE Symposium oh Mass Storage Systems, 2000.
• 26: Todd C. Mowry , Angela K. Demke , Orran Krieger, Automatic compiler-inserted I/O prefetching for out-of-core applications, Proceedings of the second USENIX symposium on Operating systems design and implementation, p.3-17, October 29-November 01, 1996, Seattle, Washington, United States
• 27: 1. Newton. Application of raeta-data standards. In First IEEE Meta-data Conference, 1996.
• 28: R. Hugo Patterson , Garth A. Gibson , M. Satyanarayanan, A status report on research in transparent informed prefetching, ACM SIGOPS Operating Systems Review, v.27 n.2, p.21-34, April 1993 [doi>10.1145/155848.155855]
• 29: B. Runman. Paragon parallel file system. In External Product Specification, Intel Supercomputer Systems Division.
• 30: Sunita Sarawagi, Query Processing in Tertiary Memory Databases, Proceedings of the 21th International Conference on Very Large Data Bases, p.585-596, September 11-15, 1995
• 31: Michael Stonebraker , Dorothy Moore , Paul Brown, Object-Relational DBMSs: Tracking the Next Great Wave, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1998
• 32: Michael Stonebraker , Lawrence A. Rowe, The design of POSTGRES, Proceedings of the 1986 ACM SIGMOD international conference on Management of data, p.340-355, May 28-30, 1986, Washington, D.C., United States
• 33: R. Thakur, W. Gropp, and E. LusL On implementing MPl-lO portably and with high performance. Preprint ANL/MCS-P'732-1098, Mathematics and Computer Science Division, Argonne National Laboratory, 1998.
• 34: Rajeev Thakur , William Gropp , Ewing Lusk, Data Sieving and Collective I/O in ROMIO, Proceedings of the The 7th Symposium on the Frontiers of Massively Parallel Computation, p.182, February 21-25, 1999
• 35: Sivan Toledo , Fred G. Gustavson, The design and implementation of SOLAR, a portable library for scalable out-of-core linear algebra computations, Proceedings of the fourth workshop on I/O in parallel and distributed systems: part of the federated computing research conference, p.28-40, May 27-27, 1996, Philadelphia, Pennsylvania, United States [doi>10.1145/236017.236029]
• 36: UmTree User Guide, Release 2.0. UniTree Software, Inc., 1998.

INDEX TERMS

Primary Classification:
  F. Theory of Computation
  F.1 COMPUTATION BY ABSTRACT DEVICES
      F.1.2 Modes of Computation
          Subjects: Parallelism and concurrency

Additional Classification:
  C. Computer Systems Organization
  C.1 PROCESSOR ARCHITECTURES

  D. Software
  D.2 SOFTWARE ENGINEERING
  D.4 OPERATING SYSTEMS

  J. Computer Applications

  K. Computing Milieux
  K.6 MANAGEMENT OF COMPUTING AND INFORMATION SYSTEMS
      K.6.3 Software Management
          Subjects: Software development

General Terms:
Design, Experimentation, Measurement, Performance, Theory