Paralex: an environment for parallel programming in distributed systems

ABSTRACT

Modern distributed systems consisting of powerful workstations and high-speed interconnection networks are an economical alternative to special-purpose super computers. The technical issues that need to be addressed in exploiting the parallelism inherent in a distributed system include heterogeneity, high-latency communication, fault tolerance and dynamic load balancing. Current software systems for parallel programming provide little or no automatic support towards these issues and require users to be experts in fault-tolerant distributed computing. The Paralex system is aimed at exploring the extent to which the parallel application programmer can be liberated from the complexities of distributed systems. Paralex is a complete programming environment and makes extensive use of graphics to define, edit, execute and debug parallel scientific applications. All of the necessary code for distributing the computation across a network and replicating it to achieve fault tolerance and dynamic load balancing is automatically generated by the system. In this paper we give an overview of Paralex and present our experiences with a prototype implementation.

REFERENCES

• 1: W. B. Ackerman. Data Flow Languages. IEEE Computer, February 1982, pp. 15-22.
• 2: R. Anand, D. Lea and D. W. Forslund. Using n~gen++. Technical Report, School of Computer and Information Science, Syracuse University, January 1991.
• 3: D. P. Anderson. The AERO Programmer's Manual. Technical Report, CS Division, EECS Department, University of California, Berkeley, October 1990.
• 4: 6. Babao~lu, L. Alvisi, A. Amoroso and R. Davoli. Mapping Parallel Computations onto Distributed Systems in Paratex. In Proc. IEEE CompEuro '91 Conference, Bologna, Italy, May 1991.
• 5: Ozalp Babaoglu , Lorenzo Alvisi , Alessandro Amoroso , Renzo Davoli , Luigi A Giachini, Run-time Support for Dynamic Load Balancing and Debugging in Paralex, Cornell University, Ithaca, NY, 1991
• 6: R. G. Babb II. Parallel Processing with Large- Grain Data Flow Techniques. IEEE Computer, July 1984, pp. 55-61.
• 7: Adam Beguelin , Jack J. Dongarra, Graphical development tools for network-based concurrent supercomputing, Proceedings of the 1991 ACM/IEEE conference on Supercomputing, p.435-444, November 18-22, 1991, Albuquerque, New Mexico, United States [doi>10.1145/125826.126059]
• 8: T. Bemmerl, A. Bode, el al. TOPSYS Tools for Parallel Systems. SFB-Bericht 342/9/90A, Technisehe Universit~it Mfinehen, Munich, Germany, January 1990.
• 9: K. Birman and K. Marzullo. ISIS and the METAL Project. Sun Technology, vol. 2, no. 3 (Summer 1989), pp. 90-104.
• 10: James C. Browne , John Werth , Taejae Lee, Experimental Evaluation of a Reusability-Oriented Parallel Programming Environment, IEEE Transactions on Software Engineering, v.16 n.2, p.111-120, February 1990 [doi>10.1109/32.44375 ]
• 11: Nicholas Carriero , David Gelernter, How to write parallel programs: a guide to the perplexed, ACM Computing Surveys (CSUR), v.21 n.3, p.323-357, Sep. 1989 [doi>10.1145/72551.72553]
• 12: J. S. Chase, F. G. Amador, E. D. Lazowska, H. M. Levy and R. J. Littlefield. The Amber System: Parallel Programming on a Network of Multiprocessors. University of Washington, Department of Computer Science Technical Report 89-04-01, April 1989.
• 13: K. Dixit. SPECulations: Defining the SPEC Benchmark. Sun Tech Journal, vol. 4, no. 1, January 1991, pp. 53-65.
• 14: E. Fairfield. Private communication. Los Alamos National Laboratory, New Mexico.
• 15: Andrew S. Grimshaw, An Introduction to Parallel Object-Oriented Programming with Mentat, University of Virginia, Charlottesville, VA, 1991
• 16: A. Hisgen, A. Birrell, C. Jerian, T. Man, M. Schroeder and G. Swart. Granularity and Semantic Level of Replication in the Echo File System, In Proc. Workshop on Management of Replicated Data, Houston, IEEE CS Press 2085, November 1990, pp. 5-10.
• 17: Computer Staff, Gigabit Network Testbeds, Computer, v.23 n.9, p.77-80, September 1990
• 18: James J. Kistler , M. Satyanarayanan, Disconnected operation in the Coda file system, Proceedings of the thirteenth ACM symposium on Operating systems principles, p.213-225, October 13-16, 1991, Pacific Grove, California, United States
• 19: Leslie Lamport, Time, clocks, and the ordering of events in a distributed system, Communications of the ACM, v.21 n.7, p.558-565, July 1978 [doi>10.1145/359545.359563]
• 20: Guy L. Steele, Jr., Making asynchronous parallelism safe for the world, Proceedings of the 17th ACM SIGPLAN-SIGACT symposium on Principles of programming languages, p.218-231, December 1989, San Francisco, California, United States [doi>10.1145/96709.96731]

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING

Additional Classification:
  C. Computer Systems Organization
  C.1 PROCESSOR ARCHITECTURES
      C.1.2 Multiple Data Stream Architectures (Multiprocessors)
          Subjects: Interconnection architectures (e.g., common bus, multiport memory, crossbar switch)

  D. Software
  D.1 PROGRAMMING TECHNIQUES
      D.1.3 Concurrent Programming
          Subjects: Parallel programming
  D.4 OPERATING SYSTEMS
      D.4.7 Organization and Design
          Subjects: Distributed systems

General Terms:
Design, Measurement, Performance, Theory