A taxonomy of scientific workflow systems for grid computing

Abstract

With the advent of Grid and application technologies, scientists and engineers are building more and more complex applications to manage and process large data sets, and execute scientific experiments on distributed resources. Such application scenarios require means for composing and executing complex workflows. Therefore, many efforts have been made towards the development of workflow management systems for Grid computing. In this paper, we propose a taxonomy that characterizes and classifies various approaches for building and executing workflows on Grids. The taxonomy not only highlights the design and engineering similarities and differences of state-of-the-art in Grid workflow systems, but also identifies the areas that need further research.

References

1. J. Almond and D. Snelling. UNICORE: Secure and Uniform Access to Distributed Resources via the World Wide Web. White Paper, October 1998,]]
2. I. Altintas et al. A Framework for the Design and Reuse of Grid Workflows, International Workshop on Scientific Applications on Grid Computing (SAG'04), LNCS 3458, Springer, 2005]]
3. F. Berman et al. The GrADS Project: Software Support for High-Level Grid Application Development. International Journal of High Performance Computing Applications(JHPCA), 15(4):327--344, SAGE Publications Inc., London, UK, Winter 2001.]]
4. J. Cao et al. GridFlow:Workflow Management for Grid Computing. In 3rd International Symposium on Cluster Computing and the Grid (CCGrid), Tokyo, Japan, IEEE CS Press, Los Alamitos, CA, USA, May 12--15, 2003.]]
5. E. Deelman, J. Blythe, Y. Gil, and C. Kesselman. Workflow Management in GriPhyN. The Grid Resource Management, Kluwer, Netherlands, 2003.]]
6. E. Deelman et al. Mapping Abstract Complex Workflows onto Grid Environments. Journal of Grid Computing, 1:25--39, Kluwer Academic Publishers, Netherlands, 2003.]]
7. T. Fahringer et al. Truong. ASKALON: a tool set for cluster and Grid computing. Concurrency and Computation: Practice and Experience, 17:143--169, Wiley InterScience, 2005.]]
8. D. Fernández-Baca. Allocating Modules to Processors in a Distributed System. IEEE Transactions on Software Engineering, 15(11): 1427--1436, November 1989.]]
9. I. Foster and C. Kesselman (editors), The Grid: Blueprint for a Future Computing Infrastructure, Morgan Kaufmann Publishers, USA, 1999.]]
10. A. Geppert, M. Kradolfer, and D. Tombros. Market-based Workflow Management. International Journal of Cooperative Information Systems, World Scientific Publishing Co., NJ, USA, 1998.]]
11. V. Hamscher et al. Evaluation of Job-Scheduling Strategies for Grid Computing. In 1st IEEE/ACM International Workshop on Grid Computing (Grid 2000), Springer-Verlag, Heidelberg, Germany, 2000; 191--202.]]
12. S. Hwang and C. Kesselman. Grid Workflow: A Flexible Failure Handling Framework for the Grid. In 12th IEEE International Symposium on High Performance Distributed Computing (HPDC'03), Seattle, Washington, USA, IEEE CS Press, Los Alamitos, CA, USA, June 22--24, 2003.]]
13. G. von Laszewski. Java CoG Kit Workflow Concepts for Scientific Experiments. Technical Report, Argonne National Laboratory, Argonne, IL, USA, 2005.]]
14. B. Ludäscher et al. Scientific Workflow Management and the KEPLER System. Concurrency and Computation: Practice & Experience, Special Issue on Scientific Workflows, to appear, 2005]]
15. A. Mayer et al. Workflow Expression: Comparison of Spatial and Temporal Approaches. In Workflow in Grid Systems Workshop, GGF-10, Berlin, March 9, 2004.]]
16. S. McGough et al. Workflow Enactment in ICENI. In UK e-Science All Hands Meeting, Nottingham, UK, IOP Publishing Ltd, Bristol, UK, Sep. 2004; 894--900.]]
17. T. Oinn et al. Taverna: a tool for the composition and enactment of bioinformatics workflows. Bioinformatics, 20(17):3045--3054, Oxford University Press, London, UK, 2004.]]
18. S. S. Song, Y. K. Kwok, and K. Hwang. Trusted Job Scheduling in Open computational Grids: Security-Driven heuristics and A Fast Genetic Algorithm. In 19th IEEE International Parallel & Distributed Processing Symposium (IPDPS-2005), Denver, CO, USA., IEEE CS Press, Los Alamitos, CA, USA., April 4--8, 2005.]]
19. T. Tannenbaum, D. Wright, K. Miller, and M. Livny. Condor - A Distributed Job Scheduler. Beowulf Cluster Computing with Linux, The MIT Press, MA, USA, 2002.]]
20. I. Taylor, M. Shields, and I. Wang. Resource Management of Triana P2P Services. Grid Resource Management, Kluwer, Netherlands, June 2003.]]
21. W3C. Extensible Markup Language (XML) 1.0]]
22. J. Yu and R. Buyya. A Novel Architecture for Realizing Grid Workflow using Tuple Spaces. In 5th IEEE/ACM International Workshop on Grid Computing (GRID 2004), Pittsburgh, USA, IEEE CS Press, Los Alamitos, CA, USA, Nov. 8, 2004.]]
23. J. Yu and R. Buyya. A Taxonomy of Workflow Management Systems for Grid Computing. Technical Report, GRIDS-TR-2005-1, Grid Computing and Distributed Systems Laboratory, University of Melbourne, Australia, March 10, 2005.]]

Index Terms

• Applied computing

  • Enterprise computing

    • Business process management

• Computer systems organization

  • Architectures

    • Distributed architectures

• Social and professional topics

  • Professional topics

    • Management of computing and information systems

      • Project and people management

• Software and its engineering

  • Software organization and properties

    • Software system structures

      • Distributed systems organizing principles