Grid middleware services for virtual data discovery, composition, and integration

ABSTRACT

We describe the services, architecture and application of the GriPhyN Virtual Data System, a suite of components and services that allow users to describe virtual data products in declarative terms, discover definitions and assemble workflows based on those definitions, and execute the resulting workflows on Grid resources. We show how these middleware-level services have been applied by specific communities to manage scientific data and workflows. In particular, we highlight and introduce <i>Chiron</i>, a portal facility that enables the interactive use of the virtual data system. Chiron has been used within the QuarkNet education project and as an online "educator" for virtual data applications. We also present applications from functional MRI-based neuroscience research.

REFERENCES

• 1: Avery, P. and Foster, I. The GriPhyN Project: Towards Petascale Virtual Data Grids, 2001. www.griphyn.org.
• 2: A. Chervenak, I. Foster, C. Kesselman, C. Salisbury, S. Tuecke. The Data Grid: Towards an Architecture for the Distributed Management and Analysis of Large Scientific Datasets. Journal of Network and Computer Applications, 23:187--200, 2001.
• 3: Foster, I., Voeckler, J., Wilde, M. and Zhao, Y., The Virtual Data Grid: A New Model and Architecture for Data-Intensive Collaboration. First Biennial Conference on Innovative Data Systems Research, Jan., 2004.
• 4: Ian T. Foster , Jens-S. Vöckler , Michael Wilde , Yong Zhao, Chimera: AVirtual Data System for Representing, Querying, and Automating Data Derivation, Proceedings of the 14th International Conference on Scientific and Statistical Database Management, p.37-46, July 24-26, 2002
• 5: A. Arbree, P. Avery, D. Bourilkov, R. Cavanaugh, S. Katageri, J. Rodriguez, G. Graham, J. Vöckler, M. Wilde. Virtual Data in CMS Productions. In Proceedings of Computing in High Energy and Nuclear Physics, March 2003.
• 6: James Annis , Yong Zhao , Jens Voeckler , Michael Wilde , Steve Kent , Ian Foster, Applying Chimera virtual data concepts to cluster finding in the Sloan Sky Survey, Proceedings of the 2002 ACM/IEEE conference on Supercomputing, p.1-14, November 16, 2002, Baltimore, Maryland
• 7: A. Rodriguez, D. Sulakhe, E. Marland, V. Nefedova, M. Wilde, N. Maltsev. Grid Enabled Server for High Throughput Analysis of Genomes, Workshop on Case Studies on Grid Applications, March 13, 2004, Berlin, Germany In conjunction with GGF10.
• 8: K. Leung, R. Heckemann, N. Saeed, K. Brooks, J. Buckton, K. Changani, D. Reid, D. Rueckert, J. Hajnal, M. Holden, D. Hill, Analysis of serial MR images of joints, IEEE International Symposium on Biomedical Imaging (ISBI) 2004, page 221--224.
• 9: James Frey , Todd Tannenbaum , Miron Livny , Ian Foster , Steven Tuecke, Condor-G: A Computation Management Agent for Multi-Institutional Grids, Cluster Computing, v.5 n.3, p.237-246, July 2002 [doi>10.1023/A:1015617019423 ]
• 10: E. Deelman, et al., "Pegasus: Mapping Scientific Workflows onto the Grid," Proc. 2nd EU Across Grids Conf., Cyprus, 2004.
• 11: Peter Buneman , Sanjeev Khanna , Wang Chiew Tan, Why and Where: A Characterization of Data Provenance, Proceedings of the 8th International Conference on Database Theory, p.316-330, January 04-06, 2001
• 12: The QuarkNet Project. http://quarknet.final.gov/, (June. 2004).
• 13: Kaizar Amin , Gregor von Laszewski , Mihael Hategan , Nestor J. Zaluzec , Shawn Hampton , Albert Rossi, GridAnt: A Client-Controllable Grid Work.ow System, Proceedings of the Proceedings of the 37th Annual Hawaii International Conference on System Sciences (HICSS'04) - Track 7, p.70210.3, January 05-08, 2004
• 14: A Collaborative Informatics Infrastructure for Multi-scale Science, Proceedings of the Second International Workshop on Challenges of Large Applications in Distributed Environments, p.24, June 07-07, 2004
• 15: Stein, G., Web Digital Authoring and Versioning (WebDAV) Resources Community Website, http://www.webdav.org/, 2004
• 16: Sriram Krishnan , Randall Bramley , Dennis Gannon , Madhusudhan Govindaraju , Rahul Indurkar , Aleksander Slominski , Benjamin Temko , Jay Alameda , Richard Alkire , Timothy Drews , Eric Webb, The XCAT science portal, Proceedings of the 2001 ACM/IEEE conference on Supercomputing (CDROM), p.49-49, November 10-16, 2001, Denver, Colorado [doi>10.1145/582034.582083]
• 17: Goble, C., Pettifer, S. and Stevens, R. Knowledge Integration: In silico Experiments in Bioinformatics. The Grid: Blueprint for a New Computing Infrastructure, Morgan Kaufmann, 2004
• 18: J. Van Horn, Online Availability of fMRI Results Images, Journal of Cognitive Neuroscience, 15(6):769--770, 2003.
• 19: Woods RP, et. al. Automated image registration: I. General methods and intrasubject, intramodality validation. Journal of Computer Assisted Tomography 1998;22:139--152.
• 20: Woods RP, et. al., Automated image registration: II. Intersubject validation of linear and nonlinear models. Journal of Computer Assisted Tomography 1998;22:153--165.
• 21: AIR 5 Suite: http://bishopw.loni.ucla.edu/AIR5/

INDEX TERMS

Primary Classification:
  D. Software
  D.2 SOFTWARE ENGINEERING
      D.2.6 Programming Environments

General Terms:
Design, Management

Keywords:
data discovery, data grid, data integration, portal, provenance, virtual data, workflow