User-friendly scheduling tools for large-scale simulation experiments

ABSTRACT

Planning and steering numerical experiments that involve many simulations are difficult tasks to automate. We describe how a simulation scheduling tool can help experimenters submit and revoke simulation jobs on the basis of the most up to date partial results and resource estimates. We show how ideas such as pre- and post-conditions; interrupt handling; rapid experiment schema creation; and sparse parameter cross-products can be used to make a generalisable and user-friendly scheduling toolset. We describe our prototype in the context of typical long-running computational experiments of a complex networks simulation problem.

REFERENCES

• 1: High Performance Parametric Modeling with Nimrod/G: Killer Application for the Global Grid?, Proceedings of the 14th International Symposium on Parallel and Distributed Processing, p.520, May 01-05, 2000
• 2: D. Abramson , R. Sosic , J. Giddy , B. Hall, Nimrod: a tool for performing parametrised simulations using distributed workstations, Proceedings of the 4th IEEE International Symposium on High Performance Distributed Computing, p.112, August 02-04, 1995
• 3: Altair Grid Technologies Available at <http://www.openpbs.org> last visited 2007. Portable Batch System.
• 4: Michael Angel, Conducting experiments with Experiment Manager, Proceedings of the 28th conference on Winter simulation, p.535-541, December 08-11, 1996, Coronado, California, United States [doi>10.1145/256562.256741]
• 5: Apple Computer Inc Available at <http://developer.apple.com/hardwaredrivers/hpc/xgrid_intro.html> last visited 2007. Xgrid: High performance computing for the rest of us.
• 6: Epperson, J. E., K. A. Hawick, C. G. Windsor, and V. S. Rainey. 1990, Nov. A small angle neutron scattering investigation of phase separation in an Fe-Cr-Ni alloy. In Proc. Materials Research Society Meeting. Boston, USA: Materials Research Society. Symposium F.
• 7: Feynman, R. P., R. Leighton, and E. Hutchings. 1997. Surely You're Joking, Mr. Feynman! (Adventures of a Curious Character). ISBN 0393316041: W. W. Norton & Company.
• 8: FlowGrid Consortium 2004. FlowGrid Project available from <http://www.unizar.es/flowgrid/>.
• 9: Ian Foster , Carl Kesselman, The grid: blueprint for a new computing infrastructure, Morgan Kaufmann Publishers Inc., San Francisco, CA, 1998
• 10: Fox, G. C., R. D. Williams, and P. C. Messina. 1994. Parallel computing works! Morgan Kaufmann Publishers, Inc. ISBN 1-55860-253-4.
• 11: Hawick, K. A., J. E. Epperson, C. G. Windsor, and V. S. Rainey. 1990, Nov. Chemical phase separation in binary Iron-Chromium alloys. In Proc. Materials Research Society Meeting. Boston, USA: Materials Research Society. Symposium F.
• 12: Hawick, K. A., and H. A. James. 2006. Ising model scaling behaviour on z-preserving small-world networks arxiv:cond-mat/0611763.
• 13: James, H. A., and K. A. Hawick. 1998, February. A web-based interface for on-demand processing of satellite imagery archives. In Proc. 21st Australasian Computer Science Conf. ACSC'98, ed. C. McDonald, Volume 20 of Australian Computer Science Communications: Springer-Verlag Pte Ltd.
• 14: James, H. A., and K. A. Hawick. 2005. Scientific data management in a grid environment. J. Grid Computing 3:39--51.
• 15: James, H. A., and K. A. Hawick. 2007. Computational data structures for high-performance complex network-based small-world simulations. submitted to ACM Transactions on Modeling and Computer Simulation.
• 16: Medawar, P. 1988. Memoir of a thinking radish. ISBN 0192829834: Oxford University Press.
• 17: Object Management Group (OMG) 2006, June. Object Contraint Language Specification 2.0 available from <http://www.omg.org>.
• 18: Anna Persson , Henrik Grimm , Amos Ng, On-line instrumentation for simulation-based optimization, Proceedings of the 38th conference on Winter simulation, December 03-06, 2006, Monterey, California
• 19: Python Software Foundation 2007. The python programming language.
• 20: Larry Smarr , Charles E. Catlett, Metacomputing, Communications of the ACM, v.35 n.6, p.44-52, June 1992 [doi>10.1145/129888.129890]
• 21: STARLIMS Corporation 2007. STARLIMS Laboratory Information Management System. available from <http://www.starlims.com>.