Sustainable adaptive grid supercomputing: multiscale simulation of semiconductor processing across the pacific

ABSTRACT

We propose a reservation-based sustainable adaptive Grid supercomputing paradigm to enable tightly coupled computations of considerable scale (involving over 1,000 processors) and duration (over tens of continuous days) on a Grid of geographically distributed parallel supercomputers. The paradigm is demonstrated for an adaptive multiscale simulation application, in which accurate but compute-intensive quantum mechanical (QM) simulations are embedded within a classical molecular dynamics (Md) simulation only when and where high fidelity is required. Key technical innovations include: 1) an embedded divide-and-conquer algorithmic framework to maximally expose data and computation localities for enhanced scalability; 2) a buffered-cluster hybridization scheme to adaptively adjust MD/QM boundaries to maintain the model accuracy; and 3) a hybrid Grid remote procedure call (GridRPC) + message passing interface (MPI) Grid application framework to combine flexibility (adaptive resource allocation and migration), fault tolerance (automated fault recovery), and efficiency (scalable management of large computing resources). We have achieved an automated execution of multiscale MD/QM simulation on a Grid consisting of 6 supercomputer centers in Japan and the US (in total of 150 thousand processor-hours) for the dynamic simulation of implanted oxygen atoms in a silicon substrate, in which the number of processors changes dynamically on demand and resources are allocated and migrated dynamically according to both reservations and unexpected faults. The simulation results reveal a strong dependence of the oxygen penetration depth on the incident oxygen-beam position, which is useful information to further advance SIMOX (separation by implanted oxygen) technique to fabricate high speed and low power-consumption semiconductor devices.

REFERENCES

• 1: Ian Foster , Carl Kesselman, The Grid 2: Blueprint for a New Computing Infrastructure, Morgan Kaufmann Publishers Inc., San Francisco, CA, 2003
• 2: Gabrielle Allen , Thomas Dramlitsch , Ian Foster , Nicholas T. Karonis , Matei Ripeanu , Edward Seidel , Brian Toonen, Supporting efficient execution in heterogeneous distributed computing environments with cactus and globus, Proceedings of the 2001 ACM/IEEE conference on Supercomputing (CDROM), p.52-52, November 10-16, 2001, Denver, Colorado [doi>10.1145/582034.582086]
• 3: Emmott, S., and Rison, S. 2006. Towards 2020 Science. Microsoft, Redmond.
• 4: Foster, I. 2006. A two-way street to science's future. Nature, 440:419.
• 5: Shirts, M., and Pande, V. S. 2000. Screen severs of the world unite. Science, 290:1903.
• 6: TSUTOMU IKEGAMI , TOYOKAZU ISHIDA , DMITRI G. FEDOROV , KAZUO KITAURA , YUICHI INADOMI , HIROAKI UMEDA , MITSUO YOKOKAWA , SATOSHI SEKIGUCHI, Full Electron Calculation Beyond 20,000 Atoms: Ground Electronic State of Photosynthetic Proteins, Proceedings of the 2005 ACM/IEEE conference on Supercomputing, p.10, November 12-18, 2005 [doi>10.1109/SC.2005.28]
• 7: Shimojo, F., Kalia, R. K., Nakano, A., and Vashishta, P. 2005. Embedded divide-and-conquer algorithm on hierarchical real-space grids: parallel molecular dynamics simulation based on linear-scaling density functional theory. Computer Physics Communications, 167:151--164.
• 8: Broughton, J. Q., Abraham, F. F., Bernstein, N., and Kaxiras, E. 1999. Concurrent coupling of length scales: methodology and application. Physical Review B, 60:2391--2403.
• 9: Ogata, S., Lidorikis, E., Shimojo, F., Nakano, A., Vashishta, P., and Kalia, R. K. 2001. Hybrid finite-element/molecular-dynamics/electronic-density-functional approach to materials simulations on parallel computers. Computer Physics Communications, 138:143--154.
• 10: Hideaki Kikuchi , Rajiv K. Kalia , Aiichiro Nakano , Priya Vashishta , Hiroshi Iyetomi , Shuji Ogata , Takahisa Kouno , Fuyuki Shimojo , Kenji Tsuruta , Subhash Saini, Collaborative simulation grid: multiscale quantum-mechanical/classical atomistic simulations on distributed PC clusters in the US and Japan, Proceedings of the 2002 ACM/IEEE conference on Supercomputing, p.1-8, November 16, 2002, Baltimore, Maryland
• 11: Nicholas T. Karonis , Brian Toonen , Ian Foster, MPICH-G2: a Grid-enabled implementation of the Message Passing Interface, Journal of Parallel and Distributed Computing, v.63 n.5, p.551-563, May 2003 [doi>10.1016/S0743-7315(03)00002-9]
• 12: Vashishta, P., Kalia, R. K., and Nakano, A. 2006. Multimillion atom simulations of dynamics of oxidation of an aluminum nanoparticle and nanoindentation on ceramics. Journal of Physical Chemistry B, 110:3727--3733.
• 13: Hohenberg, P., and Kohn, W. 1964. Inhomogeneous electron gas. Physical Review, 136:B864-B871.
• 14: Kohn, W., and Vashishta, P. 1983. General density functional theory. In Inhomogeneous Electron Gas, eds. N. H. March and S. Lundqvist, pages 79--184. Plenum, New York.
• 15: Nakano, A., Kalia, R. K., Vashishta, P., Campbell, T. J. Ogata, S., Shimojo, F., and Sairi, S. 2002. Scalable atomistic simulation algorithms for materials research. Scientific Programming 10:263--271.
• 16: Fattebert, J.-L., and Gygi, F. 2004. Linear scaling first-principles molecular dynamics with controlled accuracy. Computer Physics Communications, 162:24--36.
• 17: Chelikowsky, J. R., Saad, Y., Ögut, S., Vashiliev, I., and Stathopoulos, A. 2000. Electronic structure methods for predicting the properties of materials: Grids in space. Phyica Status Solidi (b), 217:173--195.
• 18: Dapprich, S., Komaromi, I., Byun, K. S., Morokuma, K., and Frisch, M. J. 1999. A new ONIOM implementation in Gaussian 98. I. The calculation of energies, gradients, vibrational frequencies, and electric field derivatives. Journal of Molecular Structure (Theochem) 461--462:1--21.
• 19: Ogata, S., Shimojo, F., Kalia, R. K., Nakano, A., and Vashishta, P. 2002. Hybrid quantum mechanical/molecular dynamics simulations for parallel computers: density functional theory on real-space multigrids. Computer Physics Communications, 149:30--38.
• 20: Belkada, R., and Ogata, S. 2003. Theoretical study of stress corrosion cracking in Si. Transactions of the Materials Research Society of Japan, 28:817--820.
• 21: Ogata, S., Shimojo, F., Nakano, A., Vashishta, P., and Kalia, R. K. 2004. Environment effects of H2O on fracture initiation in Si: a hybrid electronic-density-functional/molecular-dynamics study. Journal of Applied Physics, 95:5316--5323.
• 22: Ogata, S., and Belkada, R. 2004. A hybrid electronic-density-functional/molecular-dynamics simulation scheme for multiscale simulation of materials on parallel computers: application to silicon and alumina. Computational Material Science, 30:189--194.
• 23: Ogata, S. 2005. Buffered-cluster method for hybridization of density-functional theory and classical molecular dynamics: application to stress-dependent reaction of H2O on nanostructured Si. Physical Review B, 72:045348.
• 24: Keith Seymour , Hidemoto Nakada , Satoshi Matsuoka , Jack Dongarra , Craig Lee , Henri Casanova, Overview of GridRPC: A Remote Procedure Call API for Grid Computing, Proceedings of the Third International Workshop on Grid Computing, p.274-278, November 18, 2002
• 25: Henri Casanova , Thomas M. Bartol, Jr. , Joel Stiles , Francine Berman, Distributing MCell Simulations on the Grid, International Journal of High Performance Computing Applications, v.15 n.3, p.243-257, August 2001 [doi>10.1177/109434200101500304]
• 26: Takemiya, H., Shudo, K., Tanaka, Y., and Sekiguchi, S. 2003. Constructing Grid applications using standard Grid middleware. Grid Computing, 1:117--131.
• 27: Ikegami, T., Takemiya, H., Nagashima, U., Tanaka, Y., and Sekiguchi, S. 2003. Accurate molecular simulation on the Grid---replica exchange Monte Carlo simulation for C20 molecule. IPSJ Transaction of Advanced Computing Systems, 44(SIG11):14--22.
• 28: Tanaka, Y., Nakada, H., Sekiguchi, S., Suzumura, T., and Matsuoka, S. 2003. Ninf-G: a reference implementation of RPC-based programming middleware for Grid computing. Journal of Grid Computing, 1:41--51.
• 29: Yoshio Tanaka , Hiroshi Takemiya , Hidemoto Nakada , Satoshi Sekiguchi, Design, Implementation and Performance Evaluation of GridRPC Programming Middleware for a Large-Scale Computational Grid, Proceedings of the Fifth IEEE/ACM International Workshop on Grid Computing (GRID'04), p.298-305, November 08-08, 2004 [doi>10.1109/GRID.2004.20]
• 30: William Gropp , Ewing Lusk , Nathan Doss , Anthony Skjellum, A high-performance, portable implementation of the MPI message passing interface standard, Parallel Computing, v.22 n.6, p.789-828, Sept. 1996 [doi>10.1016/0167-8191(96)00024-5 ]
• 31: http://hpcsoftware.teragrid.org/Software/user/show_asset.php?asset_id=189&view=TeraGrid
• 32: http://www.g-lambda.net
• 33: Mirghani, B. Y., Tryby, M. E., Baessler, D. A., Karonis, N., Rahjithan, R., and Mahinthakumar, K. G. 2005. Development and Performance Analysis of a Simulation-Optimization Framework on TeraGrid Linux Clusters. In Proceedings of the 6th LCI International Conference on Linux Clusters: The HPC Revolution 2005.
• 34: Toshiya Kimura , Hiroshi Takemiya, Distributed Parallel Computing for Fluid-Structure Coupled Simulations on a Heterogeneous Parallel Computer Cluster, International Journal of High Performance Computing Applications, v.13 n.4, p.320-333, November 1999 [doi>10.1177/109434209901300403]
• 35: Branicio, P. S., Kalia, R. K., Nakano, A., and Vashishta, P. 2006. Shock-induced Structural Transition, Plasticity, and Brittle Cracks in Aluminum Nitride Ceramic: A Molecular Dynamics Study, Physical Review Letters, Vol. 96, pp. 065502: 1--4.
• 36: Berger, M. J., and Oliger, J. 1984. Adaptive Mesh Refinement for Hyperbolic Partial Differential Equation, Journal of Computational Physics, Vol. 53, pp. 484--512.

INDEX TERMS

Primary Classification:
  C. Computer Systems Organization
  C.5 COMPUTER SYSTEM IMPLEMENTATION
      C.5.1 Large and Medium ("Mainframe") Computers
          Subjects: Super (very large) computers

Additional Classification:
  C. Computer Systems Organization
  C.0 GENERAL
      Subjects: System architectures; Modeling of computer architecture

  I. Computing Methodologies
  I.6 SIMULATION AND MODELING
      I.6.1 Simulation Theory
          Subjects: Systems theory
      I.6.6 Simulation Output Analysis

General Terms:
Design, Theory

Keywords:
density functional theory, grid application, grid remote procedure call, message passing interface, molecular dynamics, multiscale simulation, quantum mechanics