research-article

Public Access

Optimal scheduling of in-situ analysis for large-scale scientific simulations

Authors:

Preeti Malakar,

Venkatram Vishwanath,

Christopher Knight,

Michael E. PapkaAuthors Info & Claims

SC '15: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis

Article No.: 52, Pages 1 - 11

https://doi.org/10.1145/2807591.2807656

Published: 15 November 2015 Publication History

Abstract

Today's leadership computing facilities have enabled the execution of transformative simulations at unprecedented scales. However, analyzing the huge amount of output from these simulations remains a challenge. Most analyses of this output is performed in post-processing mode at the end of the simulation. The time to read the output for the analysis can be significantly high due to poor I/O bandwidth, which increases the end-to-end simulation-analysis time. Simulation-time analysis can reduce this end-to-end time. In this work, we present the scheduling of in-situ analysis as a numerical optimization problem to maximize the number of online analyses subject to resource constraints such as I/O bandwidth, network bandwidth, rate of computation and available memory. We demonstrate the effectiveness of our approach through two application case studies on the IBM Blue Gene/Q system.

References

[1]

Vmd: Visual molecular dynamics. Journal of Molecular Graphics, 14(1):33--38, 1996.

[2]

Hasan Abbasi, Matthew Wolf, Greg Eisenhauer, Scott Klasky, Karsten Schwan, and Fang Zheng. DataStager: scalable data staging services for petascale applications. Cluster Computing, 13(3):277--290, 2010.

Digital Library

[3]

Laksono Adhianto, Sinchan Banerjee, Mike Fagan, Mark Krentel, Gabriel Marin, John Mellor-Crummey, and Nathan R. Tallent. HPCToolkit: Tools for performance analysis of optimized parallel programs. Concurrency and Computation: Practice and Experience, 22(6):685--701, 2010.

Digital Library

[4]

R Allan and A Mills. Survey of HPC Performance Modelling and Prediction Tools. Science and Technology, 2010.

[5]

M. P. Allen and D. J. Tildesley. Computer Simulation of Liquids. Oxford Science Publications, 1989.

Digital Library

[6]

David A. Bader. Petascale Computing: Algorithms and Applications. CRC Press, 2008.

[7]

J. C. Bennett, H. Abbasi, P.-T. Bremer, R. Grout, A. Gyulassy, Tong Jin, S. Klasky, H. Kolla, M. Parashar, V. Pascucci, P. Pebay, D. Thompson, Hongfeng Yu, Fan Zhang, and J. Chen. Combining in-situ and in-transit processing to enable extreme-scale scientific analysis. In Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, 2012.

Digital Library

[8]

A. Brooke, D. Kendrick, and A. Meeraus. GAMS: A User's Guide. The Scientific Press, South San Francisco, California, 1988.

[9]

I-Hsin Chung, Robert Walkup, Hui-Fang Wen, and Hao Yu. MPI Performance Analysis Tools on Blue Gene/L. In Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, 2006.

Digital Library

[10]

M. Dreher and B. Raffin. A Flexible Framework for Asynchronous in Situ and in Transit Analytics for Scientific Simulations. In Cluster, Cloud and Grid Computing (CCGrid), 2014 14th IEEE/ACM International Symposium on, pages 277--286, May 2014.

Digital Library

[11]

Brian Van Essen, Roger Pearce, Sasha Ames, and Maya Gokhale. On the Role of NVRAM in Data-intensive Architectures: An Evaluation. International Parallel and Distributed Processing Symposium, pages 703--714, 2012.

Digital Library

[12]

FLASH User Guide. http://flash.uchicago.edu/website/.

[13]

B. Fryxell, K. Olson, P. Ricker, F. X. Timmes, M. Zingale, D. Q. Lamb, P. MacNeice, R. Rosner, J. W. Truran, and H. Tufo. FLASH: An Adaptive Mesh Hydrodynamics Code for Modeling Astrophysical Thermonuclear Flashes. Astrophysical Journal, Supplement, 131:273--334, 2000.

[14]

M. Gamell, I. Rodero, M. Parashar, and S. Poole. Exploring energy and performance behaviors of data-intensive scientific workflows on systems with deep memory hierarchies. In High Performance Computing (HiPC), 2013 20th International Conference on, pages 226--235, Dec 2013.

[15]

Jonathan Geisler and Valerie Taylor. Performance Coupling: A Methodology for Predicting Application Performance using Kernel Performance. In In Proceedings of the ninth SIAM conference on parallel processing for scientific computing, 1999.

[16]

Megan Gilge. IBM System Blue Gene Solution: Blue Gene/Q Application Development. IBM Redbooks, June 2013.

[17]

L. Grinberg, V. Morozov, D. Fedosov, J. A. Insley, M. E. Papka, K. Kumaran, and G. E. Karniadakis. A new computational paradigm in multiscale simulations: Application to brain blood flow. In Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, 2011.

Digital Library

[18]

Salman Habib and et al. The Universe at Extreme Scale: Multi-petaflop Sky Simulation on the BG/Q. In Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, 2012.

Digital Library

[19]

Jiahua He, A. Jagatheesan, S. Gupta, J. Bennett, and A. Snavely. DASH: a Recipe for a Flash-based Data Intensive Supercomputer. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, 2010.

Digital Library

[20]

James W. Hurrell and et al. The Community Earth System Model: A Framework for Collaborative Research. Bulletin of the American Meteorological Society, 94(9):1339--1360, September 2013.

[21]

Tsuyoshi Ichimura, Kohei Fujita, Seizo Tanaka, Muneo Hori, Maddegedara Lalith, Yoshihisa Shizawa, and Hiroshi Kobayashi. Physics-based Urban Earthquake Simulation Enhanced by 10.7 BlnDOF x 30 K Time-step Unstructured FE Non-linear Seismic Wave Simulation. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, 2014.

Digital Library

[22]

Peter Johnsen, Mark Straka, Melvyn Shapiro, Alan Norton, and Thomas Galarneau. Petascale WRF Simulation of Hurricane Sandy Deployment of NCSA's Cray XE6 Blue Waters. In Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, 2013.

Digital Library

[23]

LAMMPS Molecular Dynamics Simulator. http://lammps.sandia.gov.

[24]

Ning Liu, J. Cope, P. Carns, C. Carothers, R. Ross, G. Grider, A. Crume, and C. Maltzahn. On the role of burst buffers in leadership-class storage systems. In IEEE 28th Symposium on Mass Storage Systems and Technologies (MSST), April 2012.

[25]

L. Lopes, J. Zilinskas, A. Costan, R. G. Cascella, G. Kecskemeti, E. Jeannot, M. Cannataro, L. Ricci, S. Benkner, S. Petit, et al. Euro-Par 2014: Parallel Processing Workshops: Euro-Par 2014 International Workshops, Porto, Portugal, August 25--26, 2014, Revised Selected Papers. Number pt. 2 in Lecture Notes in Computer Science / Theoretical Computer Science and General Issues. Springer International Publishing, 2014.

[26]

Peter MacNeice, Kevin M. Olson, Clark Mobarry, Rosalinda de Fainchtein, and Charles Packer. PARAMESH: A parallel adaptive mesh refinement community toolkit. Computer Physics Communications, 126(3):330--354, 2000.

[27]

Preeti Malakar, Thomas George, Sameer Kumar, Rashmi Mittal, Vijay Natarajan, Yogish Sabharwal, Vaibhav Saxena, and Sathish S. Vadhiyar. A Divide and Conquer Strategy for Scaling Weather Simulations with Multiple Regions of Interest. In Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, 2012.

Digital Library

[28]

Preeti Malakar, Vijay Natarajan, and Sathish S. Vadhiyar. An Adaptive Framework for Simulation and Online Remote Visualization of Critical Climate Applications in Resource-constrained Environments. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, 2010.

Digital Library

[29]

Preeti Malakar, Vijay Natarajan, and Sathish S. Vadhiyar. InSt: An Integrated Steering Framework for Critical Weather Applications. In Proceedings of the International Conference on Computational Science, volume 4, pages 116--125, 2011.

[30]

P. Mucci, S. Browne, C. Deane, and G. Ho. PAPI: A Portable Interface to Hardware Performance Counters. In Proceedings of Department of Defense HPCMP Users Group Conference, June 1999.

[31]

G. L. Nemhauser and L. A. Wolsey. Integer and Combinatorial Optimization. Wiley, New York, NY, 1988.

[32]

T. Peterka, J. Kwan, A. Pope, H. Finkel, K. Heitmann, S. Habib, Jingyuan Wang, and G. Zagaris. Meshing the universe: Integrating analysis in cosmological simulations. In High Performance Computing, Networking, Storage and Analysis (SCC), 2012 SC Companion:, pages 186--195, Nov 2012.

Digital Library

[33]

Steve Plimpton. Fast parallel algorithms for short-range molecular dynamics. Journal of Computational Physics, 117(1):1--19, 1995.

Digital Library

[34]

LAMMPS Rhodopsin Protein Benchmark. http://lammps.sandia.gov/bench.html#rhodo.

[35]

Osman Sarood, Akhil Langer, Abhishek Gupta, and Laxmikant Kale. Maximizing Throughput of Overprovisioned HPC Data Centers Under a Strict Power Budget. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, pages 807--818, 2014.

Digital Library

[36]

C. Seshadhri, Ali Pinar, David Thompson, and JanineC. Bennett. Sublinear algorithms for extreme-scale data analysis. In Janine Bennett, Fabien Vivodtzev, and Valerio Pascucci, editors, Topological and Statistical Methods for Complex Data, Mathematics and Visualization, pages 39--54. Springer Berlin Heidelberg, 2015.

[37]

Sameer S. Shende and Allen D. Malony. The Tau Parallel Performance System. International Journal of High Performance Computing Applications, 20(2):287--311, 2006.

Digital Library

[38]

V. Vishwanath, M. Hereld, M. E. Papka, R. Hudson, G. Jordan, and C. Daley. In Situ Data Analytics and I/O Acceleration of FLASH simulations on leadership-class systems with GLEAN. In Journal of Physics: Conference Series, Proceedings of SciDAC, 2011.

[39]

V. Vishwanath, M. Hereld, and M. E. Papka. Toward simulation-time data analysis and I/O acceleration on leadership-class systems. In Large Data Analysis and Visualization (LDAV), 2011 IEEE Symposium on, pages 9--14, Oct 2011.

[40]

Xingfu Wu, V. Taylor, J. Geisler, and R. Stevens. Isocoupling: reusing kernel coupling values to predict the performance of parallel applications. In Parallel and Distributed Processing Symposium, 2004. Proceedings. 18th International, April 2004.

[41]

Boyu Zhang, Trilce Estrada, Pietro Cicotti, and Michela Taufer. Enabling In-Situ Data Analysis for Large Protein-Folding Trajectory Datasets. In Proceedings of the 2014 IEEE 28th International Parallel and Distributed Processing Symposium, IPDPS '14, pages 221--230, 2014.

Digital Library

[42]

Fan Zhang, S. Lasluisa, Tong Jin, I. Rodero, Hoang Bui, and M. Parashar. In-situ feature-based objects tracking for large-scale scientific simulations. In High Performance Computing, Networking, Storage and Analysis (SCC), 2012 SC Companion:, pages 736--740, Nov 2012.

Digital Library

[43]

Fang Zheng, H. Abbasi, C. Docan, J. Lofstead, Qing Liu, S. Klasky, M. Parashar, N. Podhorszki, K. Schwan, and M. Wolf. PreDatA -- preparatory data analytics on peta-scale machines. In Parallel Distributed Processing (IPDPS), 2010 IEEE International Symposium on, April 2010.

[44]

Fang Zheng, Hongbo Zou, G. Eisenhauer, K. Schwan, M. Wolf, J. Dayal, Tuan-Anh Nguyen, Jianting Cao, H. Abbasi, S. Klasky, N. Podhorszki, and Hongfeng Yu. FlexIO: I/O Middleware for Location-Flexible Scientific Data Analytics. In Parallel Distributed Processing (IPDPS), 2013 IEEE 27th International Symposium on, pages 320--331, May 2013.

Digital Library

Cited By

Qiao ZTian QQin ZWang JLiu QPodhorszki NKlasky SZhu H(2024)Tango: A Cross-layer Approach to Managing I/O Interference over Local Ephemeral StorageSC24: International Conference for High Performance Computing, Networking, Storage and Analysis10.1109/SC41406.2024.00020(1-15)Online publication date: 17-Nov-2024
https://doi.org/10.1109/SC41406.2024.00020
Loring BBethel EWeber GMahoney M(2023)Extensions to the SENSEI In situ Framework for Heterogeneous ArchitecturesProceedings of the SC '23 Workshops of the International Conference on High Performance Computing, Network, Storage, and Analysis10.1145/3624062.3624161(868-874)Online publication date: 12-Nov-2023
https://dl.acm.org/doi/10.1145/3624062.3624161
Honore VDo TPottier LDa Silva RDeelman ESuter F(2022)SIM-SITU: A Framework for the Faithful Simulation of in situ Processing2022 IEEE 18th International Conference on e-Science (e-Science)10.1109/eScience55777.2022.00032(182-191)Online publication date: Oct-2022
https://doi.org/10.1109/eScience55777.2022.00032
Show More Cited By

Index Terms

Recommendations

Optimal execution of co-analysis for large-scale molecular dynamics simulations
SC '16: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis

The analysis of scientific simulation data enables scientists to derive insights from their simulations. This analysis of the simulation output can be performed at the same execution site as the simulation using the same resources or can be done at a ...
Persistent Data Staging Services for Data Intensive In-situ Scientific Workflows
DIDC '16: Proceedings of the ACM International Workshop on Data-Intensive Distributed Computing

Scientific simulation workflows executing on very large scale computing systems are essential modalities for scientific investigation. The increasing scales and resolution of these simulations provide new opportunities for accurately modeling complex ...
Topology-aware space-shared co-analysis of large-scale molecular dynamics simulations
SC '18: Proceedings of the International Conference for High Performance Computing, Networking, Storage, and Analysis

Analysis of scientific simulation data can be concurrently executed with simulation either in time- or space-shared mode. This mitigates the I/O bottleneck, however it results in either stalling the simulation for performing the analysis or transferring ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SC '15: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis

November 2015

985 pages

ISBN:9781450337236

DOI:10.1145/2807591

General Chair:
Jackie Kern
University of Illinois at Urbana-Champaign, Urbana, Illinois
,
Program Chair:
Jeffrey S. Vetter
Oak Ridge National Laboratory and Georgia Institute of Technology, Oak Ridge, Tennessee

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 November 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Advanced Scientific Computing Research

Conference

SC15

Sponsor:

SIGHPC
SIGARCH
IEEE-CS

SC15: The International Conference for High Performance Computing, Networking, Storage and Analysis

November 15 - 20, 2015

Texas, Austin

Acceptance Rates

SC '15 Paper Acceptance Rate 79 of 358 submissions, 22%;

Overall Acceptance Rate 1,516 of 6,373 submissions, 24%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

36
Total Citations
View Citations
531
Total Downloads

Downloads (Last 12 months)106
Downloads (Last 6 weeks)17

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Qiao ZTian QQin ZWang JLiu QPodhorszki NKlasky SZhu H(2024)Tango: A Cross-layer Approach to Managing I/O Interference over Local Ephemeral StorageSC24: International Conference for High Performance Computing, Networking, Storage and Analysis10.1109/SC41406.2024.00020(1-15)Online publication date: 17-Nov-2024
https://doi.org/10.1109/SC41406.2024.00020
Loring BBethel EWeber GMahoney M(2023)Extensions to the SENSEI In situ Framework for Heterogeneous ArchitecturesProceedings of the SC '23 Workshops of the International Conference on High Performance Computing, Network, Storage, and Analysis10.1145/3624062.3624161(868-874)Online publication date: 12-Nov-2023
https://dl.acm.org/doi/10.1145/3624062.3624161
Honore VDo TPottier LDa Silva RDeelman ESuter F(2022)SIM-SITU: A Framework for the Faithful Simulation of in situ Processing2022 IEEE 18th International Conference on e-Science (e-Science)10.1109/eScience55777.2022.00032(182-191)Online publication date: Oct-2022
https://doi.org/10.1109/eScience55777.2022.00032
Wang ZDorier MParashar M(2022)Research Perspectives Toward Autonomic Optimization of In Situ Analysis and Visualization2022 IEEE/ACM International Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization (ISAV)10.1109/ISAV56555.2022.00007(7-13)Online publication date: Nov-2022
https://doi.org/10.1109/ISAV56555.2022.00007
Huang DQin ZLiu QPodhorszki NKlasky S(2022)Identifying Challenges and Opportunities of In-Memory Computing on Large HPC SystemsJournal of Parallel and Distributed Computing10.1016/j.jpdc.2022.02.002Online publication date: Feb-2022
https://doi.org/10.1016/j.jpdc.2022.02.002
Malakar PVishwanath VKnight CMunson TPapka M(2022)Resource-Aware Optimal Scheduling of In Situ AnalysisIn Situ Visualization for Computational Science10.1007/978-3-030-81627-8_9(183-202)Online publication date: 5-May-2022
https://doi.org/10.1007/978-3-030-81627-8_9
Foster IAinsworth MBessac JCappello FChoi JDi SDi ZGok AGuo HHuck KKelly CKlasky SKleese van Dam KLiang XMehta KParashar MPeterka TPouchard LShu TTugluk Ovan Dam HWan LWolf MWozniak JXu WYakushin IYoo SMunson T(2021)Online data analysis and reduction: An important Co-design motif for extreme-scale computersThe International Journal of High Performance Computing Applications10.1177/10943420211023549(109434202110235)Online publication date: 12-Jun-2021
https://doi.org/10.1177/10943420211023549
Larsen MHarrison CTurton TSane SBrink SChilds H(2021)Trigger Happy: Assessing the Viability of Trigger-Based In Situ Analysis2021 IEEE 11th Symposium on Large Data Analysis and Visualization (LDAV)10.1109/LDAV53230.2021.00010(22-31)Online publication date: Oct-2021
https://doi.org/10.1109/LDAV53230.2021.00010
Mehta KAllen BWolf MLogan JSuchyta ESinghal SChoi JTakahashi KHuck KYakushin ISussman AMunson TFoster IKlasky S(2021)A codesign framework for online data analysis and reductionConcurrency and Computation: Practice and Experience10.1002/cpe.651934:14Online publication date: 26-Aug-2021
https://doi.org/10.1002/cpe.6519
Qiao ZLiu QPodhorszki NKlasky SChen J(2020)Taming I/O Variation on QoS-Less HPC Storage: What Can Applications Do?SC20: International Conference for High Performance Computing, Networking, Storage and Analysis10.1109/SC41405.2020.00015(1-13)Online publication date: Nov-2020
https://doi.org/10.1109/SC41405.2020.00015
Show More Cited By

View Options

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Figures

Tables

Media

View Table of Conten