research-article

Loosely Coupled In Situ Visualization: A Perspective on Why It's Here to Stay

Authors:

Norbert Podhorszki,

David PugmireAuthors Info & Claims

ISAV2015: Proceedings of the First Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization

Pages 1 - 6

https://doi.org/10.1145/2828612.2828623

Published: 15 November 2015 Publication History

Abstract

In this position paper, we argue that the loosely coupled in situ processing paradigm will play an important role in high performance computing for the foreseeable future. Loosely coupled in situ is an enabling technique that addresses many of the current issues with tightly coupled in situ, including, ease-of-integration, usability, and fault tolerance. We survey the prominent positives and negatives of both tightly coupled and loosely coupled in situ and present our recommendation as to why loosely coupled in situ is an enabling technique that is here to stay. We then report on some recent experiences with loosely coupled in situ processing, in an effort to explore each of the discussed factors in a real-world environment.

References

[1]

S. Ahern, A. Shoshani, K.-L. Ma, A. Choudhary, T. Critchlow, S. Klasky, V. Pascucci, J. Ahrens, E. Bethel, H. Childs, et al. Scientific discovery at the exascale. Report from the DOE ASCR 2011 Workshop on Exascale Data Management, 2011.

[2]

J. C. Bennett, H. Abbasi, P.-T. Bremer, R. Grout, A. Gyulassy, T. Jin, S. Klasky, H. Kolla, M. Parashar, V. Pascucci, et al. Combining in-situ and in-transit processing to enable extreme-scale scientific analysis. In High Performance Computing, Networking, Storage and Analysis (SC), 2012 International Conference for, pages 1--9. IEEE, 2012.

Digital Library

[3]

C. Chang, S. Ku, P. Diamond, Z. Lin, S. Parker, T. Hahm, and N. Samatova. Compressed ion temperature gradient turbulence in diverted tokamak edgea). Physics of Plasmas (1994-present), 16(5):056108, 2009.

[4]

H. Childs, M. Kwan-Liu, Y. Hongfeng, W. Brad, M. Jeremy, F. Jean, K. Scott, P. Norbert, S. Karsten, W. Matthew, P. Manish, and Z. Fan. In situ processing. In E. W. Bethel, H. Childs, and C. Hansen, editors, High Performance Visualization: Enabling Extreme-Scale Scientific Insight. CRC Press, Boca Raton, FL, 2012.

[5]

H. Childs, D. Pugmire, S. Ahern, B. Whitlock, M. Howison, Prabhat, G. H. Weber, and E. W. Bethel. Extreme scaling of production visualization software on diverse architectures. IEEE Comput. Graph. Appl., 30(3):22--31, May 2010.

Digital Library

[6]

H. Childs, D. Pugmire, S. Ahern, B. Whitlock, M. Howison, Prabhat, G. H. Weber, and E. W. Bethel. Visualization at extreme scale concurrency. In E. W. Bethel, H. Childs, and C. Hansen, editors, High Performance Visualization: Enabling Extreme-Scale Scientific Insight. CRC Press, Boca Raton, FL, 2012.

[7]

C. Docan, M. Parashar, and S. Klasky. Dataspaces: an interaction and coordination framework for coupled simulation workflows. Cluster Computing, 15(2):163--181, 2012.

Digital Library

[8]

C. Docan, F. Zhang, T. Jin, H. Bui, Q. Sun, J. Cummings, N. Podhorszki, S. Klasky, and M. Parashar. Activespaces: Exploring dynamic code deployment for extreme scale data processing. Concurrency and Computation: Practice and Experience, pages 1--22, 2014.

[9]

N. Fabian, K. Moreland, D. Thompson, A. Bauer, P. Marion, B. Geveci, M. Rasquin, and K. Jansen. The paraview coprocessing library: A scalable, general purpose in situ visualization library. In Large Data Analysis and Visualization (LDAV), 2011 IEEE Symposium on, pages 89--96. IEEE, 2011.

[10]

Q. Liu, J. Logan, Y. Tian, H. Abbasi, N. Podhorszki, J. Y. Choi, S. Klasky, R. Tchoua, J. Lofstead, R. Oldfield, M. Parashar, N. Samatova, K. Schwan, A. Shoshani, M. Wolf, K. Wu, and W. Yu. Hello adios: the challenges and lessons of developing leadership class i/o frameworks. Concurrency and Computation: Practice and Experience, 26(7):1453--1473, 2014.

Digital Library

[11]

L.-t. Lo, C. Sewell, and J. P. Ahrens. Piston: A portable cross-platform framework for data-parallel visualization operators. In EGPGV, pages 11--20, 2012.

[12]

J. F. Lofstead, S. Klasky, K. Schwan, N. Podhorszki, and C. Jin. Flexible io and integration for scientific codes through the adaptable io system (adios). In Proceedings of the 6th international workshop on Challenges of large applications in distributed environments, CLADE '08, pages 15--24, New York, NY, USA, 2008. ACM.

Digital Library

[13]

J. S. Meredith, S. Ahern, D. Pugmire, and R. Sisneros. EAVL: the extreme-scale analysis and visualization library. In Eurographics Symposium on Parallel Graphics and Visualization, pages 21--30. The Eurographics Association, 2012.

[14]

J. S. Meredith, R. Sisneros, D. Pugmire, and S. Ahern. A distributed data-parallel framework for analysis and visualization algorithm development. In Proceedings of the 5th Annual Workshop on General Purpose Processing with Graphics Processing Units, GPGPU-5, pages 11--19, New York, NY, USA, 2012. ACM.

Digital Library

[15]

K. Moreland, U. Ayachit, B. Geveci, and K.-L. Ma. Dax toolkit: A proposed framework for data analysis and visualization at extreme scale. In Large Data Analysis and Visualization (LDAV), 2011 IEEE Symposium on, pages 97--104, Oct 2011.

[16]

K. Moreland, R. Oldfield, P. Marion, S. Jourdain, N. Podhorszki, V. Vishwanath, N. Fabian, C. Docan, M. Parashar, M. Hereld, et al. Examples of in transit visualization. In Proceedings of the 2nd international workshop on Petascale data analytics: challenges and opportunities, pages 1--6. ACM, 2011.

Digital Library

[17]

D. Pugmire, J. Kress, J. Meredith, N. Podhorszki, J. Choi, and S. Klasky. Towards scalable visualization plugins for data staging workflows. In Big Data Analytics: Challenges and Opportunities (BDAC-14) Workshop at Supercomputing Conference, November 2014.

[18]

C. Sewell, J. Meredith, K. Moreland, T. Peterka, D. DeMarle, L.-t. Lo, J. Ahrens, R. Maynard, and B. Geveci. The sdav software frameworks for visualization and analysis on next-generation multi-core and many-core architectures. In High Performance Computing, Networking, Storage and Analysis (SCC), 2012 SC Companion:, pages 206--214. IEEE, 2012.

Digital Library

[19]

V. Vishwanath, M. Hereld, and M. Papka. Toward simulation-time data analysis and i/o acceleration on leadership-class systems. In Large Data Analysis and Visualization (LDAV), 2011 IEEE Symposium on, pages 9--14, 2011.

[20]

B. Whitlock, J. M. Favre, and J. S. Meredith. Parallel in situ coupling of simulation with a fully featured visualization system. In Proceedings of the 11th Eurographics conference on Parallel Graphics and Visualization, pages 101--109. Eurographics Association, 2011.

Digital Library

Cited By

Zhang BDavis PZhang ZTeranishi KParashar M(2024)Dual Channel Dual Staging: Hierarchical and Portable Staging for GPU-Based In-Situ Workflow2024 IEEE 31st International Conference on High Performance Computing, Data, and Analytics (HiPC)10.1109/HiPC62374.2024.00027(188-198)Online publication date: 18-Dec-2024
https://doi.org/10.1109/HiPC62374.2024.00027
Kress JHolst GDasari HAfzal SHoteit ITheußl T(2023)Inshimtu – A Lightweight In Situ Visualization “Shim”High Performance Computing10.1007/978-3-031-40843-4_19(257-268)Online publication date: 25-Aug-2023
https://doi.org/10.1007/978-3-031-40843-4_19
Zhang BDavis PMorales NZhang ZTeranishi KParashar M(2023)Optimizing Data Movement for GPU-Based In-Situ Workflow Using GPUDirect RDMAEuro-Par 2023: Parallel Processing10.1007/978-3-031-39698-4_22(323-338)Online publication date: 24-Aug-2023
https://doi.org/10.1007/978-3-031-39698-4_22
Show More Cited By

Index Terms

Loosely Coupled In Situ Visualization: A Perspective on Why It's Here to Stay
1. Information systems
  1. Information retrieval
    1. Document representation
  2. Information storage systems

Recommendations

Interactive in Situ Visualization
High Performance Computing. ISC High Performance 2024 International Workshops
Abstract
To achieve interactive in situ visualization in immersive virtual environments, the parallel visualization tool Vistle has been coupled with the SENSEI in situ framework. Vistle’s modular architecture, designed for distributed systems, allows in ...
Cinema image-based in situ analysis and visualization of MPAS-ocean simulations

We created an in situ exploration visualization of an MPAS-Ocean simulation.We leveraged compositing in Cinema to provide interactive exploration.We decreased the storage footprint of the analysis and visualization results. Due to power and I/O ...
LOOM: Interweaving tightly coupled visualization and numeric simulation framework
ISAV'21: ISAV'21: In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization

Traditional post-hoc high-fidelity scientific visualization (HSV) of numerical simulations requires multiple I/O check-pointing to inspect the simulation progress. The costs of these I/O operations are high and can grow exponentially with increasing ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ISAV2015: Proceedings of the First Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization

November 2015

51 pages

ISBN:9781450340038

DOI:10.1145/2828612

Publications Chair:
Gunther H. Weber
Lawrence Berkeley National Laboratory, USA

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 15 November 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Funding Sources

Scientific Data Management, Analysis, and Visualization

Conference

SC15

Sponsor:

SC15: The International Conference for High Performance Computing, Networking, Storage and Analysis

November 15 - 20, 2015

TX, Austin, USA

Acceptance Rates

ISAV2015 Paper Acceptance Rate 8 of 19 submissions, 42%;

Overall Acceptance Rate 23 of 63 submissions, 37%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

21
Total Citations
View Citations
300
Total Downloads

Downloads (Last 12 months)7
Downloads (Last 6 weeks)1

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhang BDavis PZhang ZTeranishi KParashar M(2024)Dual Channel Dual Staging: Hierarchical and Portable Staging for GPU-Based In-Situ Workflow2024 IEEE 31st International Conference on High Performance Computing, Data, and Analytics (HiPC)10.1109/HiPC62374.2024.00027(188-198)Online publication date: 18-Dec-2024
https://doi.org/10.1109/HiPC62374.2024.00027
Kress JHolst GDasari HAfzal SHoteit ITheußl T(2023)Inshimtu – A Lightweight In Situ Visualization “Shim”High Performance Computing10.1007/978-3-031-40843-4_19(257-268)Online publication date: 25-Aug-2023
https://doi.org/10.1007/978-3-031-40843-4_19
Zhang BDavis PMorales NZhang ZTeranishi KParashar M(2023)Optimizing Data Movement for GPU-Based In-Situ Workflow Using GPUDirect RDMAEuro-Par 2023: Parallel Processing10.1007/978-3-031-39698-4_22(323-338)Online publication date: 24-Aug-2023
https://doi.org/10.1007/978-3-031-39698-4_22
Wang ZDorier MParashar M(2022)Research Perspectives Toward Autonomic Optimization of In Situ Analysis and Visualization2022 IEEE/ACM International Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization (ISAV)10.1109/ISAV56555.2022.00007(7-13)Online publication date: Nov-2022
https://doi.org/10.1109/ISAV56555.2022.00007
Usher WPark HLee MNavrátil PFussell DPascucci V(2022)A Simulation-Oblivious Data Transport Model for Flexible In Transit VisualizationIn Situ Visualization for Computational Science10.1007/978-3-030-81627-8_18(399-419)Online publication date: 5-May-2022
https://doi.org/10.1007/978-3-030-81627-8_18
Pugmire DPodhorszki NKlasky SWolf MKress JKim MThompson NLogan JWang RMehta KSuchyta EGodoy WChoi JOstrouchov GWan LChen JAtkins BRoss CEisenhauer GGu JWu JHuebl ATsutsumi S(2022)The Adaptable IO System (ADIOS)In Situ Visualization for Computational Science10.1007/978-3-030-81627-8_11(233-254)Online publication date: 5-May-2022
https://doi.org/10.1007/978-3-030-81627-8_11
Krämer AMaier BRau THuber FKlotz TErtl TGöddeke DMehl MReina GRöhrle O(2022)Multi-physics Multi-scale HPC Simulations of Skeletal MusclesHigh Performance Computing in Science and Engineering '2010.1007/978-3-030-80602-6_13(185-203)Online publication date: 1-Jan-2022
https://doi.org/10.1007/978-3-030-80602-6_13
Ribés ARaffin B(2020)The Challenges of In Situ Analysis for Multiple SimulationsISAV'20 In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization10.1145/3426462.3426468(32-37)Online publication date: 12-Nov-2020
https://dl.acm.org/doi/10.1145/3426462.3426468
Kress JLarsen MChoi JKim MWolf MPodhorszki NKlasky SChilds HPugmire D(2020)Opportunities for Cost Savings with In-Transit VisualizationHigh Performance Computing10.1007/978-3-030-50743-5_8(146-165)Online publication date: 22-Jun-2020
https://dl.acm.org/doi/10.1007/978-3-030-50743-5_8
Malakar PNatarajan VVadhiyar S(2020)Adaptive and Efficient Transfer for Online Remote Visualization of Critical Weather ApplicationsComputational Science – ICCS 202010.1007/978-3-030-50417-5_50(674-693)Online publication date: 15-Jun-2020
https://doi.org/10.1007/978-3-030-50417-5_50
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten