short-paper

Performance Impacts of In Situ Wavelet Compression on Scientific Simulations

Authors:

Matthew Larsen,

Hank ChildsAuthors Info & Claims

ISAV'17: Proceedings of the In Situ Infrastructures on Enabling Extreme-Scale Analysis and Visualization

Pages 37 - 41

https://doi.org/10.1145/3144769.3144773

Published: 12 November 2017 Publication History

Abstract

In situ compression is a compromise between traditional post hoc and emerging in situ visualization and analysis. While the merits and limitations of various compressor options have been well studied, their performance impacts on scientific simulations are less clear, especially on large scale supercomputer systems. This study fills in this gap by performing in situ compression experiments on a leading supercomputer system. More specifically, we measured the computational and I/O impacts of a lossy wavelet compressor and analyzed the results with respect to various in situ processing concerns. We believe this study provides a better understanding of in situ compression as well as new evidence supporting its viability, in particular for wavelets.

References

[1]

James Ahrens, Sébastien Jourdain, Patrick O'Leary, John Patchett, David H Rogers, and Mark Petersen. 2014. An image-based approach to extreme scale in situ visualization and analysis. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis. IEEE Press, 424--434.

Digital Library

[2]

Allison H Baker, Haiying Xu, John M Dennis, Michael N Levy, Doug Nychka, Sheri A Mickelson, Jim Edwards, Mariana Vertenstein, and Al Wegener. 2014. A methodology for evaluating the impact of data compression on climate simulation data. In Proceedings of the 23rd international symposium on High-performance parallel and distributed computing. ACM, 203--214.

Digital Library

[3]

A. C. Bauer, H. Abbasi, J. Ahrens, H. Childs, B. Geveci, S. Klasky, K. Moreland, P. O'Leary, V. Vishwanath, B. Whitlock, and E. W. Bethel. 2016. In Situ Methods, Infrastructures, and Applications on High Performance Computing Platforms. In Proceedings of the Eurographics: State of the Art Reports (EuroVis '16). Eurographics Association, Goslar, Germany, 577--597.

[4]

M. Burtscher and P. Ratanaworabhan. 2009. FPC: A high-speed compressor for double-precision floating-point data. IEEE Trans. Comput. 58, 1 (2009), 18--31.

Digital Library

[5]

Albert Cohen, Ingrid Daubechies, and J-C Feauveau. 1992. Biorthogonal bases of compactly supported wavelets. Communications on pure and applied mathematics 45, 5 (1992), 485--560.

[6]

Computational and Information Systems Laboratory. 2017. Cheyenne: SGI ICE XA System. Boulder, CO: National Center for Atmospheric Research.

[7]

Sheng Di and Frank Cappello. 2016. Fast Error-bounded Lossy HPC Data Compression with SZ. In 2016 IEEE International Parallel and Distributed Processing Symposium (IPDPS). IEEE, IEEE, Chicago, IL.

[8]

Nathan Fabian, Kenneth Moreland, David Thompson, Andrew C Bauer, Pat Marion, Berk Gevecik, Michel Rasquin, and Kenneth E Jansen. 2011. The paraview coprocessing library: A scalable, general purpose in situ visualization library. In IEEE Symposium on Large Data Analysis and Visualization (LDAV). IEEE, 89--96.

[9]

Ian Karlin, Jeff Keasler, and Rob Neely. 2013. LULESH 2.0 Updates and Changes. Technical Report LLNL-TR-641973. 1--9 pages.

[10]

Beong-Jo Kim and William A Pearlman. 1997. An embedded wavelet video coder using three-dimensional set partitioning in hierarchical trees (SPIHT). In Proceedings of Data Compression Conference (DCC'97). IEEE, 251--260.

[11]

Sriram Lakshminarasimhan, Neil Shah, Stephane Ethier, Scott Klasky, Rob Latham, Rob Ross, and Nagiza F Samatova. 2011. Compressing the incompressible with ISABELA: In-situ reduction of spatio-temporal data. In Euro-Par 2011 Parallel Processing. Springer, 366--379.

[12]

Daniel Laney, Steven Langer, Christopher Weber, Peter Lindstrom, and Al Wegener. 2013. Assessing the effects of data compression in simulations using physically motivated metrics. In Proceedings of SC13: International Conference for High Performance Computing, Networking, Storage and Analysis. ACM, 76.

Digital Library

[13]

Matthew Larsen et al. 2015. Strawman: A Batch In Situ Visualization and Analysis Infrastructure for Multi-Physics Simulation Codes. In Proceedings of the Workshop on In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization (ISAV2015). ACM, New York, NY, USA, 30--35.

Digital Library

[14]

Matthew Larsen, James Aherns, Utkarsh Ayachit, Eric Brugger, Hank Childs, Berk Geveci, and Cyrus Harrison. 2017. The ALPINE In Situ Infrastructure: Ascending from the Ashes of Strawman. In Proceedings of the In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization Workshop (ISAV2017). ACM, New York, NY, USA.

Digital Library

[15]

Shaomeng Li, Kenny Gruchalla, Kristin Potter, John Clyne, and Hank Childs. 2015. Evaluating the Efficacy of Wavelet Configurations on Turbulent-Flow Data. In IEEE Symposium on Large Data Analysis and Visualization (LDAV). 81--89.

Digital Library

[16]

Shaomeng Li, Nicole Marsaglia, Vincent Chen, Christopher Sewell, John Clyne, and Hank Childs. 2017. Achieving Portable Performance For Wavelet Compression Using Data Parallel Primitives. In Proceedings of EuroGraphics Symposium on Parallel Graphics and Visualization (EGPGV). The Eurographics Association.

[17]

Peter Lindstrom. 2014. Fixed-rate compressed floating-point arrays. IEEE Transactions on Visualization and Computer Graphics 20, 12 (2014), 2674--2683.

[18]

Peter Lindstrom and Martin Isenburg. 2006. Fast and efficient compression of floating-point data. IEEE Transactions on Visualization and Computer Graphics 12, 5 (2006), 1245--1250.

Digital Library

[19]

Kenneth Moreland, Christopher Sewell, William Usher, Li-ta Lo, Jeremy Meredith, David Pugmire, James Kress, Hendrik Schroots, Kwan-Liu Ma, Hank Childs, et al. 2016. Vtk-m: Accelerating the visualization toolkit for massively threaded architectures. IEEE computer graphics and applications 36, 3 (2016), 48--58.

[20]

Jurriaan D. Mulder, Jarke J. van Wijk, and Robert van Liere. 1999. A survey of computational steering environments. Future Generation Computer Systems 15, 1 (1999), 119--129.

Digital Library

[21]

A. Norton and J. Clyne. 2012. The VAPOR Visualization Application. In High Performance Visualization, E.W. Bethel, H. Childs, and C. Hanson (Eds.). 415--428.

[22]

Xiaoli Tang, William A Pearlman, and James W Modestino. 2003. Hyperspectral image compression using three-dimensional wavelet coding. In Electronic Imaging 2003. International Society for Optics and Photonics, 1037--1047.

[23]

Dingwen Tao, Sheng Di, Zizhong Chen, and Franck Cappello. 2017. Significantly Improving Lossy Compression for Scientific Data Sets Based on Multidimensional Prediction and Error-Controlled Quantization. Proceedings of the 31th IEEE International Parallel and Distributed Processing Symposium (2017).

[24]

Anna Tikhonova, Carlos D Correa, and Kwan-Liu Ma. 2010. Visualization by proxy: A novel framework for deferred interaction with volume data. IEEE Transactions on Visualization and Computer Graphics 16, 6 (2010), 1551--1559.

Digital Library

[25]

Brad Whitlock, Jean M Favre, and Jeremy S Meredith. 2011. Parallel in situ coupling of simulation with a fully featured visualization system. In Proceedings of the 11th Eurographics conference on Parallel Graphics and Visualization. Eurographics Association, 101--109.

Digital Library

[26]

J. Woodring, S. Mniszewski, C. Brislawn, D. DeMarle, and J. Ahrens. 2011. Revisiting wavelet compression for large-scale climate data using JPEG 2000 and ensuring data precision. In IEEE Symposium on Large Data Analysis and Visualization (LDAV). 31--38.

[27]

Sean B Ziegeler. 2016. An I/O Mini-App dedicated to in situ visualization. In Proceedings of the In Situ Infrastructures for Enabling Extreme-Scale Analysis and Visualization Workshop (ISAV2016). IEEE, 29--34.

Digital Library

Cited By

Moon ASon SKim HKim M(2022)Lossy Predictive Models for Accurate Classification Algorithms2022 IEEE International Conference on Big Data (Big Data)10.1109/BigData55660.2022.10020381(4576-4582)Online publication date: 17-Dec-2022
https://doi.org/10.1109/BigData55660.2022.10020381
Li SClyne JChilds H(2022)In Situ Wavelet Compression on Supercomputers for Post Hoc ExplorationIn Situ Visualization for Computational Science10.1007/978-3-030-81627-8_3(37-59)Online publication date: 5-May-2022
https://doi.org/10.1007/978-3-030-81627-8_3
Li SJaroszynski SPearse SOrf LClyne J(2019)VAPOR: A Visualization Package Tailored to Analyze Simulation Data in Earth System ScienceAtmosphere10.3390/atmos1009048810:9(488)Online publication date: 25-Aug-2019
https://doi.org/10.3390/atmos10090488
Show More Cited By

Recommendations

Optimizing in-situ data compression for large-scale scientific simulations
HPC '16: Proceedings of the 24th High Performance Computing Symposium

Large-scale scientific simulations running in HPC environments can produce massive amounts of data whose sheer size poses challenges to data storage, transport, and analysis. Moreover, as improvements of file systems do not match the progress in the ...
A Comparative Study of Wavelet Coders for Image Compression
MIKE 2013: Proceedings of the First International Conference on Mining Intelligence and Knowledge Exploration - Volume 8284

This paper focuses on comparison of different wavelet coders such as SPIHT, SPECK, BISK and TARP for efficient storage and better transmission. Set partition methods like SPIHT, SPECK and BISK (variant of SPECK) are based on the popular bit-plane coding ...
Wavelet Image Compression

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

ISAV'17: Proceedings of the In Situ Infrastructures on Enabling Extreme-Scale Analysis and Visualization

November 2017

53 pages

ISBN:9781450351393

DOI:10.1145/3144769

Copyright © 2017 ACM.

Publication rights licensed to ACM. ACM acknowledges that this contribution was authored or co-authored by an employee, contractor or affiliate of the United States government. As such, the Government retains a nonexclusive, royalty-free right to publish or reproduce this article, or to allow others to do so, for Government purposes only.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 November 2017

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Short-paper
Research
Refereed limited

Conference

SC '17

Sponsor:

SIGHPC

SC '17: The International Conference for High Performance Computing, Networking, Storage and Analysis

November 12 - 17, 2017

CO, Denver, USA

Acceptance Rates

ISAV'17 Paper Acceptance Rate 9 of 28 submissions, 32%;

Overall Acceptance Rate 23 of 63 submissions, 37%

Upcoming Conference

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
81
Total Downloads

Downloads (Last 12 months)2
Downloads (Last 6 weeks)0

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Moon ASon SKim HKim M(2022)Lossy Predictive Models for Accurate Classification Algorithms2022 IEEE International Conference on Big Data (Big Data)10.1109/BigData55660.2022.10020381(4576-4582)Online publication date: 17-Dec-2022
https://doi.org/10.1109/BigData55660.2022.10020381
Li SClyne JChilds H(2022)In Situ Wavelet Compression on Supercomputers for Post Hoc ExplorationIn Situ Visualization for Computational Science10.1007/978-3-030-81627-8_3(37-59)Online publication date: 5-May-2022
https://doi.org/10.1007/978-3-030-81627-8_3
Li SJaroszynski SPearse SOrf LClyne J(2019)VAPOR: A Visualization Package Tailored to Analyze Simulation Data in Earth System ScienceAtmosphere10.3390/atmos1009048810:9(488)Online publication date: 25-Aug-2019
https://doi.org/10.3390/atmos10090488
Marsaglia NLi SChilds H(2019)Enabling Explorative Visualization with Full Temporal Resolution via In Situ Calculation of Temporal IntervalsHigh Performance Computing10.1007/978-3-030-02465-9_19(273-293)Online publication date: 25-Jan-2019
https://doi.org/10.1007/978-3-030-02465-9_19
Li SMarsaglia NGarth CWoodring JClyne JChilds H(2018)Data Reduction Techniques for Simulation, Visualization and Data AnalysisComputer Graphics Forum10.1111/cgf.1333637:6(422-447)Online publication date: 30-Mar-2018
https://doi.org/10.1111/cgf.13336
Moon AKim JZhang JSon S(2018)Evaluating fidelity of lossy compression on spatiotemporal data from an IoT enabled smart farmComputers and Electronics in Agriculture10.1016/j.compag.2018.08.045154(304-313)Online publication date: Nov-2018
https://doi.org/10.1016/j.compag.2018.08.045

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten