skip to main content
10.1145/2535571.2535593acmconferencesArticle/Chapter ViewAbstractPublication PagesscConference Proceedingsconference-collections
research-article

A model for optimizing file access patterns using spatio-temporal parallelism

Published:17 November 2013Publication History

ABSTRACT

For many years now, I/O read time has been recognized as the primary bottleneck for parallel visualization and analysis of large-scale data. In this paper, we introduce a model that can estimate the read time for a file stored in a parallel filesystem when given the file access pattern. Read times ultimately depend on how the file is stored and the access pattern used to read the file. The file access pattern will be dictated by the type of parallel decomposition used. We employ spatio-temporal parallelism, which combines both spatial and temporal parallelism, to provide greater flexibility to possible file access patterns. Using our model, we were able to configure the spatio-temporal parallelism to design optimized read access patterns that resulted in a speedup factor of approximately 400 over traditional file access patterns.

References

  1. UV-CDAT Spatio-Temporal Parallel Processing Tools. http://uv-cdat.llnl.gov/presentations/PDF/ParaViewSTPWiki.pdf, 2013.Google ScholarGoogle Scholar
  2. J. Biddiscombe, B. Geveci, K. Martin, K. Moreland, and D. Thompson. Time dependent processing in a parallel pipeline architecture. IEEE Transactions on Visualization and Computer Graphics, 13(6): 1376--1383, Nov. 2007. Google ScholarGoogle ScholarDigital LibraryDigital Library
  3. D. Camp, H. Childs, A. Chourasia, C. Garth, and K. I. Joy. Evaluating the benefits of an extended memory hierarchy for parallel streamline algorithms. In Large Data Analysis and Visualization (LDAV), 2011 IEEE Symposium on, pages 57--64. IEEE, 2011.Google ScholarGoogle ScholarCross RefCross Ref
  4. H. Childs, D. Pugmire, S. Ahern, B. Whitlock, M. Howison, G. H. Weber, E. W. Bethel, et al. Extreme scaling of production visualization software on diverse architectures. Computer Graphics and Applications, IEEE, 30(3): 22--31, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. N. Fabian, K. Moreland, D. Thompson, A. C. Bauer, P. Marion, B. Gevecik, M. Rasquin, and K. E. Jansen. The paraview coprocessing library: A scalable, general purpose in situ visualization library. In Large Data Analysis and Visualization (LDAV), 2011 IEEE Symposium on, pages 89--96. IEEE, 2011.Google ScholarGoogle ScholarCross RefCross Ref
  6. W. Kendall, J. Huang, T. Peterka, R. Latham, and R. Ross. Toward a general i/o layer for parallel-visualization applications. Computer Graphics and Applications, IEEE, 31(6): 6--10, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. C. Michell, J. Ahrens, and J. Wang. Visio: Enabling interactive visualization of ultra-scale, time series data via high-bandwidth distributed i/o systems. pages 1--12. IEEE International Parallel and Distributed Processing Symposium, May 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. M. L. Norman and A. Snavely. Accelerating data-intensive science with gordon and dash. In Proceedings of the 2010 TeraGrid Conference, page 14. ACM, 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. T. Peterka, R. Ross, A. Gyulassy, V. Pascucci, W. Kendall, H.-W. Shen, T.-Y. Lee, and A. Chaudhuri. Scalable parallel building blocks for custom data analysis. In Large Data Analysis and Visualization (LDAV), 2011 IEEE Symposium on, pages 105--112. IEEE, 2011.Google ScholarGoogle ScholarCross RefCross Ref
  10. Prabhat, O. Rbel, S. Byna, K. Wu, F. Li, M. Wehner, and W. Bethel. Teca: A parallel toolkit for extreme climate analysis. Procedia Computer Science, 9(0): 866--876, 2012. Proceedings of the International Conference on Computational Science, 2012.Google ScholarGoogle Scholar
  11. V. Vishwanath, M. Hereld, and M. E. Papka. Toward simulation-time data analysis and i/o acceleration on leadership-class systems. In Large Data Analysis and Visualization (LDAV), 2011 IEEE Symposium on, pages 9--14. IEEE, 2011.Google ScholarGoogle ScholarCross RefCross Ref
  12. B. Whitlock, J. M. Favre, and J. S. Meredith. Parallel in situ coupling of simulation with a fully featured visualization system. In Proceedings of the 11th Eurographics conference on Parallel Graphics and Visualization, pages 101--109. Eurographics Association, 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. D. Williams, C. Doutriaux, J. Patchett, S. Williams, G. Shipman, R. Miller, C. Steed, H. Krishnan, C. Silva, A. Chaudhary, P. Bremer, D. Pugmire, W. Bethel, H. Childs, M. Prabhat, B. Geveci, A. Bauer, A. Pletzer, J. Poco, T. Ellqvist, E. Santos, G. Potter, B. Smith, T. Maxwell, D. Kindig, and D. Koop. The ultra-scale visualization climate data analysis tools (uv-cdat): Data analysis and visualization for geoscience data. Computer, PP(99): 1--1, 2013.Google ScholarGoogle Scholar
  14. M. Woitaszek, J. M. Dennis, and T. R. Sines. Parallel high-resolution climate data analysis using swift. In Proceedings of the 2011 ACM international workshop on Many task computing on grids and supercomputers, MTAGS '11, pages 5--14, New York, NY, USA, 2011. ACM. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. J. Woodring, S. Mniszewski, C. Brislawn, D. DeMarle, and J. Ahrens. Revisiting wavelet compression for large-scale climate data using jpeg 2000 and ensuring data precision. In Large Data Analysis and Visualization (LDAV), 2011 IEEE Symposium on, pages 31--38. IEEE, 2011.Google ScholarGoogle ScholarCross RefCross Ref
  16. H. Yu and K.-L. Ma. A study of i/o methods for parallel visualization of large-scale data. Parallel Computing, 31(2): 167--183, 2005. Parallel Graphics and Visualization. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. H. Yu, K.-L. Ma, and J. Welling. A parallel visualization pipeline for terascale earthquake simulations. In Proceedings of the 2004 ACM/IEEE conference on Supercomputing, SC '04, pages 49--, Washington, DC, USA, 2004. IEEE Computer Society. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. A model for optimizing file access patterns using spatio-temporal parallelism

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in
        • Published in

          cover image ACM Conferences
          UltraVis '13: Proceedings of the 8th International Workshop on Ultrascale Visualization
          November 2013
          56 pages
          ISBN:9781450325004
          DOI:10.1145/2535571

          Copyright © 2013 ACM

          Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 17 November 2013

          Permissions

          Request permissions about this article.

          Request Permissions

          Check for updates

          Qualifiers

          • research-article

          Acceptance Rates

          UltraVis '13 Paper Acceptance Rate6of7submissions,86%Overall Acceptance Rate6of7submissions,86%
        • Article Metrics

          • Downloads (Last 12 months)2
          • Downloads (Last 6 weeks)0

          Other Metrics

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader