Cited By
View all- Miller PLi SMei C(2011)Asynchronous Collective Output with Non-dedicated CoresProceedings of the 2011 IEEE International Conference on Cluster Computing10.1109/CLUSTER.2011.82(494-502)Online publication date: 26-Sep-2011
A layout-aware optimization strategy for collective I/O
Pages 360 - 363
Abstract
In this study, we propose an optimization strategy to promote a better integration of the parallel I/O middleware and parallel file systems. We illustrate that a layout-aware optimization strategy can improve the performance of current collective I/O in parallel I/O system. We present the motivation, prototype design and initial verification of the proposed layout-aware optimization strategy. The analytical and initial experimental testing results demonstrate that the proposed strategy has a potential in improving the parallel I/O system performance.
References
[1]
}}J. Bent, G. Gibson, G. Grider, B. McClelland, P. Nowoczynski, J. Nunez, M. Polte, M. Wingate, "PLFS: A Checkpoint Filesystem for Parallel Applications," in Proc. of ACM/IEEE Supercomputing'09, 2009.
[2]
}}P. H. Carns, W. B. Ligon III, R. B. Ross, and R. Thakur, "PVFS: A Parallel File System For Linux Clusters," in Proceedings of the 4th Annual Linux Showcase and Conference, 2000.
[3]
}}Y. Chen, S. Byna, X.-H. Sun, R. Thakur, and W. Gropp, "Hiding I/O Latency with Pre-execution Prefetching for Parallel Applications," in Proc. of the ACM/IEEE SuperComputing Conference (SC'08), 2008.
[4]
}}Interleaved or Random (IOR) Benchmark, http://sourceforge.net/projects/ior-sio/.
[5]
}}A. Gharaibeh and M. Ripeanu. Exploring Data Reliability Tradeoffs in Replicated Storage Systems. in Proc. of High Performance Distributed Computing (HPDC), 2009
[6]
}}D. Kotz. Disk-directed I/O for MIMD Multiprocessors. ACM Transactions on Computer Systems, 15(1):41--74, 1997.
[7]
}}J. F. Lofstead, S. Klasky, K. Schwan, N. Podhorszki and C. Jin, "Flexible IO and Integration for Scientific Codes Through the Adaptable IO System (ADIOS)," in Proc. of the 6th International Workshop on Challenges of Large Applications in Distributed Environments, 2008.
[8]
}}J. May, "Parallel I/O for High Performance Computing," Morgan Kaufmann Publishing, 2001.
[9]
}}PVFS2 Development Team, "PVFS Developer's Guide," http://www.pvfs.org/cvs/pvfs-2-8-branch-docs/doc//pvfs2-guide.pdf.
[10]
}}I. Raicu, I. Foster, Y. Zhao, P. Little, C. Moretti, A. Chaudhary, D. Thain. "The Quest for Scalable Support of Data Intensive Workloads in Distributed Systems", in Proc. of ACM High Performance Distributed Computing (HPDC), 2009.
[11]
}}J. del Rosario, R. Bordawekar, and A. Choudhary, "Improved Parallel I/O via a Two-Phase Run-time Access Strategy", in Proc. of the Workshop on I/O in Parallel Computer Systems at IPPS '93, 1993.
[12]
}}K. Seamons, Y. Chen, P. Jones, J. Jozwiak and M. Winslett, "Server-Directed Collective I/O in Panda", in Proc. of Supercomputing'95. ACM Press, 1995.
[13]
}}X.-H. Sun, Y. Chen and Y. Yin, "Data Layout Optimization for Petascale File Systems," in Proc. of The 4th Petascale Data Storage Workshop, 2009.
[14]
}}R. Thakur, W. Gropp and E. Lusk, "Data Sieving and Collective I/O in ROMIO," in Proceedings of the 7th Symposium on the Frontiers of Massively Parallel Computation, 1999.
[15]
}}R. Thakur and A. Choudhary, "An Extended Two-Phase Method for Accessing Sections of Out-of-Core Arrays," Scientific Programming, (5)4:301--317, Winter 1996.
[16]
}}Top 500 Supercomputing Website. http://www.top500.org.
Index Terms
- A layout-aware optimization strategy for collective I/O
Recommendations
Improving Parallel I/O Performance with Data Layout Awareness
CLUSTER '10: Proceedings of the 2010 IEEE International Conference on Cluster ComputingParallel applications can benefit greatly from massive computational capability, but their performance suffers from large latency of I/O accesses. The poor I/O performance has been attributed as a critical cause of the low sustained performance of ...
Data layout optimization for petascale file systems
PDSW '09: Proceedings of the 4th Annual Workshop on Petascale Data StorageIn this study, the authors propose a simple performance model to promote a better integration between the parallel I/O middleware layer and parallel file systems. They show that application-specific data layout optimization can improve overall data ...
Cost-intelligent application-specific data layout optimization for parallel file systems
Parallel file systems have been developed in recent years to ease the I/O bottleneck of high-end computing system. These advanced file systems offer several data layout strategies in order to meet the performance goals of specific I/O workloads. However,...
Comments
Information & Contributors
Information
Published In
Copyright © 2010 ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]
Sponsors
- University of Arizona: University of Arizona
- SIGARCH: ACM Special Interest Group on Computer Architecture
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 21 June 2010
Check for updates
Author Tags
Qualifiers
- Research-article
Conference
HPDC '10
Sponsor:
- University of Arizona
- SIGARCH
HPDC '10: The 19th International Symposium on High Performance Distributed Computing
June 21 - 25, 2010
Illinois, Chicago
Acceptance Rates
Overall Acceptance Rate 166 of 966 submissions, 17%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 137Total Downloads
- Downloads (Last 12 months)1
- Downloads (Last 6 weeks)0
Reflects downloads up to 05 Mar 2025
Other Metrics
Citations
Cited By
View all- Miller PLi SMei C(2011)Asynchronous Collective Output with Non-dedicated CoresProceedings of the 2011 IEEE International Conference on Cluster Computing10.1109/CLUSTER.2011.82(494-502)Online publication date: 26-Sep-2011
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in