skip to main content
column

The case for a versatile storage system

Published: 12 March 2010 Publication History

Abstract

Storage systems in emerging large-scale (a.k.a. peta-scale) computing systems often introduce a performance or scalability bottleneck. To deal with these limitations we propose a new operational approach: versatile storage, an application-optimized and highly configurable storage system that harnesses node-local resources, is configured and deployed at application deployment time, and has a lifetime dependent on the application lifetime. Our prototype evaluation, using synthetic and application-level benchmarks, on a small cluster as well as on a 96K processor machine, provides evidence that the versatile storage approach can bring valuable benefits to large scale deployments in terms of storage system performance and scalability.

References

[1]
S.F. Altschul, W. Gish, W. Miller, E. Myers, et al., Basic Local Alighnment Tool. Molecular Biology, 1990. 215: p. 403--410.
[2]
Overview of DOCK. {cited 2009}; http://dock.compbio.ucsf.edu/Overview_of_DOCK/index.htm
[3]
S. Ghemawat, H. Gobioff, and S.-T. Leung. The Google File System. in 19th ACM Symposium on Operating Systems Principles. 2003. Lake George, NY.
[4]
P.H. Carns, W.B. Ligon-III, R.B. Ross, and R. Thakur. PVFS: A Parallel File System for Linux Clusters. in 4th Annual Linux Showcase and Conference. 2000. Atlanta, GA.
[5]
Z. Zhang, A. Espinosa, K. Iskra, I. Raicu, et al. Design and Evaluation of a Collective I/O Model for Loosely-coupled Programming. in Workshop on Many-TaskComputing on Grids and Supercomputers (MTAGS). 2008.
[6]
S. Al-Kiswany, M. Ripeanu, S. Vazhkudai, and A. Gharaibeh. stdchk: A Checkpoint Storage System for Desktop Grid Computing. in International Conference on Distributed Computing Systems (ICDCS '08). 2008. Beijing, China.
[7]
A. Muthitacharoen, B. Chen, and D. Mazieres. A Lowbandwidth Network File System. SOSP. 2001. Banff, Canada.
[8]
A. Gharaibeh, S. Al-Kiswany, and M. Ripeanu. Configurable Security for Scavenged Storage Systems. in Workshop on Storage Security and Survivability (StorageSS). 2008.
[9]
J. Bent, D. Thain, A.C. Arpaci-Dusseau, R.H. Arpaci-Dusseau, et al. Explicit Control in a Batch-Aware Distributed File System. NSDI.2004. San Francisco, California.
[10]
F. Schmuck and R. Haskin. GPFS: A Shared-Disk File System for Large Computing Clusters. FAST. 2002.
[11]
S.S. Vazhkudai, X. Ma, V.W. Freeh, J.W. Strickland, et al., Constructing collaborative desktop storage caches for large scientific datasets. ACM Transaction on Storage (TOS), 2006. 2(3): p. 221--254.
[12]
W.J. Bolosky, J.R. Douceur, D. Ely, and M. Theimer. Feasibility of a Serverless Distributed File System Deployed on an Existing Set of Desktop PCs. SIGMETRICS. 2000.
[13]
E. Kıcıman, B. Livshits, and M. Musuvathi. FLUXO: A Simple Service Compiler. in Workshop on Hot Topics in Operating Systems (HotOS). 2009.

Cited By

View all
  • (2020)Fine-grained management of I/O optimizations based on workload characteristicsFrontiers of Computer Science: Selected Publications from Chinese Universities10.1007/s11704-020-9344-115:3Online publication date: 31-Dec-2020
  • (2017)Toward Managing HPC Burst Buffers Effectively: Draining Strategy to Regulate Bursty I/O Behavior2017 IEEE 25th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS)10.1109/MASCOTS.2017.35(87-98)Online publication date: Sep-2017
  • (2017)Evaluating a Data-Aware Scheduling Approach to Reduce Processing Costs of DMCF Workflows2017 International Conference on High Performance Computing & Simulation (HPCS)10.1109/HPCS.2017.107(699-706)Online publication date: Jul-2017
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM SIGOPS Operating Systems Review
ACM SIGOPS Operating Systems Review  Volume 44, Issue 1
January 2010
115 pages
ISSN:0163-5980
DOI:10.1145/1740390
Issue’s Table of Contents
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 12 March 2010
Published in SIGOPS Volume 44, Issue 1

Check for updates

Author Tags

  1. dynamic deployment
  2. high performance storage
  3. storage system specialization
  4. versatile storage system

Qualifiers

  • Column

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)6
  • Downloads (Last 6 weeks)0
Reflects downloads up to 20 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2020)Fine-grained management of I/O optimizations based on workload characteristicsFrontiers of Computer Science: Selected Publications from Chinese Universities10.1007/s11704-020-9344-115:3Online publication date: 31-Dec-2020
  • (2017)Toward Managing HPC Burst Buffers Effectively: Draining Strategy to Regulate Bursty I/O Behavior2017 IEEE 25th International Symposium on Modeling, Analysis, and Simulation of Computer and Telecommunication Systems (MASCOTS)10.1109/MASCOTS.2017.35(87-98)Online publication date: Sep-2017
  • (2017)Evaluating a Data-Aware Scheduling Approach to Reduce Processing Costs of DMCF Workflows2017 International Conference on High Performance Computing & Simulation (HPCS)10.1109/HPCS.2017.107(699-706)Online publication date: Jul-2017
  • (2017)Storage Solution of Spatial-Temporal Data for Water Monitoring Infrastructures Used in Smart Cities2017 21st International Conference on Control Systems and Computer Science (CSCS)10.1109/CSCS.2017.94(617-621)Online publication date: May-2017
  • (2017)Experimental evaluation of a flexible I/O architecture for accelerating workflow engines in ultrascale environmentsParallel Computing10.1016/j.parco.2016.10.00361(52-67)Online publication date: Jan-2017
  • (2017)JULEA: A Flexible Storage Framework for HPCHigh Performance Computing10.1007/978-3-319-67630-2_51(712-723)Online publication date: 20-Oct-2017
  • (2017)A data‐aware scheduling strategy for workflow execution in cloudsConcurrency and Computation: Practice and Experience10.1002/cpe.422929:24Online publication date: 14-Aug-2017
  • (2016)Support for Provisioning and Configuration Decisions for Data Intensive WorkflowsIEEE Transactions on Parallel and Distributed Systems10.1109/TPDS.2015.249769327:9(2725-2739)Online publication date: 1-Sep-2016
  • (2016)Exploiting in-memory storage for improving workflow executions in cloud platformsThe Journal of Supercomputing10.1007/s11227-016-1678-y72:11(4069-4088)Online publication date: 1-Nov-2016
  • (2015)Experimental evaluation of a flexible I/O architecture for accelerating workflow engines in cloud environmentsProceedings of the 2015 International Workshop on Data-Intensive Scalable Computing Systems10.1145/2831244.2831248(1-8)Online publication date: 15-Nov-2015
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media