skip to main content
10.1145/1383519.1383526acmconferencesArticle/Chapter ViewAbstractPublication PageshpdcConference Proceedingsconference-collections
research-article

Amazon S3 for science grids: a viable solution?

Published: 24 June 2008 Publication History

Abstract

Amazon.com has introduced the Simple Storage Service (S3), a commodity-priced storage utility. S3 aims to provide storage as a low-cost, highly available service, with a simple 'pay-as-you-go' charging model. This article makes three contributions. First, we evaluate S3's ability to provide storage support to large-scale science projects from a cost, availability, and performance perspective. Second, we identify a set of additional functionalities that storage services targeting data-intensive science applications should support. Third, we propose unbundling the success metrics for storage utility performance as a solution, to reduce storage costs.

References

[1]
The DZero Experiment. http://www-d0.fnal.gov
[2]
The Large Hadron Collider. http://lcg.web.cern.ch/LCG
[3]
The Stanford Linear Collider. http://www2.slac.stanford.edu/vvc/experiments/slc.html
[4]
The Compact Muon Solenoid at CERN. http://cmsinfo.cern.ch
[5]
Amazon Web Services. http:/s3.amazonaws.com
[6]
PlanetLab Consortium. http://planet-lab.org
[7]
Garfinkel, S. 2007. Commodity Grid Computing with Amazon's S3 and EC2, Login, USENIX, February 2007.
[8]
Kirkpatrick, M. 2006. Amazon releases early info on S3 storage use, http://www.techcrunch.com/tag/s3, July 2006.
[9]
Bezos, J. keynote talk at Web 2.0 Expo, April 2007.
[10]
Krawczyk, H., Bellare, M., and Canetti, R. 1997. HMAC: Keyed-Hashing for Message Authentication, IETF - Network Working Group, 1997.
[11]
W3C, Soap Version 1.2, June 2003, http://www.w3.org/TR/soap/
[12]
Fielding, R. T. 2000. Architectural Styles and the Design of Network-Based Software Architectures, PhD Dissertation, University of California, Irvine, 2000.
[13]
BitTorrent. http://www.bittorrent.com
[14]
Doraimani, S. 2007. Filecules: A New Granularity for Resource Management in Grids, Masters Thesis, University of South Florida, April 2007.
[15]
Iamnitchi, A., Doraimani, S., and Garzoglio, G. 2006. Filecules in High-Energy Physics: Characteristics and Impact on Resource Management, 15th IEEE International Symposium on High Performance Distributed Computing (HPDC), June 2006.
[16]
Oreily. REST vs. SOAP at Amazon. http://www.oreillynet.com/pub/wlg/3005?wlg=yes
[17]
Paxson, V. 1996. End-to-end routing behavior in the Internet, Proceedings of the ACM SIGCOMM Conference on Applications, Technologies, Architectures, and Protocols for Computer Communications, vol 26.4, ACM SiGCOMM Computer Communication Review, pages 25--38, ACM Press, New York, August 1996.
[18]
Garfinkel, S. 2007. An Evaluation of Amazon's Grid Computing Services: EC2, S3 and SQS, Technical Report TR-08-07, School for Engineering and Applied Sciences, Harvard University, MA. July 2007.
[19]
TeraGrid. http://www.teragrid.org
[20]
Odlyzko, A. M. 2000. The Internet and other networks: Utilization rates and their implications, Information Economics & Policy, vol. 12, 2000.
[21]
Odlyzko, A. M. 1999. The current state and likely evolution of the Internet, in Proceedings of Globecom'99, pp. 1869--1875, IEEE, 1999.
[22]
Western Canada Research Grid. http://www.westgrid.ca
[23]
Ripeanu, M. and Iamnitchi, A. 2007. S4: A Simple Storage Service for Sciences, 16th IEEE International Symposium on High Performance Distributed Computing (HPDC), Monterey Bay, CA, June 2007.
[24]
Palankar, M., Onibokun, A., Iamnitchi, A., and Ripeanu, M. 2007. Amazon S3 for Science Grids: a Viable Solution?, poster, 4th USENIX Symposium on Networked Systems Design & Implementation (NSDI'07), April 2007.

Cited By

View all
  • (2025)Parallel Peeling of Invertible Bloom Lookup Tables in a Constant Number of RoundsSOFSEM 2025: Theory and Practice of Computer Science10.1007/978-3-031-82697-9_6(70-84)Online publication date: 21-Jan-2025
  • (2024)Building a FAIR data ecosystem for incorporating single-cell transcriptomics data into agricultural genome to phenome researchFrontiers in Genetics10.3389/fgene.2024.146035115Online publication date: 29-Nov-2024
  • (2024)Fairness-Efficiency Scheduling for Pay-as-You-Go Shared Caching Systems With Long-Term Fairness GuaranteesIEEE Transactions on Services Computing10.1109/TSC.2023.333504817:5(2417-2431)Online publication date: Sep-2024
  • Show More Cited By

Index Terms

  1. Amazon S3 for science grids: a viable solution?

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    DADC '08: Proceedings of the 2008 international workshop on Data-aware distributed computing
    June 2008
    72 pages
    ISBN:9781605581545
    DOI:10.1145/1383519
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 24 June 2008

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. amazon simple storage service
    2. performance evaluation
    3. utility computing

    Qualifiers

    • Research-article

    Conference

    HPDC '08
    Sponsor:

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)104
    • Downloads (Last 6 weeks)16
    Reflects downloads up to 20 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2025)Parallel Peeling of Invertible Bloom Lookup Tables in a Constant Number of RoundsSOFSEM 2025: Theory and Practice of Computer Science10.1007/978-3-031-82697-9_6(70-84)Online publication date: 21-Jan-2025
    • (2024)Building a FAIR data ecosystem for incorporating single-cell transcriptomics data into agricultural genome to phenome researchFrontiers in Genetics10.3389/fgene.2024.146035115Online publication date: 29-Nov-2024
    • (2024)Fairness-Efficiency Scheduling for Pay-as-You-Go Shared Caching Systems With Long-Term Fairness GuaranteesIEEE Transactions on Services Computing10.1109/TSC.2023.333504817:5(2417-2431)Online publication date: Sep-2024
    • (2024)A Lightweight Certificateless Multi-User Matchmaking Encryption for Mobile Devices: Enhancing Security and PerformanceIEEE Transactions on Information Forensics and Security10.1109/TIFS.2023.332196119(251-264)Online publication date: 2024
    • (2024)SoK: Decentralized storage networkHigh-Confidence Computing10.1016/j.hcc.2024.1002394:3(100239)Online publication date: Sep-2024
    • (2024)StructMesh: A storage framework for serverless computing continuumFuture Generation Computer Systems10.1016/j.future.2024.05.033159(353-369)Online publication date: Oct-2024
    • (2024)Knowledge Graph for Solubility Big Data: Construction and ApplicationsWIREs Data Mining and Knowledge Discovery10.1002/widm.1570Online publication date: Nov-2024
    • (2023)SoK: Distributed Computing in ICNProceedings of the 10th ACM Conference on Information-Centric Networking10.1145/3623565.3623712(88-100)Online publication date: 9-Oct-2023
    • (2023)Study of Cloud Providers (Azure, Amazon, and Oracle) According To Service Availability and Price2023 3rd International Conference on Pervasive Computing and Social Networking (ICPCSN)10.1109/ICPCSN58827.2023.00200(1177-1188)Online publication date: Jun-2023
    • (2023)A smart chicken farming platform for chicken behavior identification and feed residual estimation2023 IEEE International Conference on Bioinformatics and Biomedicine (BIBM)10.1109/BIBM58861.2023.10385923(1627-1634)Online publication date: 5-Dec-2023
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media