Yang Tang
ABSTRACT
Web applications are getting ubiquitous every day because they offer many useful services to consumers and businesses. Many of these web applications are quite storage-intensive. Cloud computing offers attractive and economical choices for meeting their storage needs. Unfortunately, it remains challenging for developers to best leverage them to minimize cost. This paper presents Grandet, an extensible storage system that significantly reduces storage cost for web applications deployed in the cloud. Grandet provides both a key-value interface and a file system interface, supporting a broad spectrum of web applications. Under the hood, it supports multiple heterogeneous stores and unifies them by placing each data object at the store deemed most economical. We implemented Grandet on Amazon Web Services and evaluated Grandet on a diverse set of four popular open-source web applications. Our results show that Grandet reduces their cost by an average of 42.4%, and it is fast, scalable, and easy to use. The source code of Grandet is at http://columbia.github.io/grandet.
- a Filey System for Amazon S3 written in Go. https://github.com/kahing/goofys.Google Scholar
- H. Abu-Libdeh, L. Princehouse, and H. Weatherspoon. RACS: a case for cloud storage diversity. In Proceedings of the 1st ACM symposium on Cloud computing, pages 229--240. ACM, 2010. Google ScholarDigital Library
- S. Agarwala, D. Jadav, and L. A. Bathen. iCostale: Adaptive cost optimization for storage clouds. In 2011 IEEE International Conference on Cloud Computing (CLOUD), 2011. Google ScholarDigital Library
- Amazon EBS Product Details. http://aws.amazon.com/ebs/details/.Google Scholar
- Amazon EBS Volume Types. http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSVolumeTypes.html.Google Scholar
- Amazon speeds up its cloud with SSD block storage. http://www.networkworld.com/article/2364506/cloud-storage/amazon-speeds-up-its-cloud-with-ssd-block-storage.html.Google Scholar
- Amazon Web Services leads war on cloud price reductions. http://www.techrepublic.com/article/amazon-web-services-lead-the-war-on-cloud-price-reductions/.Google Scholar
- An Amazon Web Services C++ Library. http://libaws.sourceforge.net/.Google Scholar
- D. Beaver, S. Kumar, H. C. Li, J. Sobel, and P. Vajgel. Finding a needle in haystack: Facebook's photo storage. In Proceedings of the 9th USENIX Conference on Operating Systems Design and Implementation, OSDI'10, pages 1--8, 2010. Google ScholarDigital Library
- D. Bermbach, M. Klems, S. Tai, and M. Menzel. Metastorage: A federated cloud storage system to manage consistency-latency tradeoffs. In Cloud Computing (CLOUD), 2011 IEEE International Conference on, pages 452--459. IEEE, 2011. Google ScholarDigital Library
- A. Bessani, M. Correia, B. Quaresma, F. André, and P. Sousa. DepSky: dependable and secure storage in a cloud-of-clouds. ACM Transactions on Storage Systems, 9(4):12, 2013. Google ScholarDigital Library
- A. Bessani, R. Mendes, T. Oliveira, N. Neves, M. Correia, M. Pasin, and P. Verissimo. SCFS: A shared cloud-backed file system. In Proceedings of the 2014 USENIX Conference on USENIX Annual Technical Conference, pages 169--180, 2014. ISBN 978-1-931971-10-2. Google ScholarDigital Library
- X. Cheng, C. Dale, and J. Liu. Statistics and social network of youtube videos. In Quality of Service, 2008. IWQoS 2008. 16th International Workshop on, pages 229--238, June 2008.Google ScholarCross Ref
- comScore Releases January 2014 U.S. Online Video Rankings. http://www.comscore.com/Insights/Press-Releases/2014/2/comScore-Releases-January-2014-US-Online-Video-Rankings.Google Scholar
- CumulusClips. http://cumulusclips.org/.Google Scholar
- Dropbox. https://www.dropbox.com.Google Scholar
- Dropbox Fact Sheet. https://www.dropbox.com/static/docs/DropboxFactSheet.pdf.Google Scholar
- Elgg. http://www.elgg.org/.Google Scholar
- Facebook. http://www.facebook.com.Google Scholar
- FileSender. http://www.filesender.org/.Google Scholar
- Filesystem in Userspace. http://fuse.sourceforge.net/.Google Scholar
- Flexible I/O Tester. https://github.com/axboe/fio.Google Scholar
- Flickr. http://www.flickr.com.Google Scholar
- Fully Automated Storage Tiering (FAST). http://www.emc.com/corporate/glossary/fully-automated-storage-tiering.htm.Google Scholar
- J. Guerra, H. Pucha, J. Glider, W. Belluomini, and R. Rangaswami. Cost effective storage using extent based dynamic tiering. In Proceedings of the 9th USENIX Conference on File and Stroage Technologies, FAST'11. Google ScholarDigital Library
- HPE 3PAR StoreServ 7000 Storage. http://www8.hp.com/us/en/products/disk-storage/product-detail.html?oid=5335712.Google Scholar
- Y. Hu, H. C. Chen, P. P. Lee, and Y. Tang. NCCloud: applying network coding for the storage repair in a cloud-of-clouds. In Proceedings of the 10th USENIX Conference on File and Storage Technologies, page 21, 2012. Google ScholarDigital Library
- IBM DS8000 Easy Tier. http://www.redbooks.ibm.com/abstracts/redp4667.html.Google Scholar
- K. Lerman and L. A. Jones. Social browsing on flickr. In International Conference on Weblogs and Social Media, 2007.Google Scholar
- A. Li, X. Yang, S. Kandula, and M. Zhang. CloudCmp: Comparing public cloud providers. In Proceedings of the 10th ACM SIGCOMM Conference on Internet Measurement, pages 1--14, 2010. Google ScholarDigital Library
- T. G. Papaioannou, N. Bonvin, and K. Aberer. Scalia: An adaptive scheme for efficient multi-cloud storage. In Proceedings of the International Conference on High Performance Computing, Networking, Storage and Analysis, SC '12, pages 20:1--20:10, 2012. ISBN 978-1-4673-0804-5. Google ScholarDigital Library
- Piwigo. http://piwigo.org/.Google Scholar
- Protocol Buffers. https://developers.google.com/protocol-buffers/.Google Scholar
- K. P. Puttaswamy, T. Nandagopal, and M. Kodialam. Frugal storage for cloud file systems. In Proceedings of the 2012 ACM European Conference on Computer Systems (EUROSYS '12), pages 71--84. ACM, 2012. Google ScholarDigital Library
- Redis. http://redis.io/.Google Scholar
- s3fs. https://github.com/s3fs-fuse/s3fs-fuse.Google Scholar
- s3ql. https://bitbucket.org/nikratio/s3ql.Google Scholar
- R. P. Spillane, P. J. Shetty, E. Zadok, S. Dixit, and S. Archak. An efficient multi-tier tablet server storage architecture. In Proceedings of the 2Nd ACM Symposium on Cloud Computing, SOCC '11. Google ScholarDigital Library
- Startups and Amazon Web Services. http://aws.amazon.com/start-ups/.Google Scholar
- Statistics - Youtube. https://www.youtube.com/yt/press/statistics.html.Google Scholar
- B. C. Tak, B. Urgaonkar, and A. Sivasubramaniam. To move or not to move: The economics of cloud computing. In Proceedings of the 3rd USENIX Conference on Hot Topics in Cloud Computing, 2011. Google ScholarDigital Library
- The Best Video Length for Different Videos on YouTube. http://www.minimatters.com/blog/youtube-best-video-length/.Google Scholar
- The man behind Flickr on making the service 'awesome again'. http://www.theverge.com/2013/3/20/4121574/flickr-chief-markus-spiering-talks-photos-and-marissa-mayer.Google Scholar
- M. Vrable, S. Savage, and G. M. Voelker. Cumulus: Filesystem backup to the cloud. Trans. Storage, 5(4):14:1--14:28, Dec. 2009. Google ScholarDigital Library
- M. Vrable, S. Savage, and G. M. Voelker. BlueSky: A cloud-backed file system for the enterprise. In Proceedings of the 10th USENIX Conference on File and Storage Technologies, FAST'12, 2012. Google ScholarDigital Library
- K. Walsh and E. G. Sirer. Experience with an object reputation system for peer-to-peer filesharing. In Proceedings of the 3rd Conference on Networked Systems Design & Implementation - Volume 3, NSDI'06, 2006. Google ScholarDigital Library
- H. Wang and P. Varman. Balancing fairness and efficiency in tiered storage systems with bottleneck-aware allocation. In Proceedings of the 12th USENIX Conference on File and Storage Technologies, FAST'14. Google ScholarDigital Library
- A. Wieder, P. Bhatotia, A. Post, and R. Rodrigues. Orchestrating the deployment of computations in the cloud with conductor. In Proceedings of the 9th USENIX Conference on Networked Systems Design and Implementation, 2012. Google ScholarDigital Library
- Wikimedia. Media storage. https://wikitech.wikimedia.org/wiki/Media_storage.Google Scholar
- Z. Wu, M. Butkiewicz, D. Perkins, E. Katz-Bassett, and H. V. Madhyastha. SPANStore: Cost-effective geo-replicated storage spanning multiple cloud services. In Proceedings of the 24th ACM Symposium on Operating Systems Principles (SOSP '13), pages 292--308, 2013. ISBN 978-1-4503-2388-8. Google ScholarDigital Library
- YouTube. http://www.youtube.com.Google Scholar
- G. Zhang, L. Chiu, and L. Liu. Adaptive data migration in multi-tiered storage based cloud environment. In IEEE 3rd International Conference on Cloud Computing (CLOUD), 2010. Google ScholarDigital Library
Index Terms
- Grandet: A Unified, Economical Object Store for Web Applications
Recommendations
MyCassandra: a cloud storage supporting both read heavy and write heavy workloads
SYSTOR '12: Proceedings of the 5th Annual International Systems and Storage ConferenceA cloud storage with persistence shows solid performance only with a read heavy or write heavy workload. There is a trade-off between the read-optimized and write-optimized design of a cloud storage. This is dominated by its storage engine, which is a ...
Middleware enabled data sharing on cloud storage services
MW4SOC '10: Proceedings of the 5th International Workshop on Middleware for Service Oriented ComputingWith the emergence of public cloud storage platforms like Amazon, Microsoft and Google etc, individual applications and some enterprise storage are being increasingly deployed on Clouds. However, dynamic data sharing in public clouds face problems of ...
Cloud Storage as the Infrastructure of Cloud Computing
ICICCI '10: Proceedings of the 2010 International Conference on Intelligent Computing and Cognitive InformaticsAs an emerging technology and business paradigm, Cloud Computing has taken commercial computing by storm. Cloud computing platforms provide easy access to a company’s high-performance computing and storage infrastructure through web services. With cloud ...
Comments