Rubato DB: A Highly Scalable Staged Grid Database System for OLTP and Big Data Applications

ABSTRACT

This paper proposes a new formula protocol for distributed concurrency control, and specifies a staged grid architecture for highly scalable database management systems. The paper also describes novel implementation techniques of Rubato DB based on the proposed protocol and architecture. We have conducted extensive experiments which clearly show that Rubato DB is highly scalable with efficient performance under both TPC-C and YCSB benchmarks. Our paper verifies that the formula protocol and the staged grid architecture provide a satisfactory solution to one of the important challenges in the database systems: to develop a highly scalable database management system that supports various consistency levels from ACID to BASE.

References

1. A. Abouzeid and etc. Hadoopdb: an architectural hybrid of mapreduce and dbms technologies for analytical workloads. In PVLDB, pages 922--933, 2009.
2. M. K. Aguilera, A. Merchant, and etc. Sinfonia: A new paradigm for building scalable distributed systems. volume 27, pages 5:1--5:48.
3. P. Alvaro, N. Conway, J. Hellerstein, and W. R. Marczak. Consistency analysis in bloom: a calm and collected approach. In CIDR, pages 249--260, 2011.
4. J. Baker, C. Bond, J. Corbett, and etc. Megastore: Providing scalable, highly available storage for interactive services. In CIDR, pages 223--234, 2011.
5. S. Blott and H. F. Korth. An almost-serial protocol for transaction execution in main-memory database systems. In PVLDB, pages 706--717, 2002.
6. M. Brantner, D. Florescu, and etc. Building a database on s3. In SIGMOD, pages 251--264, 2008.
7. R. Chaiken, B. Jenkins, and etc. Scope: easy and efficient parallel processing of massive data sets. Proc. VLDB Endow., 1(2):1265--1276, Aug. 2008.
8. F. Chang, J. Dean, S. Ghemawat, and etc. Bigtable: A distributed storage system for structured data. In ACM TOCS, volume 26, pages 1--26, 2008.
9. J. Cohen, B. Dolan, and etc. Mad skills: new analysis practices for big data. PVLDB, 2(2):1481--1492, 2009.
10. B. F. Cooper and etc. Pnuts: Yahoo!'s hosted data serving platform. In VLDB, pages 1277--1288, 2008.
11. B. F. Cooper and etc. Benchmarking cloud serving systems with ycsb. In SoCC, pages 143--154, 2010.
12. J. C. Corbett and etc. Spanner: Google's globally-distributed database. In OSDI, pages 251--264, 2012.
13. J. Dean and S. Ghemawat. Mapreduce: Simplified data processing on large clusters. In Commun. of ACM, volume 51, pages 107--113, 2008.
14. G. DeCandia, D. Hastorun, and etc. Dynamo: Amazon's highly available key-value store. In SOSP, pages 205--220, 2007.
15. A. Fekete, D. Liarokapis, and etc. Making snapshot isolation serializable. In TODS, volume 30, pages 492--528, 2005.
16. S. Harizopoulos and A. Ailamaki. A case for staged database systems. In CIDR, 2003.
17. M. Isard, M. Budiu, and etc. Dryad: distributed data-parallel programs from sequential building blocks. ACM SIGOPS, 41(3):59--72, 2007.
18. R. Kallman and etc. H-store: a high-performance, distributed main memory transaction processing system. In VLDB, pages 1496--1499, 2008.
19. A. Lakshman. Cassandra: a decentralized structured storage system. In SIGOPS, pages 35--40, 2010.
20. P.-A. Larson, S. Blanas, and etc. High-performance concurrency control mechanisms for main-memory databases. In PVLDB, volume 5, pages 298--309, 2011.
21. K. Manassiev and etc. Exploiting distributed version concurrency in a transactional memory cluster. In ACM SIGPLAN, pages 198--208. ACM, 2006.
22. S. Melnik and etc. Dremel: interactive analysis of web-scale datasets. PVLDB, 3(1-2):330--339, 2010.
23. J. Rao and etc. Using paxos to build a scalable, consistent, and highly available datastore. volume 4, pages 243--254. VLDB Endowment, 2011.
24. M. Stonebraker and R. Cattell. 10 rules for scalable performance in 'simple operation' datastores. Commun. ACM, 54(6):72--80, June 2011.
25. M. Stonebraker, S. Madden, and etc. The end of an architectural era: (it's time for a complete rewrite). In VLDB, pages 1150--1160, 2007.
26. R. Thomas. A solution to the concurrency control problem for multiple copy databases. In Digest of papers IEEE COMPCON spring, pages 56--62, 1984.
27. A. Thomson, T. Diamond, and etc. Calvin: fast distributed transactions for partitioned database systems. In SIGMOD, pages 1--12, 2012.
28. A. Thusoo, J. S. Sarma, and etc. Hive: a warehousing solution over a map-reduce framework. volume 2, pages 1626--1629. VLDB Endowment, Aug. 2009.
29. TPC-C. http://www.tpc.org/tpcc/. 2010.
30. VoltDB. http://voltdb.com/products/technology.
31. M. Welsh, D. Culler, and E. Brewer. Seda: an architecture for well-conditioned, scalable internet services. In SOSP, pages 230--243, 2001.

Index Terms

• Computing methodologies

  • Concurrent computing methodologies

• Information systems

  • Data management systems

    • Database management system engines

      • Database transaction processing

  • Information retrieval

    • Search engine architectures and scalability

      • Distributed retrieval

      • Peer-to-peer retrieval

  • Information storage systems

    • Storage architectures

      • Distributed storage

• Theory of computation

  • Models of computation

    • Concurrency