Blurring the Lines between Blockchains and Database Systems: the Case of Hyperledger Fabric

ABSTRACT

Within the last few years, a countless number of blockchain systems have emerged on the market, each one claiming to revolutionize the way of distributed transaction processing in one way or the other. Many blockchain features, such as byzantine fault tolerance, are indeed valuable additions in modern environments. However, despite all the hype around the technology, many of the challenges that blockchain systems have to face are fundamental transaction management problems. These are largely shared with traditional database systems, which have been around for decades already. These similarities become especially visible for systems, that blur the lines between blockchain systems and classical database systems. A great example of this is Hyperledger Fabric, an open-source permissioned blockchain system under development by IBM. By implementing parallel transaction processing, Fabric's workflow is highly motivated by optimistic concurrency control mechanisms in classical database systems. This raises two questions: (1)~Which conceptual similarities and differences do actually exist between a system such as Fabric and a classical distributed database system? (2)~Is it possible to improve on the performance of Fabric by transitioning technology from the database world to blockchains and thus blurring the lines between these two types of systems even further? To tackle these questions, we first explore Fabric from the perspective of database research, where we observe weaknesses in the transaction pipeline. We then solve these issues by transitioning well-understood database concepts to Fabric, namely transaction reordering as well as early transaction abort. Our experimental evaluation under the Smallbank benchmark as well as under a custom workload shows that our improved version Fabric++ significantly increases the throughput of successful transactions over the vanilla version by up to a factor of 12x, while decreasing the average latency to almost half.

References

1. 2019. https://bitcoin.org/bitcoin.pdf
2. 2019. https://github.com/ethereum/wiki/wiki/White-Paper
3. 2019. http://www.tpc.org/tpcc/
4. 2019. http://www.tpc.org/tpch/
5. 2019. https://github.com/brianfrankcooper/YCSB
6. 2019. https://github.com/hyperledger/caliper
7. 2019. https://github.com/persistentsystems/gauge
8. 2019. https://github.com/persistentsystems/gauge/blob/master/docs/ caliper-changes.md
9. 2019. http://hstore.cs.brown.edu/documentation/deployment/ benchmarks/smallbank/
10. Elli Androulaki, Artem Barger, Vita Bortnikov, et al. 2018. Hyperledger fabric: a distributed operating system for permissioned blockchains. In EuroSys 2018, Porto, Portugal, April 23--26. 30:1--30:15.
11. Bailu Ding, Lucja Kot, and Johannes Gehrke. 2018. Improving Optimistic Concurrency Control Through Transaction Batching and Operation Reordering. PVLDB 12, 2 (2018), 169--182. http://www.vldb.org/ pvldb/vol12/p169-ding.pdf
12. Tien Tuan Anh Dinh, Rui Liu, Meihui Zhang, Gang Chen, Beng Chin Ooi, and Ji Wang. 2018. Untangling Blockchain: A Data Processing View of Blockchain Systems. IEEE Trans. Knowl. Data Eng. 30, 7 (2018), 1366--1385.
13. Jose M. Faleiro, Daniel Abadi, and Joseph M. Hellerstein. 2017. High Performance Transactions via Early Write Visibility. PVLDB 10, 5 (2017), 613--624.
14. Zhengyu He and Bo Hong. 2009. Impact of early abort mechanisms on lock-based software transactional memory. In 16th International Conference on High Performance Computing, HiPC 2009, December 16- 19, 2009, Kochi, India, Proceedings. 225--234.
15. Donald B. Johnson. 1975. Finding All the Elementary Circuits of a Directed Graph. SIAM J. Comput. 4, 1 (1975), 77--84.
16. Per-Åke Larson, Spyros Blanas, Cristian Diaconu, Craig Freedman, Jignesh M. Patel, and Mike Zwilling. 2011. High-Performance Concurrency Control Mechanisms for Main-Memory Databases. PVLDB 5, 4 (2011), 298--309.
17. Hyeontaek Lim, Michael Kaminsky, and David G. Andersen. 2017. Cicada: Dependably Fast Multi-Core In-Memory Transactions. In Proceedings of the 2017 ACM International Conference on Management of Data, SIGMOD Conference 2017, Chicago, IL, USA, May 14--19, 2017. 21--35.
18. Gang Luo, Jeffrey F. Naughton, Curt J. Ellmann, and Michael Watzke. 2010. Transaction reordering. Data Knowl. Eng. 69, 1 (2010), 29--49.
19. Thomas Neumann, Tobias Mühlbauer, and Alfons Kemper. 2015. Fast Serializable Multi-Version Concurrency Control for Main-Memory Database Systems. In SIGMOD 2015, Melbourne, Victoria, Australia, May 31 - June 4. 677--689.
20. Thamir M. Qadah and Mohammad Sadoghi. 2018. QueCC: A Queueoriented, Control-free Concurrency Architecture. In Proceedings of the 19th International Middleware Conference, Middleware 2018, Rennes, France, December 10--14, 2018. 13--25.
21. Ankur Sharma, Felix Martin Schuhknecht, and Jens Dittrich. 2018. Accelerating Analytical Processing in MVCC using Fine-Granular High- Frequency Virtual Snapshotting. In SIGMOD 2018, Houston, TX, USA, June 10--15, 2018. 245--258.
22. Robert Endre Tarjan. 1972. Depth-First Search and Linear Graph Algorithms. SIAM J. Comput. 1, 2 (1972), 146--160.
23. Tianzheng Wang, Ryan Johnson, Alan Fekete, and Ippokratis Pandis. 2017. Effciently making (almost) any concurrency control mechanism serializable. The VLDB Journal 26, 4 (01 Aug 2017), 537--562.
24. Tianzheng Wang and Hideaki Kimura. 2016. Mostly-Optimistic Concurrency Control for Highly Contended Dynamic Workloads on a Thousand Cores. PVLDB 10, 2 (2016), 49--60. http://www.vldb.org/pvldb/vol10/p49-wang.pdf
25. Yingjun Wu, Joy Arulraj, Jiexi Lin, Ran Xian, and Andrew Pavlo. 2017. An Empirical Evaluation of In-Memory Multi-Version Concurrency Control. PVLDB 10, 7 (2017), 781--792.
26. Chang Yao, Divyakant Agrawal, Gang Chen, Qian Lin, Beng Chin Ooi, Weng-Fai Wong, and Meihui Zhang. 2016. Exploiting Single-Threaded Model in Multi-Core In-Memory Systems. IEEE Trans. Knowl. Data Eng. 28, 10 (2016), 2635--2650.
27. Xiangyao Yu, George Bezerra, Andrew Pavlo, Srinivas Devadas, and Michael Stonebraker. 2014. Staring into the Abyss: An Evaluation of Concurrency Control with One Thousand Cores. PVLDB 8, 3 (2014), 209--220.
28. Yuan Yuan, Kaibo Wang, Rubao Lee, Xiaoning Ding, Jing Xing, Spyros Blanas, and Xiaodong Zhang. 2016. BCC: Reducing False Aborts in Optimistic Concurrency Control with Low Cost for In-Memory Databases. PVLDB 9, 6 (2016), 504--515.
29. Bo Zhang, Binoy Ravindran, and Roberto Palmieri. 2015. Reducing Aborts in Distributed Transactional Systems through Dependency Detection. In Proceedings of the 2015 International Conference on Distributed Computing and Networking, ICDCN 2015, Goa, India, January 4--7, 2015. 13:1--13:10.
30. Ningnan Zhou, Xuan Zhou, Xiao Zhang, et al. 2017. Reordering Transaction Execution to Boost High-Frequency Trading Applications. Data Science and Engineering 2, 4 (2017), 301--315.

Index Terms

• Information systems

  • Data management systems

    • Database management system engines

      • Distributed database transactions