Distributed operation in the Borealis stream processing engine

ABSTRACT

Borealis is a distributed stream processing engine that is being developed at Brandeis University, Brown University, and MIT. Borealis inherits core stream processing functionality from Aurora and inter-node communication functionality from Medusa.We propose to demonstrate some of the key aspects of distributed operation in Borealis, using a multi-player network game as the underlying application. The demonstration will illustrate the dynamic resource management, query optimization and high availability mechanisms employed by Borealis, using visual performance-monitoring tools as well as the gaming experience.

REFERENCES

• 1: Arvind Arasu , Brian Babcock , Shivnath Babu , Mayur Datar , Keith Ito , Itaru Nishizawa , Justin Rosenstein , Jennifer Widom, STREAM: the stanford stream data manager (demonstration description), Proceedings of the 2003 ACM SIGMOD international conference on Management of data, June 09-12, 2003, San Diego, California [doi>10.1145/872757.872854]
• 2: A. Arasu et. al. Linear Road: A Stream Data Management Benchmark. In VLDB Conference, August 2004.
• 3: A. Arasu, S. Babu, and J. Widom. CQL: A Language for Continuous Queries Over Streams and Relations. In DBPL Workshop, September 2003.
• 4: Cube. http://www.cubeengine.com.
• 5: D. Abadi , D. Carney , U. Çetintemel , M. Cherniack , C. Convey , C. Erwin , E. Galvez , M. Hatoun , A. Maskey , A. Rasin , A. Singer , M. Stonebraker , N. Tatbul , Y. Xing , R. Yan , S. Zdonik, Aurora: a data stream management system, Proceedings of the 2003 ACM SIGMOD international conference on Management of data, June 09-12, 2003, San Diego, California [doi>10.1145/872757.872855]
• 6: D. Abadi et. al. The Design of the Borealis Stream Processing Engine. In CIDR Conference, January 2005.
• 7: D. Carney et. al. Operator Scheduling in a Data Stream Manager. In VLDB Conference, September 2003.
• 8: Jeong-Hyon Hwang , Magdalena Balazinska , Alexander Rasin , Ugur Cetintemel , Michael Stonebraker , Stan Zdonik, High-Availability Algorithms for Distributed Stream Processing, Proceedings of the 21st International Conference on Data Engineering (ICDE'05), p.779-790, April 05-08, 2005 [doi>10.1109/ICDE.2005.72]
• 9: N. Tatbul et. al. Load Shedding in a Data Stream Manager. In VLDB Conference, September 2003.
• 10: Sirish Chandrasekaran , Owen Cooper , Amol Deshpande , Michael J. Franklin , Joseph M. Hellerstein , Wei Hong , Sailesh Krishnamurthy , Samuel R. Madden , Fred Reiss , Mehul A. Shah, TelegraphCQ: continuous dataflow processing, Proceedings of the 2003 ACM SIGMOD international conference on Management of data, June 09-12, 2003, San Diego, California [doi>10.1145/872757.872857]
• 11: N. Tatbul and S. Zdonik. Window-aware Load Shedding for Data Streams. Technical Report CS-04-13, Brown University, Computer Science, November 2004.
• 12: Ying Xing , Stan Zdonik , Jeong-Hyon Hwang, Dynamic Load Distribution in the Borealis Stream Processor, Proceedings of the 21st International Conference on Data Engineering (ICDE'05), p.791-802, April 05-08, 2005 [doi>10.1109/ICDE.2005.53]
• 13: S. Zdonik, M. Stonebraker, M. Cherniack, U. Çetintemel, M. Balazinska, and H. Balakrishnan. The Aurora and Medusa Projects. IEEE Data Engineering Bulletin, 26(1), March 2003.