On Improving User Response Times in Tableau

ABSTRACT

The rapid increase in data volumes and complexity of applied analytical tasks poses a big challenge for visualization solutions. It is important to keep the experience highly interactive, so that users stay engaged and can perform insightful data exploration. Query processing usually dominates the cost of visualization generation. Therefore, in order to achieve acceptable response times, one needs to utilize backend capabilities to the fullest and apply techniques, such as caching or prefetching. In this paper we discuss key data processing components in Tableau: the query processor, query caches, Tableau Data Engine [1, 2] and Data Server. Furthermore, we cover recent performance improvements related to the number and quality of remote queries, broader reuse of cached data, and application of inter and intra query parallelism.

References

1. Richard Wesley, Matthew Eldridge, and Pawel T. Terlecki. 2011. An analytic data engine for visualization in tableau. In Proceedings of the 2011 ACM SIGMOD International Conference on Management of data (SIGMOD '11). ACM, New York, NY, USA, 1185--1194. DOI= http://doi.acm.org/10.1145/1989323.1989449
2. Richard Michael Grantham Wesley and Pawel Terlecki. 2014. Leveraging compression in the tableau data engine. In Proceedings of the 2014 ACM SIGMOD international conference on Management of data (SIGMOD '14). ACM, New York, NY, USA, 563--573. DOI=http://doi.acm.org/10.1145/2588555.2595639
3. Boncz, P., Zukowski, M., and Nes, N. MonetDB/X100: Hyper-Pipelining Query Execution. In International Conference on Innovative Data Systems Research (CIDR), Jan. 2005, 225--237.
4. G. Graefe, "Volcano: An extensible and parallel query evaluation system," IEEE Transactions on Knowledge and Data Engineering, 120--135, 1994.
5. J. Zhou, P. Larson, and R. Chaiken. Incorporating partitioning and parallel plans into the SCOPE optimizer. In ICDE, 2010.
6. LibXL http://www.libxl.com/
7. Abadi, D. J., Madden, S. R., and Hachem, N. 2008. Column-stores vs. row-stores: how different are they really? In Proceedings of the 2008 ACM SIGMOD international Conference on Management of Data (Vancouver, Canada, June 09 - 12, 2008). SIGMOD '08. ACM, New York, NY, 967--980.
8. Boncz, P. Monet: A Next-Generation DBMS Kernel For Query-Intensive Applications. Doctoral Thesis, Universiteit van Amsterdam, Amsterdam, The Netherlands, May 2002.
9. Zukowski, Marcin, and Peter A. Boncz. "Vectorwise: Beyond column stores."IEEE Data Engineering Bulletin 35.1 (2012): 21--27.
10. Shivnath Babu and Herodotos Herodotou (2013), "Massively Parallel Databases and MapReduce Systems", Foundations and Trends® in Databases: Vol. 5: No. 1, pp 1--104. http://dx.doi.org/10.1561/1900000036
11. Franz Färber, Norman May, Wolfgang Lehner, Philipp Große, Ingo Müller, Hannes Rauhe, and Jonathan Dees. The SAP HANA Database -- An Architecture Overview. IEEE Data Engineering Bulletin, 35(1):28'33, 2012.
12. Anikiej K. Multi-core Parallelization of Vectorized Queries {dissertation}. University of Warsaw and VU University of Amsterdam, 2010.
13. P. G. Selinger, M. M. Astrahan, D. D. Chamberlin, R. A. Lorie, and T. G. Price. Access path selection in a relational database management system. In Proceedings of SIGMOD Conference, 1979.
14. M. Majster-Cederbaum. Elimination of redundant operations in relational queries with general selection operators. Computing, 34(4):303--323, 1984.
15. A. V. Aho, C. Beeri, and J. D. Ullman. The theory of joins in relational databases. ACM Trans. on Database Systems, 4(3): 297--314, 1979.
16. A. V. Aho, Y. Sagiv, and J. D. Ullman. Efficient optimization of a class of relational expression. ACM Trans. on Database Systems, 4(4):435--454, 1979.
17. Nikolaus Ott, Klaus Horländer, Removing redundant join operations in queries involving views, Information Systems, Volume 10, Issue 3, 1985, Pages 279--288
18. Y. Sagiv and M. Yannakakis. Equivalences among relational expressions with the union and difference operator. Journal of the ACM, 27(4):633--655, 1980.
19. Halevy, Alon Y. "Answering queries using views: A survey." The VLDB Journal 10.4 (2001): 270--294.
20. Sara Cohen, Werner Nutt, and Yehoshua Sagiv. 2003. Containment of Aggregate Queries. In Proceedings of the 9th International Conference on Database Theory (ICDT '03), Diego Calvanese, Maurizio Lenzerini, and Rajeev Motwani (Eds.). Springer-Verlag, London, UK, UK, 111--125.
21. Chandra A.K., Merlin P.M. Optimal implementation of conjunctive queries in relational databases. In: Proc. Ninth AnnualACMSymposium on Theory of Computing.pp 77'90, 1977
22. Zhang X., Ozsoyoglu M.Z. On efficient reasoning with implication constraints. In: Proc. of DOOD. pp 236'252, 1993
23. Chaudhuri S., Vardi M. Optimizing real conjunctive queries. In: Proc. of PODS. pp 59'70, Washington D.C., USA, 1993
24. Chaudhuri S., Vardi M. On the complexity of equivalence between recursive and nonrecursive datalog programs. In: Proc. of PODS. pp 55'66, Minneapolis, Minn., USA, 1994
25. Kolaitis P., Martin D., Thakur M. On the complexity of the containment problem for conjunctive queries with built-in predicates. In: Proc. of PODS. pp 197'204, Seattle,Wash., USA, 1998
26. Tsatalos O.G., Solomon M.H., Ioannidis Y.E. The GMAP: a versatile tool for physical data independence. In: Proc. of VLDB. pp 367'378, Santiago, Chile, 1994
27. Tsatalos O.G., Solomon M.H., Ioannidis Y.E. The GMAP: a versatile tool for physical data independence. VLDB J. (2):101'118, 1996
28. Chaudhuri, S., Krishnamurthy, R., Potamianos, S., & Shim, K. (1995, March). Optimizing queries with materialized views. In 2013 IEEE 29th International Conference on Data Engineering (ICDE) (pp. 190--190). IEEE Computer Society.
29. Goldstein J., Larson P.A. Optimizing queries using materialized views: a practical, scalable solution. In: Proc. of SIGMOD. pp 331'342, 2001
30. JarkeM. Common subexpression isolation in multiple query optimization. Query Processing in Database Systems, KimW, Reiner DS, Batory DS (eds.). Springer: Berlin, 1985
31. Park J, Segev A. Using common subexpressions to optimize multiple queries. Proceedings of the 4th International Conference on Data Engineering. IEEE Computer Society: Washington, DC, 1988; 311'319.
32. Sellis T. Multiple query optimization. ACM Transactions on Database Systems 1988; 13(1):23'52.
33. Cosar A, Lim E, Srivastava J. Multiple query optimization with depth-first branch-and-bound and dynamic query ordering. CIKM 93, Proceedings of the Second International Conference on Information and Knowledge Management. ACM, 1993; 433'438.
34. Chen F, Dunham M. Common subexpression processing in multiple-query processing. IEEE Transactions on Knowledge and Data Engineering 1988; 10(3):493'499.
35. Roy P et al. Efficient and extensible algorithms for multi query optimization. Proceedings of the 2000 ACM SIGMOD International Conference on Management of Data. ACM Press: New York, 2000; 249'260.
36. Tan K, Lu H. Workload scheduling for multiple query processing. Information Processing Letters 1995; 55(5):251'257.
37. Tan K, Lu H. Scheduling multiple queries in symmetric multiprocessors. Information Sciences 1996; 95(1/2):125'153.
38. Dalvi N et al. Pipelining in multi-query optimization. J. Comput. Syst. Sci. 2003; 66(4):728'762.
39. O'Gorman, Kevin, Amr El Abbadi, and Divyakant Agrawal. "Multiple query optimization in middleware using query teamwork." Software: Practice and Experience 35.4 (2005): 361--391.
40. Stolte, C., Tang, D., and Hanrahan, P. 2008. Polaris: a system for query, analysis, and visualization of multidimensional databases. Commun. ACM 51, 11 (Nov. 2008), 75--84.
41. http://redis.io/
42. Lakshman, Avinash, and Prashant Malik. "Cassandra: a decentralized structured storage system." ACM SIGOPS Operating Systems Review 44.2 (2010): 35--40.
43. https://www.faa.gov/data_research/
44. Milena G. Ivanova, Martin L. Kersten, Niels J. Nes, and Romulo A.P. Gonçalves. 2009. An architecture for recycling intermediates in a column-store. In Proceedings of the 2009 ACM SIGMOD International Conference on Management of data (SIGMOD '09), Carsten Binnig and Benoit Dageville (Eds.). ACM, New York, NY, USA, 309--320
45. Parag Agrawal , Daniel Kifer , Christopher Olston, Scheduling shared scans of large data files, Proceedings of the VLDB Endowment, v.1 n.1, August 2008
46. Prasanth Jayachandran, Karthik Tunga, Niranjan Kamat, Arnab Nandi. Combining User Interaction, Speculative Query Execution and Sampling in the DICE System. PVLDB 7(13): 1697--1700 (2014)
47. Kristi Morton, Ross Bunker, Jock Mackinlay, Robert Morton, and Chris Stolte. 2012. Dynamic workload driven data integration in tableau. In Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data (SIGMOD '12). ACM, New York, NY, USA, 807--816. http://doi.acm.org/10.1145/2213836.2213961
48. Shaul Dar , Michael J. Franklin , Björn Þór Jónsson , Divesh Srivastava , Michael Tan. Semantic Data Caching and Replacement, Proceedings of the 22th International Conference on Very Large Data Bases, p.330--341, September 03-06, 1996

Index Terms

• Information systems