skip to main content
10.1145/1385269.1385272acmconferencesArticle/Chapter ViewAbstractPublication PagesmodConference Proceedingsconference-collections
research-article

Testing challenges for extending SQL server's query processor: a case study

Published: 13 June 2008 Publication History

Abstract

With enterprise-class database systems like Microsoft SQL Server, changing mission-critical components such as the relational database engine pose significant challenges to the engineering teams involved. This is because customers carefully configure, tune, and test applications and the underlying database system for their specific scenario to ensure they achieve desirable performance. This situation poses particular challenges for developing and extending existing database components of high complexity such as relational query processing where a change to a policy or heuristics may adversely affect performance of a customer workload [1]. Mitigating risk of regressions through a variety of testing and validation techniques during development is essential for the database system engineering team. In this paper, we present a case study to demonstrate the challenges of testing an extension to the query processor in Microsoft SQL Server 2008 targeted at improved query performance for star join queries [2]. We discuss the unique range of problem spaces which must be addressed, and present an overview of the variety of techniques that are used to provide a high level of quality. Our main contribution is an iterative process of interleaved development and performance test phases using customer workloads to quantify performance improvement and to limit regressions. We present our experiences with this approach and explore some future opportunities with the hope that they will encourage discussion and additional research in the area of query processor architecture and testing.

References

[1]
Giakoumakis, L and Galindo-Legaria, C. Testing SQL Server's Query Optimizer: Challenges, Techniques and Experiences. IEEE Bulletin of the Technical Committee on Data Engineering, 31, 1 (March 2008), 37--44
[2]
Galindo-Legaria, C., Grabs, T., et. al. Optimizing Star Join Queries for Data Warehousing in Microsoft SQL Server. In Proceedings of the 24th International Conference on Data Engineering, (Cancun, Mexico), 2008
[3]
Dalal, S. R., Jain, A., Karunanithi, N., Leaton, J. M., Lott, C. M., Patton, G. C., Horowitz, B. M. Model-Based Testing in Practice. In Proceedings of 21st Annual Conference on Software Engineering, (Los Angeles, California), 1999
[4]
Stonebraker, M., et. al. C-Store: A column-oriented DBMS. In Proceedings of 31st Annual Conference on Very Large Databases, (Trondheim, Norway), 2005, 553--564
[5]
Kimball, R., Ross, M. The Data Warehouse Toolkit: The Complete Guide to Dimensional Modeling (Second Edition), April 26, 2002.
[6]
SAP: Multi-Dimensional Modeling with BI: A background to the techniques used to create BI InfoCubes (Version 1.0), May 16, 2006, https://www.sdn.sap.com/irj/servlet/prt/portal/prtroot/docs/library/uuid/6ce7b0a4-0b01-0010-52ac-a6e813c35a84
[7]
Campbell, C., Grieskamap, W., Nachmanson, L., Schlute, W., Tillmann, N., Veanes, M. Model-Based Testing of Object-Oriented Reactive Systems with Spec Explorer, 2005 http://research.microsoft.com/research/pubs/view.aspx?type=Technical%20Report&id=912
[8]
Nachmanson, L., Veanes, M., Schlute, W., Tillmann, N., Grieskamp, W. Optimal Strategies for Testing Nondeterministic Systems. In ISSTA '04, volume 29 of Software Engineering Notes, 55--64. ACM July 2004.
[9]
Microsoft SQL Server. http://www.microsoft.com/sql
[10]
Transaction Processing Performance Council. http://www.tpc.org

Cited By

View all
  • (2020)Detecting optimization bugs in database engines via non-optimizing reference engine constructionProceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3368089.3409710(1140-1152)Online publication date: 8-Nov-2020
  • (2016)Performance Monitoring in SAP HANA's Continuous Integration ProcessACM SIGMETRICS Performance Evaluation Review10.1145/2897356.289736243:4(43-52)Online publication date: 25-Feb-2016
  • (2012)Testing cardinality estimation models in SQL serverProceedings of the Fifth International Workshop on Testing Database Systems10.1145/2304510.2304526(1-7)Online publication date: 21-May-2012

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
DBTest '08: Proceedings of the 1st international workshop on Testing database systems
June 2008
74 pages
ISBN:9781605582337
DOI:10.1145/1385269
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 June 2008

Permissions

Request permissions for this article.

Check for updates

Qualifiers

  • Research-article

Conference

SIGMOD/PODS '08
Sponsor:

Acceptance Rates

Overall Acceptance Rate 31 of 56 submissions, 55%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)2
  • Downloads (Last 6 weeks)0
Reflects downloads up to 17 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2020)Detecting optimization bugs in database engines via non-optimizing reference engine constructionProceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering10.1145/3368089.3409710(1140-1152)Online publication date: 8-Nov-2020
  • (2016)Performance Monitoring in SAP HANA's Continuous Integration ProcessACM SIGMETRICS Performance Evaluation Review10.1145/2897356.289736243:4(43-52)Online publication date: 25-Feb-2016
  • (2012)Testing cardinality estimation models in SQL serverProceedings of the Fifth International Workshop on Testing Database Systems10.1145/2304510.2304526(1-7)Online publication date: 21-May-2012

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media