Finding min-repros in database software

ABSTRACT

Testing and debugging database system applications is often challenging and time consuming. A database tester (or DB tester for short) has to detect a problem, determine why it happened, set up an environment to reproduce it, and then create a fix to resolve the problem. In many cases, problems appear in complex scenarios, and thus the setup to reproduce a problem may be large and difficult to understand. This makes the task of finding the root cause of a problem very challenging. As a consequence, a very arduous task for a DB tester is finding a min-repro -- a process of weeding out irrelevant inputs and finding the simplest way to reproduce a problem. Currently, a great deal of searching for minrepros is carried out manually using non-database-specific tools, which is both slow and error-prone. In this paper, we present a system designed to ease and speed up searches for minrepros in database-related products. The min-repro system employs several effective tools, including: the novel simplification transformations, the high-level language for creating custom scripts and automation, the record-and-replay functionality, and the visualization of results and the search space. In addition to the standard application mode, the min-repro system can be interacted with in the game mode. The latter can provide an intrinsically motivating environment for developing successful search strategies, which can be data-mined and generalized into search patterns and used as recommendations for DB testers.

References

1. B. Beizer. Software Testing Techniques. John Wiley & Sons, Inc., New York, NY, USA, 1990.
2. http://www.w3.org/TR/xexpr/.
3. T. J. Ostrand and M. J. Balcer. The category-partition method for specifying and generating fuctional tests. Communications of the ACM, 31 (6):676--686, 1988.
4. S. Reid. The art of software testing, second edition. Software Testing, Verification and Reliability, 15(2):136--137, 2005.
5. L. Von Ahn and L. Dabbish. Designing games with a purpose. Communications of the ACM, 51(8):58--67, 2008.
6. A. Zeller and R. Hildebrandt. Simplifying and isolating failure-inducing input. IEEE Transactions on Software Engineering, 28(2):183--200, 2002.

Index Terms

• Information systems

  • Data management systems

    • Database administration

• Software and its engineering

  • Software creation and management

    • Software verification and validation

      • Software defect analysis

        • Software testing and debugging