Teodora Sandra Buda
ABSTRACT
Populating the testing environment with relevant data represents a great challenge in software validation, generally requiring expert knowledge about the system under development, as its data critically impacts the outcome of the tests designed to assess the system. Current practices of populating the testing environments generally focus on developing efficient algorithms for generating synthetic data or use the production environment for testing purposes. The latter is an invaluable strategy to provide real test cases in order to discover issues that critically impact the user of the system. However, the production environment generally consists of large amounts of data that are difficult to handle and analyze. Database sampling from the production environment is a potential solution to overcome these challenges.
In this research, we propose two database sampling methods, VFDS and CoDS, with the objective of populating the testing environment. The first method is a very fast random sampling approach, while the latter aims at preserving the distribution of data in order to produce a representative sample. In particular, we focus on the dependencies between the data from different tables and the method tries to preserve the distributions of these dependencies.
- IBM DB2 Test Database Generator. http://www-306.ibm.com/software/data/ db2imstools/db2tools/db2tdbg/.Google Scholar
- S. Acharya, P. B. Gibbons, V. Poosala, and S. Ramaswamy. Join synopses for approximate query answering. In International Conference on Management of Data (SIGMOD), pages 275–286, 1999. Google ScholarDigital Library
- S. Agarwal, A. P. Iyer, A. Panda, S. Madden, B. Mozafari, and I. Stoica. Blink and it’s done: interactive queries on very large data. VLDB Endowment, 5(12):1902–1905, 2012. Google ScholarDigital Library
- N. Bruno and S. Chaudhuri. Flexible database generators. In Proceedings of the 31st international conference on Very large data bases (VLDB), pages 1097–1107, 2005. Google ScholarDigital Library
- V. T. Chakaravarthy, V. Pandit, and Y. Sabharwal. Analysis of sampling techniques for association rule mining. In 12th ACM International Conference on Database Theory (ICST), pages 276–283, 2009. Google ScholarDigital Library
- R. Gemulla, P. Rösch, and W. Lehner. Linked bernoulli synopses: Sampling along foreign keys. In 20th International Conference on Scientific and Statistical Database Management (SSDBM), pages 6–23, 2008. Google ScholarDigital Library
- Y. E. Ioannidis and V. Poosala. Histogram-based approximation of set-valued query-answers. In 25th International Conference on Very Large Data Bases (VLDB), pages 174–185, 1999. Google ScholarDigital Library
- G. John and P. Langley. Static versus dynamic sampling for data mining. In 2nd International Conference on Knowledge Discovery and Data Mining (KDD), pages 367–370, 1996.Google Scholar
- X. Lu and S. Bressan. Sampling connected induced subgraphs uniformly at random. In 24th International Conference on Scientific and Statistical Database Management (SSDBM), pages 195–212, 2012. Google ScholarDigital Library
- F. Olken. Random Sampling from Databases. PhD thesis, University of California at Berkeley, 1993.Google Scholar
- C. Olston, S. Chopra, and U. Srivastava. Generating example data for dataflow programs. In Proc. ACM International Conference on Management of data (SIGMOD), pages 245–256, 2009. Google ScholarDigital Library
- T. Rabl, M. Frank, H. M. Sergieh, and H. Kosch. A data generator for cloud-scale benchmarking. In Proceedings of the Second TPC technology conference on Performance evaluation, measurement and characterization of complex systems (TPCTC), pages 41–56, 2011. Google ScholarDigital Library
- K. Taneja, Y. Zhang, and T. Xie. Moda: Automated test generation for database applications via mock objects. In Proc. IEEE/ACM International Conference on Automated Software Engineering (ASE 2010), short paper, 2010. Google ScholarDigital Library
- X. Wu, Y. Wang, S. Guo, and Y. Zheng. Privacy preserving database generation for database application testing. Fundam. Inf., 78(4):595–612, Dec. 2007. Google ScholarDigital Library
Index Terms
- Generation of test databases using sampling methods
Recommendations
Towards realistic sampling: generating dependencies in a relational database
ICUIMC '13: Proceedings of the 7th International Conference on Ubiquitous Information Management and CommunicationManaging large amounts of information is one of the most expensive, time-consuming and non-trivial activities and it usually requires expert knowledge. In a wide range of application areas, such as data mining, histogram construction, approximate query ...
VFDS: An Application to Generate Fast Sample Databases
CIKM '14: Proceedings of the 23rd ACM International Conference on Conference on Information and Knowledge ManagementLarge amounts of data often require expensive and time-consuming analysis. Therefore, highly scalable and efficient techniques are necessary to process, analyze and discover useful information. Database sampling has proven to be a powerful method to ...
Test-Driven Development of Relational Databases
Developers can use a test-driven development with database schema just as they use it with application code. Implementing test-driven database development (TDDD) involves three relatively simple steps: database refactoring, database regression testing, ...
Comments