ABSTRACT
Combinatorial Interaction Testing has shown great potential for effectively testing Software Product Lines (SPLs). An important part of this type of testing is determining a subset of SPL products in which interaction errors are more likely to occur. Such sets of products are obtained by computing a so called t-wise Covering Array (tCA), whose computation is known to be NP-complete. Recently, the ICPL algorithm has been proposed to compute these covering arrays. In this research-in-progress paper, we propose a set of rules that exploit basic feature model knowledge to reduce the number of elements (i.e. t-sets) required by ICPL without weakening the strength of the generated arrays. We carried out a comparison of runtime performance that shows a significant reduction of the needed execution time for the majority of our SPL case studies.
- Software Product Line Online Tools(SPLOT), Accesed July 2011. http://www.splot-research.org/.Google Scholar
- N. Alon, D. Moshkovitz, and S. Safra. Algorithmic construction of sets for k-restrictions. ACM Transactions on Algorithms, 2(2):153--177, 2006. Google ScholarDigital Library
- D. Batory, J. N. Sarvela, and A. Rauschmayer. Scaling Step-Wise Refinement. IEEE TSE, 30(6), 2004. Google ScholarDigital Library
- D. Benavides, S. Segura, and A. Ruiz-Cortés. Automated analysis of feature models 20 years later: A literature review. Information Systems, In Press, Corrected Proof:--, 2010. Google ScholarDigital Library
- M. Cohen, M. Dwyer, and J. Shi. Constructing interaction test suites for highly-configurable systems in the presence of constraints: A greedy approach. Software Engineering, IEEE Transactions on, 34(5):633--650, sept.--oct. 2008. Google ScholarDigital Library
- K. Czarnecki and U. Eisenecker. Generative Programming: Methods, Tools, and Applications. Addison-Wesley, 2000. Google ScholarDigital Library
- E. S. de Almeida, C. Schwanninger, and D. Benavides, editors. 16th International Software Product Line Conference, SPLC '12, Salvador, Brazil - September 2--7, 2012, Volume 1. ACM, 2012.Google Scholar
- E. Engström and P. Runeson. Software product line testing - a systematic mapping study. Information & Software Technology, 53(1):2--13, 2011. Google ScholarDigital Library
- J. Ferrer, P. M. Kruse, J. F. Chicano, and E. Alba. Evolutionary algorithm for prioritized pairwise test data generation. In T. Soule and J. H. Moore, editors, GECCO, pages 1213--1220. ACM, 2012. Google ScholarDigital Library
- S. Fouché, M. B. Cohen, and A. A. Porter. Incremental covering array failure characterization in large configuration spaces. In G. Rothermel and L. K. Dillon, editors, ISSTA, pages 177--188. ACM, 2009. Google ScholarDigital Library
- B. J. Garvin, M. B. Cohen, and M. B. Dwyer. Evaluating improvements to a meta-heuristic search for constrained interaction testing. Empirical Software Engineering, 16(1):61--102, 2011. Google ScholarDigital Library
- M. F. Johansen, Ø. Haugen, and F. Fleurey. Properties of realistic feature models make combinatorial testing of product lines feasible. In J. Whittle, T. Clark, and T. Kühne, editors, MoDELS, volume 6981 of Lecture Notes in Computer Science, pages 638--652. Springer, 2011. Google ScholarDigital Library
- M. F. Johansen, Ø. Haugen, and F. Fleurey. An algorithm for generating t-wise covering arrays from large feature models. In de Almeida et al. {7}, pages 46--55. Google ScholarDigital Library
- K. Kang, S. Cohen, J. Hess, W. Novak, and A. Peterson. Feature-Oriented Domain Analysis (FODA) Feasibility Study. Technical Report CMU/SEI-90-TR-21, Software Engineering Institute, Carnegie Mellon University, 1990.Google Scholar
- R. M. Karp. Reducibility among combinatorial problems. In R. E. Miller and J. W. Thatcher, editors, Complexity of Computer Computations, The IBM Research Symposia Series, pages 85--103. Plenum Press, New York, 1972.Google ScholarCross Ref
- D. R. Kuhn, D. R. Wallace, and A. M. Gallo, Jr. Software fault interactions and implications for software testing. IEEE Trans. Softw. Eng., 30(6):418--421, June 2004. Google ScholarDigital Library
- J. Lee, S. Kang, and D. Lee. A survey on software product line testing. In de Almeida et al. {7}, pages 31--40. Google ScholarDigital Library
- Y. Lei, R. Kacker, D. R. Kuhn, V. Okun, and J. Lawrence. Ipog: A general strategy for t-way software testing. In ECBS, pages 549--556. IEEE Computer Society, 2007. Google ScholarDigital Library
- C. Nie and H. Leung. A survey of combinatorial testing. ACM Comput. Surv., 43(2):11:1--11:29, Feb. 2011. Google ScholarDigital Library
- S. Oster, F. Markert, and P. Ritter. Automated incremental pairwise testing of software product lines. In J. Bosch and J. Lee, editors, SPLC, volume 6287 of Lecture Notes in Computer Science, pages 196--210. Springer, 2010. Google ScholarDigital Library
- K. Pohl, G. Bockle, and F. J. van der Linden. Software Product Line Engineering: Foundations, Principles and Techniques. Springer, 2005. Google ScholarCross Ref
- A. Reuys, S. Reis, E. Kamsties, and K. Pohl. The scented method for testing software product lines. In T. Käkölä and J. C. Dueñas, editors, Software Product Lines, pages 479--520. Springer, 2006.Google ScholarCross Ref
- S. Segura. Automated analysis of feature models using atomic sets. In S. Thiel and K. Pohl, editors, SPLC (2), pages 201--207. Lero Int. Science Centre, University of Limerick, Ireland, 2008.Google Scholar
- E. Uzuncaova, S. Khurshid, and D. S. Batory. Incremental test generation for software product lines. IEEE Trans. Software Eng., 36(3):309--322, 2010. Google ScholarDigital Library
- P. Zave. Faq sheet on feature interaction. http://www.research.att.com/pamela/faq.html.Google Scholar
Index Terms
- Using feature model knowledge to speed up the generation of covering arrays
Recommendations
An algorithm for generating t-wise covering arrays from large feature models
SPLC '12: Proceedings of the 16th International Software Product Line Conference - Volume 1A scalable approach for software product line testing is required due to the size and complexity of industrial product lines. In this paper, we present a specialized algorithm (called ICPL) for generating covering arrays from feature models. ICPL makes ...
Industrial evaluation of pairwise SPL testing with MoSo-PoLiTe
VaMoS '12: Proceedings of the 6th International Workshop on Variability Modeling of Software-Intensive SystemsTesting Software Product Lines is a very challenging task due to variability. Frequently, approaches such as combinatorial testing are used to generate representative sets of products for testing purposes instead of testing each individual product of ...
Pairwise feature-interaction testing for SPLs: potentials and limitations
SPLC '11: Proceedings of the 15th International Software Product Line Conference, Volume 2A fundamental problem of testing Software Product Lines (SPLs) is that variability enables the production of a large number of instances and it is difficult to construct and run test cases even for SPLs with a small number of variable features. ...
Comments