Chang Hwan Peter Kim
Marcelo D'Amorim
ABSTRACT
Many programs can be configured through dynamic and/or static selection of configuration variables. A software product line (SPL), for example, specifies a family of programs where each program is defined by a unique combination of features. Systematically testing SPL programs is expensive as it can require running each test against a combinatorial number of configurations. Fortunately, a test is often independent of many configuration variables and need not be run against every combination. Configurations that are not required for a test can be pruned from execution. This paper presents SPLat, a new way to dynamically prune irrelevant configurations: the configurations to run for a test can be determined during test execution by monitoring accesses to configuration variables. SPLat achieves an optimal reduction in the number of configurations and is lightweight compared to prior work that used static analysis and heavyweight dynamic execution. Experimental results on 10 SPLs written in Java show that SPLat substantially reduces the total test execution time in many cases. Moreover, we demonstrate the scalability of SPLat by applying it to a large industrial code base written in Ruby on Rails.
- S. Apel and D. Beyer. Feature cohesion in software product lines: an exploratory study. In ICSE, pages 421–430, 2011. Google Scholar
Digital Library
- S. Apel, A. von Rhein, P. Wendler, A. Groblinger, and D. Beyer. Strategies for Product-Line Verification: Case Studies and Experiments. In ICSE, pages 482–491, 2013. Google ScholarDigital Library
- T. H. Austin and C. Flanagan. Multiple facets for dynamic information flow. In POPL, pages 165–178, 2012. Google ScholarDigital Library
- C. Boyapati, S. Khurshid, and D. Marinov. Korat: Automated testing based on Java predicates. In ISSTA, pages 123–133, July 2002. Google ScholarDigital Library
- I. Cabral, M. B. Cohen, and G. Rothermel. Improving the testing and testability of software product lines. In SPLC, pages 241–255, 2010. Google ScholarDigital Library
- S. Chandra, E. Torlak, S. Barman, and R. Bod´ık. Angelic debugging. In ICSE, pages 121–130, 2011. Google ScholarDigital Library
- A. Classen, P. Heymans, P.-Y. Schobbens, A. Legay, and J.-F. Raskin. Model checking lots of systems: Efficient verification of temporal properties in software product lines. In ICSE, pages 335–344, 2010. Google ScholarDigital Library
- M. B. Cohen, M. B. Dwyer, and J. Shi. Coverage and adequacy in software product line testing. In ROSATEA, pages 53–63, 2006. Google ScholarDigital Library
- M. B. Cohen, M. B. Dwyer, and J. Shi. Interaction testing of highly-configurable systems in the presence of constraints. In ISSTA, pages 129–139, 2007. Google ScholarDigital Library
- M. d’Amorim, S. Lauterburg, and D. Marinov. Delta execution for efficient state-space exploration of object-oriented programs. In ISSTA, pages 50–60, 2007. Google ScholarDigital Library
- B. Daniel, T. Gvero, and D. Marinov. On test repair using symbolic execution. In ISSTA, pages 207–218, 2010. Google ScholarDigital Library
- FEST. Fixtures for Easy Software Testing. http://fest.easytesting.org/.Google Scholar
- P. Godefroid. Model checking for programming languages using verisoft. In POPL, pages 174–186, 1997. Google ScholarDigital Library
- R. J. Hall. Fundamental nonmodularity in electronic mail. ASE Journal, 12(1):41–79, 2005. Google ScholarDigital Library
- P. Hosek and C. Cadar. Safe Software Updates via Multi-version Execution. In ICSE, 2013. Google ScholarDigital Library
- Human-resource management system. 101Companies. http://101companies.org/wiki/@system.Google Scholar
- Java tokenizer and parser tools. JTopas. http:// jtopas.sourceforge.net/jtopas/index.html.Google Scholar
- K. Kang, S. Cohen, J. Hess, W. Nowak, and S. Peterson. Feature-oriented domain analysis (FODA) feasibility study. Technical Report 90-TR-21, CMU/SE, Nov. 1990.Google ScholarCross Ref
- C. Kästner, A. von Rhein, S. Erdweg, J. Pusch, S. Apel, T. Rendel, and K. Ostermann. Toward variability-aware testing. In FOSD, pages 1–8, 2012. Google ScholarDigital Library
- C. H. P. Kim, D. Batory, and S. Khurshid. Reducing Combinatorics in Testing Product Lines. In AOSD, pages 57–68, 2011. Google ScholarDigital Library
- C. H. P. Kim, E. Bodden, D. S. Batory, and S. Khurshid. Reducing Configurations to Monitor in a Software Product Line. In RV, pages 285–299, 2010. Google ScholarDigital Library
- C. H. P. Kim, S. Khurshid, and D. Batory. Shared Execution for Efficiently Testing Product Lines. In ISSRE, pages 221–230, 2012.Google ScholarDigital Library
- C. H. P. Kim, D. Marinov, S. Khurshid, D. Batory, S. Souto, P. Barros, and M. d’Amorim. SPLat: Evaluation. http://www.cs.utexas.edu/~chpkim/splat, 2013.Google Scholar
- C. Kolbitsch, B. Livshits, B. Zorn, and C. Seifert. Rozzle: De-cloaking internet malware. In Oakland, 2012.Google Scholar
- Korat home page. http://mir.cs.illinois.edu/korat/.Google Scholar
- J. Kramer. Conic: an integrated approach to distributed computer control systems. Computers and Digital Techniques, IEE Proceedings E, 130(1), 1983.Google Scholar
- Library for object persistence. Prevayler. www.prevayler.org/.Google Scholar
- Library to serialize objects to XML and back again. XStream. http://xstream.codehaus.org/.Google Scholar
- R. E. Lopez-herrejon and D. Batory. A standard problem for evaluating product-line methodologies. In GPCE, pages 10–24. Springer, 2001.Google Scholar
- J. McGregor. Testing a Software Product Line. Technical Report CMU/SEI-2001-TR-022, CMU/SEI, Mar. 2001.Google ScholarCross Ref
- M. Plath and M. Ryan. Feature integration using a feature construct. SCP Journal, 41(1):53–84, 2001. Google ScholarDigital Library
- Puzzle game. Sudoku. https://code.launchpad.net/ ~spl-devel/spl/default-branch.Google Scholar
- A. V. Rhein, S. Apel, and F. Raimondi. Introducing Binary Decision Diagrams in the Explicit-State Verification of Java Code. In JPF Workshop, 2011.Google Scholar
- SAT4J. http://www.sat4j.org/.Google Scholar
- D. Schuler, V. Dallmeier, and A. Zeller. Efficient mutation testing by checking invariant violations. In ISSTA, pages 69–80, 2009. Google ScholarDigital Library
- S. Thaker, D. S. Batory, D. Kitchin, and W. R. Cook. Safe composition of product lines. In C. Consel and J. L. Lawall, editors, GPCE, pages 95–104. ACM, 2007. Google ScholarDigital Library
- N. Tillmann and W. Schulte. Unit Tests Reloaded: Parameterized Unit Testing with Symbolic Execution. Technical Report MSR-TR-2005-153, Microsoft Research, November 2005.Google Scholar
- J. Tucek, W. Xiong, and Y. Zhou. Efficient online validation with delta execution. In ASPLOS, pages 193–204, 2009. Google ScholarDigital Library
- E. Weyuker and T. Ostrand. Theories of program testing and the application of revealing subdomains. IEEE TSE, 6(3):236–246, 1980. Google ScholarDigital Library
Index Terms
- SPLat: lightweight dynamic analysis for reducing combinatorics in testing configurable systems
Recommendations
Safely evolving configurable systems
SPLASH Companion 2015: Companion Proceedings of the 2015 ACM SIGPLAN International Conference on Systems, Programming, Languages and Applications: Software for HumanityDevelopers use configuration options to tailor systems to different platforms. This configurability leads to exponential configuration spaces and traditional tools (e.g., gcc) check only one configuration at a time. As a result, developers introduce ...
ConfigCrusher: towards white-box performance analysis for configurable systems
AbstractStakeholders of configurable systems are often interested in knowing how configuration options influence the performance of a system to facilitate, for example, the debugging and optimization processes of these systems. Several black-box ...
iGen: dynamic interaction inference for configurable software
FSE 2016: Proceedings of the 2016 24th ACM SIGSOFT International Symposium on Foundations of Software EngineeringTo develop, analyze, and evolve today's highly configurable software systems, developers need deep knowledge of a system's configuration options, e.g., how options need to be set to reach certain locations, what configurations to use for testing, etc. ...
Comments