AGRippin: a novel search based testing technique for Android applications

ABSTRACT

Recent studies have shown a remarkable need for testing automation techniques in the context of mobile applications. The main contributions in literature in the field of testing automation regard techniques such as Capture/Replay, Model Based, Model Learning and Random techniques. Unfortunately, only the last two typologies of techniques are applicable if no previous knowledge about the application under testing is available. Random techniques are able to generate effective test suites (in terms of source code coverage) but they need a remarkable effort in terms of machine time and the tests they generate are quite inefficient due to their redundancy. Model Learning techniques generate more efficient test suites but often they do not not reach good levels of coverage. In order to generate test suites that are both effective and efficient, we propose in this paper AGRippin, a novel Search Based Testing technique founded on the combination of genetic and hill climbing techniques. We carried out a case study involving five open source Android applications that has demonstrated how the proposed technique is able to generate test suites that are more effective and efficient than the ones generated by a Model Learning technique.

References

1. W. Afzal, R. Torkar, and R. Feldt. A systematic review of search-based testing for non-functional system properties. Inf. Softw. Technol., 51(6):957–976, June 2009.
2. S. Ali, L. Briand, H. Hemmati, and R. Panesar-Walawege. A systematic review of the application and empirical investigation of search-based test case generation. Software Engineering, IEEE Transactions on, 36(6):742–762, Nov 2010.
3. D. Amalfitano, N. Amatucci, A. R. Fasolino, P. Tramontana, E. Kowalczyk, and A. Memon. Exploiting the saturation effect in automatic random testing of android applications. In The Proceedings of the 2nd ACM International Conference on Mobile Software Engineering and Systems (MOBILESoft 2015), 2015.
4. D. Amalfitano, A. Fasolino, P. Tramontana, B. Ta, and A. Memon. Mobiguitar – a tool for automated model-based testing of mobile apps. Software, IEEE, PP(99):1–1, 2014.
5. D. Amalfitano, A. R. Fasolino, P. Tramontana, S. De Carmine, and A. M. Memon. Using gui ripping for automated testing of android applications. In Proc. of the 27th IEEE/ACM International Conference on Automated Software Engineering, ASE 2012, pages 258–261, New York, NY, USA, 2012. ACM.
6. T. Azim and I. Neamtiu. Targeted and depth-first exploration for systematic testing of android apps. SIGPLAN Not., 48(10):641–660, Oct. 2013.
7. J. Baker. Adaptive selection methods for genetic algorithms. volume Proceedings of the First International Conference on Genetic Algorithms and Their Applications. Erlbaum, 1985.
8. I. Banerjee, B. Nguyen, V. Garousi, and A. Memon. Graphical user interface (gui) testing: Systematic mapping and repository. Information and Software Technology, 2013.
9. J. Clarke, J. Dolado, M. Harman, R. Hierons, B. Jones, M. Lumkin, B. Mitchell, S. Mancoridis, K. Rees, M. Roper, and M. Shepperd. Reformulating software engineering as a search problem. Software, IEE Proceedings -, 150(3):161–175, June 2003.
10. S. Hao, B. Liu, S. Nath, W. G. Halfond, and R. Govindan. Puma: Programmable ui-automation for large-scale dynamic analysis of mobile apps. In Proceedings of the 12th Annual International Conference on Mobile Systems, Applications, and Services, MobiSys ’14, pages 204–217, New York, NY, USA, 2014. ACM.
11. M. Harman and B. F. Jones. Search-based software engineering. Information and Software Technology, 43(14):833 – 839, 2001.
12. C. S. Jensen, M. R. Prasad, and A. Møller. Automated testing with targeted event sequence generation. In Proceedings of the 2013 International Symposium on Software Testing and Analysis, ISSTA 2013, pages 67–77, New York, NY, USA, 2013. ACM.
13. M. Joorabchi, A. Mesbah, and P. Kruchten. Real challenges in mobile app development. In Empirical Software Engineering and Measurement, 2013 ACM / IEEE International Symposium on, pages 15–24, Oct 2013.
14. P. S. Kochhar, F. Thung, N. Nagappan, T. Zimmermann, and D. Lo. Understanding the test automation culture of app developers. In Software Testing, Verification and Validation (ICST), 2015 IEEE 8th International Conference on, pages 1–10, April 2015.
15. C.-H. Liu, C.-Y. Lu, S.-J. Cheng, K.-Y. Chang, Y.-C. Hsiao, and W.-M. Chu. Capture-replay testing for android applications. In Computer, Consumer and Control (IS3C), 2014 International Symposium on, pages 1129–1132, June 2014.
16. A. Machiry, R. Tahiliani, and M. Naik. Dynodroid: An input generation system for android apps. In Proceedings of the 2013 9th Joint Meeting on Foundations of Software Engineering, ESEC/FSE 2013, pages 224–234, New York, NY, USA, 2013. ACM.
17. R. Mahmood, N. Mirzaei, and S. Malek. Evodroid: Segmented evolutionary testing of android apps. In Proceedings of the 22Nd ACM SIGSOFT International Symposium on Foundations of Software Engineering, FSE 2014, pages 599–609, New York, NY, USA, 2014. ACM.
18. H. Mühlenbein. How genetic algorithms really work: Mutation and hillclimbing. In R. Männer and B. Manderick, editors, PPSN, pages 15–26. Elsevier, 1992.
19. M. Mitchell, S. Forrest, and J. H. Holland. The royal road for genetic algorithms: Fitness landscapes and ga performance. In Proceedings of the First European Conference on Artificial Life, pages 245–254. MIT Press, 1991.
20. H. Muccini, A. Di Francesco, and P. Esposito. Software testing of mobile applications: Challenges and future research directions. In Automation of Software Test (AST), 2012 7th International Workshop on, pages 29–35, June 2012.
21. M. Srinivas and L. Patnaik. Adaptive probabilities of crossover and mutation in genetic algorithms. Systems, Man and Cybernetics, IEEE Transactions on, 24(4):656–667, Apr 1994.
22. G. Syswerda. A study of reproduction in generational and steady-state genetic algorithms. In G. J. Rawlins, editor, Foundations of genetic algorithms, pages 94–101. Morgan Kaufmann, San Mateo, CA, 1991.
23. T. Takala, M. Katara, and J. Harty. Experiences of system-level model-based gui testing of an android application. In Proceedings of the 2011 Fourth IEEE International Conference on Software Testing, Verification and Validation, ICST ’11, pages 377–386, Washington, DC, USA, 2011. IEEE Computer Society.
24. M. White, M. Linares-Vásquez, P. Johnson, C. Bernal-Cárdenas, and D. Poshyvanyk. User guided automation for testing mobile apps. In 23rd IEEE International Conference on Program Comprehension, (ICPC’15), volume 1, page to appear, May 2015.
25. D. Whitley and K. Kauth. GENITOR: A different genetic algorithm. In Proceedings of the 1988 Rocky Mountain Conference on Artificial Intelligence, pages 118–130, 1988.
26. W. Yang, M. R. Prasad, and T. Xie. A grey-box approach for automated gui-model generation of mobile applications. In Proceedings of the 16th International Conference on Fundamental Approaches to Software Engineering, FASE’13, pages 250–265, Berlin, Heidelberg, 2013. Springer-Verlag.

Index Terms

• Software and its engineering

  • Software creation and management

    • Software verification and validation

      • Software defect analysis

        • Software testing and debugging