An effective testing method for end-user programmers

Abstract

End-user programmers do not have extensive knowledge of various software testing methodologies used by professional testers. While they are creating the vast majority of software today, errors are pervasive in the programs due to the lack of testing techniques readily adoptable by end-user programmers. In this article we argue that the technique of metamorphic testing is both practical and effective for end-user programmers.

References

1. P. E. Ammann and J. C. Knight. Data diversity: an approach to software fault tolerance. IEEE Transactions on Computers, 37(4):418--425, 1988.
2. B. Beizer. Software Testing Techniques. Van Nostrand Reinhold, New York, 1990.
3. M. Blum and S. Kannan. Designing programs that check their work. In Proceedings of the 31st Annual ACM Symposium on Theory of Computing (STOC'89), pages 86--97. ACM Press, New York, 1989.
4. M. Blum and S. Kannan. Designing programs that check their work. Journal of the ACM, 42(1):269--291, 1995.
5. M. Blum, M. Luby, and R. Rubinfeld. Self-testing/correcting with applications to numerical problems. Journal of Computer and System Sciences, 47(3):549--595, 1993.
6. B. Boehm, C. Abts, A. Brown, S. Chulani, B. Clark, E. Horowitz, R. Madachy, D. Reifer, and B. Steece. Software Cost Estimation with Cocomo II. Prentice Hall PTR, Upper Saddle River, NJ, 2000.
7. M. Burnett, C. Cook, and G. Rothermel. End-user software engineering. Communications of the ACM, 47(9):53--58, 2004.
8. T. Y. Chen, S. C. Cheung, and S. M. Yiu. Metamorphic testing: a new approach for generating next test cases. Technical Report HKUST-CS98-01, Department of Computer Science, Hong Kong University of Science and Technology, Hong Kong, 1998.
9. T. Y. Chen, J. Feng, and T. H. Tse. Metamorphic testing of programs on partial differential equations: a case study. In Proceedings of the 26th Annual International Computer Software and Applications Conference (COMPSAC 2002), pages 327--333. IEEE Computer Society Press, Los Alamitos, California, 2002.
10. T. Y. Chen, D. H. Huang, T. H. Tse, and Z. Q. Zhou, Case studies on the selection of useful relations in metamorphic testing. In Proceedings of the 4th Ibero-American Symposium on Software Engineering and Knowledge Engineering (JIISIC 2004), pages 569--583, Madrid, Spain, 2004. Polytechnic University of Madrid.
11. T. Y. Chen, T. H. Tse, and Z. Q. Zhou. Semi-proving: an integrated method based on global symbolic evaluation and metamorphic testing. In Proceedings of the ACM SIGSOFT International Symposium on Software Testing and Analysis (ISSTA 2002), pages 191--195. ACM Press, New York, 2002.
12. T. Y. Chen, T. H. Tse, and Z. Q. Zhou. Fault-based testing without the need of oracles. Information and Software Technology, 45(1):1--9, 2003.
13. W. J. Cody, Jr and W. Waite. Software Manual for the Elementary Functions. Prentice Hall, Englewood Cliffs, New Jersey, 1980.
14. C. F. Gerald and P. O. Wheatley. Applied Numerical Analysis. Addison Wesley, Reading, Massachusetts, 1999.
15. A. Gotlieb. Exploiting symmetries to test programs. In Proceedings of the 14th International Symposium on Software Reliability Engineering (ISSRE 2003), pages 365--374, 2003.
16. A. Gotlieb and B. Botella. Automated metamorphic testing. In Proceedings of the 27th Annual International Computer Software and Applications Conference (COMPSAC 2003), pages 34--40. IEEE Computer Society Press, Los Alamitos, California, 2003.
17. G. J. Myers. The Art of Software Testing. Wiley, New York, 1979.
18. R. Panko. Finding spreadsheet errors. Information Week, Issue 529, page 100, May 1995.
19. G. Rothermel, M. Burnett, L. Li, C. Dupuis, and A. Sheretov. A methodology for testing spreadsheets. ACM Transactions on Software Engineering and Methodology, 10(1):110--147, 2001.
20. E. J. Weyuker. On testing non-testable programs. The Computer Journal, 25(4):465--470, 1982.

Index Terms

• Software and its engineering

  • Software creation and management

    • Software verification and validation

      • Software defect analysis

        • Software testing and debugging