A Strategy to Combine Test-Driven Development and Test Criteria to Improve Learning of Programming Skills

ABSTRACT

Students usually have difficulties assimilating basic contents of introductory programming disciplines. Test-Driven Development (TDD) is an interesting approach to address this issue, but it does not focus on quality with respect to software testing. This study aims to verify the effectiveness of the use of software testing criteria for producing test cases with higher quality in the context of education with TDD. We used the test-driven development technique with a change in the refactoring step, adding an activity for creating test cases using software testing criteria. We performed two experimental studies to evaluate the effectiveness of the technique. Each study comprised two groups: one using traditional TDD and the other using TDD with test criteria, both having developed the same software. The results showed that groups applying the proposed intervention produced better test cases, with greater coverage, and not disrupting TDD. Moreover, we could observe that, in general, the test set for groups using TDD with criteria was more efficient, getting greater coverage with fewer test cases than groups using plain TDD.

References

1. M. F. Aniche and M. A. Gerosa. How the practice of TDD influences class design in object-oriented systems: Patterns of unit tests feedback. In 26th Brazilian Symposium on Software Engineering, pages 1--10, 2012.
2. K. Beck. Test-Driven Development: By Example. Addison-Wesley Professional, USA, 1 edition, Nov. 2002.
3. K. Buffardi and S. H. Edwards. Exploring influences on student adherence to test-driven development. In 17th Conference on Innovation and Technology in Computer Science Education, pages 105--110, 2012.
4. K. Buffardi and S. H. Edwards. Effective and ineffective software testing behaviors by novice programmers. In 9th Conference on International Computing Education Research, pages 83--90, New York, NY,USA, 2013.
5. A. Causevic, S. Punnekkat, and D. Sundmark. TDD^HQ: Achieving higher quality testing in test driven development. In 39th Euromicro Conference Series on Software Engineering and Advanced Applications, pages 33--36, 2013.
6. A. Causevic, R. Shukla, S. Punnekkat, and D. Sundmark. Effects of negative testing on TDD: An industrial experiment. In 14th Conference on Agile Software Development, pages 91--105, 2013.
7. A. Causevic, D. Sundmark, and S. Punnekkat. Impact of test design technique knowledge on test driven development: A controlled experiment. In 13th Conference on Agile Software Development, pages 138--152, 2012.
8. D. M. de Souza, S. Isotani, and E. F. Barbosa. Teaching novice programmers using ProgTest. International Journal of Knowledge and Learning, 10(1):60--77, 2015.
9. S. H. Edwards. Rethinking computer science education from a test-first perspective. In 18th Conference on Object-Oriented Programming, Systems, Languages, and Applications, pages 148--155, New York, NY, USA, 2003. ACM.
10. S. H. Edwards. Using software testing to move students from trial-and-error to reflection-in-action. In 35th Technical Symposium on Computer Science Education, pages 26--30, New York, NY,USA, 2004.
11. S. H. Edwards and Z. Shams. Do student programmers all tend to write the same software tests? In 19th Conference on Innovation and Technology in Computer Science Education, pages 171--176, New York, NY,USA, 2014.
12. D. Fucci and B. Turhan. On the role of tests in test-driven development: A differentiated and partial replication. Empirical Software Engineering, 19(2):277--302, Apr. 2014.
13. B. George and L. Williams. A structured experiment of test-driven development. Information and Software Technology, 46(5):337--342, Apr. 2004.
14. M. Hilton and D. S. Janzen. On teaching arrays with test-driven learning in WebIDE. In 17th Conference on Innovation and Technology in Computer Science Education, pages 93--98, 2012.
15. M. R. Hoffmann et al. EclEmma - Java code coverage for Eclipse. Software, Aug. 2006.
16. V. Isomöttönen and V. Lappalainen. CSI with games and an emphasis on TDD and unit testing: piling a trend upon a trend. Inroads, 3(3):62--68, Sept. 2012.
17. D. Janzen and H. Saiedian. Test-driven learning in early programming courses. In 39th Technical Symposium on Computer Science Education, pages 532--536, 2008.
18. D. S. Janzen and H. Saiedian. Test-driven learning: intrinsic integration of testing into the CS/SE curriculum. In 37th Technical Symposium on Computer Science Education, pages 254--258, 2006.
19. R. C. Martin. The Bowling Game Kata. Site: http://butunclebob.com/ArticleS.UncleBob.TheBowlingGameKata, 2005.
20. H. Munir, M. Moayyed, and K. Petersen. Considering rigor and relevance when evaluating test driven development: A systematic review. Information and Software Technology, 56(4):375--394, Apr. 2014.
21. G. J. Myers. The Art of Software Testing. John Wiley & Sons, New York, NY,USA, 2 edition, 2004.
22. S. L. Pfleeger. Design and analysis in software engineering: the language of case studies and formal experiments. Software Engineering Notes, 19(4):16--20, Oct. 1994.
23. S. Schaub. Teaching CS1 with web applications and test-driven development. Inroads, 41(2):113--117, June 2009.
24. W. Shelton, N. Li, P. Ammann, and J. Offutt. Adding criteria-based tests to test driven development. In 5th International Conference on Software Testing, Verification and Validation, pages 878--886, 2012.
25. D. M. d. Souza, J. C. Maldonado, and E. F. Barbosa. ProgTest: An environment for the submission and evaluation of programming assignments based on testing activities. In 24th Conference on Software Engineering Education and Training, pages 1--10, 2011.
26. J. Spacco, D. Hovemeyer, W. Pugh, F. Emad, J. K. Hollingsworth, and N. Padua-Perez. Experiences with Marmoset: designing and using an advanced submission and testing system for programming courses. In 11th Conference on Innovation and Technology in Computer Science Education, pages 13--17, New York, NY,USA, 2006.
27. M. Thornton, S. H. Edwards, R. P. Tan, and M. A. Pérez-Quiñones. Supporting student-written tests of GUI programs. In 39th Technical Symposium on Computer Science Education, pages 537--541, 2008.

Index Terms

• Applied computing

  • Education

• Software and its engineering

  • Software creation and management

    • Software verification and validation

      • Software defect analysis

        • Software testing and debugging