Automatic assessment of OpenGL computer graphics assignments

ABSTRACT

Teaching and learning computer graphics is often considered challenging due to it requiring a diverse range of skills such as mathematics, programming, problem solving, and art and design. Assignments are a popular tool to support learning and to assess students' understanding. The value of such assignments depends on the ability to give fast (and ideally formative) feedback, and enabling students to interactively explore the solution space. This is often a problem, in particular for large classes, where assignment marking can take many days or even weeks. By the time feedback is received students often don't remember details, and there is usually no opportunity to resubmit and hence little motivation to reflect on and correct mistakes.

Previous work on assessing Computer Graphics assignments is rare and restricted to evaluating the quality of 3D models produced by students - usually using some form of image or mesh comparison, which only considers the final result, but not how it was obtained. In this paper we describe how to adapt CodeRunner, a free open-source question-type plug-in for Moodle, to OpenGL assignments, and our experience of using it with a class of about 300 students. Results were overwhelmingly positive and students perceived the tool as having significantly improved their learning.

References

1. Kirsti M. Ala-Mutka. 2005. A Survey of Automated Assessment Approaches for Programming Assignments. Computer Science Education 15, 2 (2005), 83–102.
2. Eamon Costello. 2013. Opening up to open source: looking at how Moodle was adopted in higher education. Open Learning: The Journal of Open, Distance and e-Learning 28, 3 (2013), 187–200.
3. Beth R. Crisp. 2007. Is it worth the effort? How feedback influences students’ subsequent submission of assessable work. Assessment & evaluation in higher education 32, 5 (2007), 571–581.
4. Christopher Douce, David Livingstone, and James Orwell. 2005. Automatic Testbased Assessment of Programming: A Review. J. Educ. Resour. Comput. 5, 3, Article 4 (Sept. 2005).
5. Hui Du and Lianqing Shu. 2011. Teaching Computer Graphics in Digital Game Specialty. Springer Berlin Heidelberg, Berlin, Heidelberg, 91–97.
6. Neil Duncan. 2007. ‘Feed-forward’: improving students’ use of tutors’ comments. Assessment & Evaluation in Higher Education 32, 3 (2007), 271–283.
7. Šárka Gergelitsová and Tomáš Holan. 2012. An Automatic Evaluation of Construction Geometry Assignments. In Proceedings of the 7th European Conference of Technology Enhanced Learning (EC-TEL 2012). 447–452.
8. Thomas Haig, Katrina Falkner, and Nickolas Falkner. 2013. Visualisation of Learning Management System Usage for Detecting Student Behaviour Patterns. In Proc. of the 15th Australasian Computing Education Conference. 107–115.
9. John Hattie and Helen Timperley. 2007. The power of feedback. Review of Educational Research 77, 1 (2007), 81–112.
10. Blake Hodgkinson, Christof Lutteroth, and Burkhard Wünsche. 2016. glGetFeedback - Towards automatic feedback and assessment for OpenGL 3D modelling assignments. In Proceedings of Image and Vision Computing New Zealand (IVCNZ 2016). 1–6.
11. Petri Ihantola, Tuukka Ahoniemi, Ville Karavirta, and Otto Seppälä. 2010. Review of Recent Systems for Automatic Assessment of Programming Assignments. In Proceedings of the 10th Koli Calling International Conference on Computing Education Research. 86–93.
12. Hieke Keuning, Johan Jeuring, and Bastiaan Heeren. 2016. Towards a Systematic Review of Automated Feedback Generation for Programming Exercises. In Proceedings of the 2016 ACM Conference on Innovation and Technology in Computer Science Education (ITiCSE ’16). ACM, 41–46.
13. F. Lamberti, A. Sanna, G. Paravati, and G. Carlevaris. 2014. Automatic Grading of 3D Computer Animation Laboratory Assignments. IEEE Transactions on Learning Technologies 7, 3 (2014), 280–290.
14. Lucas Layman, Laurie Williams, and Kelli Slaten. 2007. Note to Self: Make Assignments Meaningful. In Proceedings of the 38th SIGCSE Technical Symposium on Computer Science Education. 459–463.
15. Robert R. Lewis. 2012. Coaster: Teaching Computer Graphics with a Comprehensive Project – Work in Progress. Journal of Computing Sciences in Colleges 28, 1 (Oct. 2012), 192–199.
16. Richard Lobb and Jenny Harlow. 2016. Coderunner: A Tool for Assessing Computer Programming Skills. ACM Inroads 7, 1 (Feb. 2016), 47–51.
17. Mesa. {n. d.}. The Mesa 3D Graphics Library. https://www.mesa3d.org
18. Radu P. Mihail, Beth Rubin, and Judy Goldsmith. 2014. Online Discussions: Improving Education in CS?. In Proceedings of the 45th ACM Technical Symposium on Computer Science Education (SIGCSE ’14). 409–414. 2538862.2538898
19. Claudia Ott, Anthony Robins, and Kerry Shephard. 2016. Translating Principles of Effective Feedback for Students into the CS1 Context. Trans. Comput. Educ. 16, 1, Article 1 (Jan. 2016), 27 pages.
20. Achille Peternier, Daniel Thalmann, and Frédéric Vexo. 2006. Mental Vision: A Computer Graphics Teaching Platform. Springer Berlin Heidelberg, Berlin, Heidelberg, 223–232.
21. Vreda Pieterse. 2013. Automated Assessment of Programming Assignments. In Proceedings of the 3rd Computer Science Education Research Conference on Computer Science Education Research (CSERC ’13). 4:45–4:56.
22. A. Sanna, F. Lamberti, G. Paravati, and C. Demartini. 2012. Automatic Assessment of 3D Modeling Exams. IEEE Trans. on Learning Technologies 5, 1 (2012), 2–10.
23. Christopher Scaffidi, Aniket Dahotre, and Yan Zhang. 2012. How Well Do Online Forums Facilitate Discussion and Collaboration Among Novice Animation Programmers?. In Proceedings of the 43rd ACM Technical Symposium on Computer Science Education (SIGCSE ’12). 191–196.
24. Anne Morgan Spalter and Dana K. Tenneson. 2006. The Graphics Teaching Tool. In ACM SIGGRAPH 2006 Educators Program (SIGGRAPH ’06). ACM, New York, NY, USA, Article 41.

Index Terms

• Applied computing

  • Education

    • Computer-assisted instruction

    • Interactive learning environments

• Computing methodologies

  • Computer graphics