Paul Denny
ABSTRACT
Badge-based achievement systems are being used increasingly to drive user participation and engagement across a variety of platforms and contexts. Despite positive anecdotal reports, there is currently little empirical evidence to support their efficacy in particular domains. With the recent rapid growth of tools for online learning, an interesting open question for educators is the extent to which badges can positively impact student participation.
In this paper, we report on a large-scale (n > 1000) randomized, controlled experiment measuring the impact of incorporating a badge-based achievement system within an online learning tool. We discover a highly significant positive effect on the quantity of students' contributions, without a corresponding reduction in their quality, as well as on the period of time over which students engaged with the tool. Students enjoyed being able to earn badges, and indicated a strong preference for having them available in the user interface.
- Australasian Association for Engineering Education, Award for innovation in curricula, learning and teaching, "PeerWise", 2009.Google Scholar
- Aleven, V., and Koedinger, K. An effective metacognitive strategy - learning by doing and explaining with a computer-based cognitive tutor. Cognitive Science 26, 2 (2002), 147--179.Google ScholarCross Ref
- Antin, J., and Churchill, E. Badges in social media: A social psychological perspective. In CHI 2011 Gamification Workshop Proceedings (Vancouver, BC, Canada, 2011).Google Scholar
- Baker, R., Walonoski, J., Heffernan, N., Roll, I., Corbett, A., and Koedinger, K. Why students engage in "gaming the system" behavior in interactive learning environments. Journal of Interactive Learning Research 19, 2 (April 2008), 185--224.Google Scholar
- Bottomley, S., and Denny, P. A participatory learning approach to biochemistry using student authored and evaluated multiple-choice questions. Biochemistry and Molecular Biology Education 39, 5 (2011), 352--361.Google ScholarCross Ref
- Burguillo, J. C. Using game theory and competition-based learning to stimulate student motivation and performance. Comput. Educ. 55, 2 (Sept. 2010), 566--575. Google ScholarDigital Library
- Chi, M., Bassok, M., Lewis, M., Reimann, P., and Glaser, R. Self-explanations: How students study and use examples in learning to solve problems. Cognitive Science 13 (1989), 145--182.Google ScholarCross Ref
- Chi, M., Leeuw, N., Chiu, M.-H., and LaVancher, C. Eliciting self-explanations improves understanding. Cognitive Science 18 (1994), 439--477.Google Scholar
- Denny, P., Luxton-Reilly, A., and Hamer, J. The PeerWise system of student contributed assessment questions. In Proceedings of the tenth conference on Australasian computing education - Volume 78, ACE '08, Australian Computer Society, Inc. (Darlinghurst, Australia, Australia, 2008), 69--74. Google ScholarDigital Library
- Denny, P., Luxton-Reilly, A., and Hamer, J. Student use of the PeerWise system. SIGCSE Bull. 40, 3 (June 2008), 73--77. Google ScholarDigital Library
- Deterding, S. Gamification: designing for motivation. interactions 19, 4 (July 2012), 14--17. Google ScholarDigital Library
- Deterding, S., Khaled, R., Nacke, L., and Dixon, D. Gamification: Toward a definition. In CHI 2011 Gamification Workshop Proceedings (Vancouver, BC, Canada, 2011).Google Scholar
- Electronic Entertainment Design and Research. EEDAR study shows more achievements in games leads to higher review scores, increased sales. Press release, Oct. 2007.Google Scholar
- Farzan, R., DiMicco, J. M., Millen, D. R., Brownholtz, B., Geyer, W., and Dugan, C. When the experiment is over: Deploying an incentive system to all the users. In Proceedings of the Symposium on Persuasive Technology, In conjunction with the AISB 2008 Convention (2008).Google Scholar
- Fitz-Walter, Z., Tjondronegoro, D., and Wyeth, P. Orientation passport: using gamification to engage university students. In Proceedings of the 23rd Australian Computer-Human Interaction Conference, OzCHI '11, ACM (Canberra, Australia, 2011), 122--125. Google ScholarDigital Library
- Foos, P. W. Effects of student-written questions on student test performance. Teaching of Psychology 16, 2 (1989), 77--78.Google ScholarCross Ref
- Haladyna, T., and Downing, S. How many options is enough for a multiple-choice test item? Educational and Psychological Measurement 53, 4 (1993), 999--1010.Google ScholarCross Ref
- Hamer, J., Cutts, Q., Jackova, J., Luxton-Reilly, A., McCartney, R., Purchase, H., Riedesel, C., Saeli, M., Sanders, K., and Sheard, J. Contributing student pedagogy. SIGCSE Bull. 40, 4 (Nov. 2008), 194--212. Google ScholarDigital Library
- Jakobsson, M. The achievement machine: Understanding Xbox 360 achievements in gaming practices. The International Journal of Computer Game Research 11, 1 (2011).Google Scholar
- Lee, J. J., and Hammer, J. Gamification in education: What, how, why bother? Academic Exchange Quarterly 15, 2 (2011).Google Scholar
- Luxton-Reilly, A., and Denny, P. Constructive evaluation: a pedagogy of student-contributed assessment. Computer Science Education 20, 2 (2010), 145--167.Google ScholarCross Ref
- Montola, M., Nummenmaa, T., Lucero, A., Boberg, M., and Korhonen, H. Applying game achievement systems to enhance user experience in a photo sharing service. In Academic MindTrek 2009 Conference, ACM Press (Tampere, Finland, 2009), 94--97. Google ScholarDigital Library
- Nicol, D. E-assessment by design: using multiple-choice tests to good effect. Journal of Further and Higher Education 31, 1 (Feb. 2007), 53--64.Google ScholarCross Ref
- Nokia. Nokia Image Space. http://research.nokia.com/page/4987, 2009. {Online; accessed 20-March-2012}.Google Scholar
- PeerWise. Online learning tool (http://peerwise.cs.auckland.ac.nz/).Google Scholar
- Random.org. True random number service (http://www.random.org/).Google Scholar
- Roediger, H. L., and Karpicke, J. D. Test-enhanced learning: Taking memory tests improves long-term retention. Psychological Science 17 (2006), 249--255.Google ScholarCross Ref
- Sykes, A., Denny, P., and Nicolson, L. PeerWise - the marmite of veterinary student learning (Brighton Business School, University of Brighton, Brighton, UK, Nov. 2011). 820--830.Google Scholar
- Vyas, R., and Supe, A. Multiple choice questions: a literature review on the optimal number of options. The National Medical Journal of India 21, 3 (June 2008), 130--133.Google Scholar
- Webb, N. Peer interaction and learning in small groups. International Journal of Education Research 13 (1989), 21--39.Google ScholarCross Ref
Index Terms
- The effect of virtual achievements on student engagement
Recommendations
Improving Student Experience and Learning Performance with Traditional Instructional Methods and New Digital Media
Human-Computer Interaction – INTERACT 2021AbstractStudies in Technology-enhanced learning (TEL) involve different instructional media and methods to understand their effects on two main outcomes: learning performance and user experience. This paper investigated the impact of three gamification ...
Learner autonomy, microcredentials and self-reflection: a review of a Moodle-based medical English review course
English has become increasingly important for medical professionals in Japan. This means English language teachers in medical schools are tasked to not only deliver an intensive time constrained English curriculum, but also expected to design and ...
Student engagement in massive open online courses
Completion rates in massive open online courses MOOCs are disturbingly low. Existing analysis has focused on patterns of resource access and prediction of drop-out using learning analytics. In contrast, the effectiveness of teaching programs in ...
Comments