skip to main content
10.1145/3230977.3231000acmconferencesArticle/Chapter ViewAbstractPublication PagesicerConference Proceedingsconference-collections
research-article
Public Access

Evaluating the Efficiency and Effectiveness of Adaptive Parsons Problems

Published: 08 August 2018 Publication History

Abstract

Practice is essential for learning. There is evidence that solving Parsons problems (putting mixed up code blocks in order) is a more efficient, but just as effective, form of practice than writing code from scratch. However, not all students successfully solve every Parsons problem. Making the problems adaptive, so that the difficulty changes based on the learner's performance, should keep the learner in Vygotsky's zone of proximal development and maximize learning gains. This paper reports on a study comparing the efficiency and effectiveness of learning from solving adaptive Parsons problems vs non-adaptive Parsons problem vs writing the equivalent code. The adaptive Parsons problems used both intra-problem and inter-problem adaptation. Intra-problem adaptation means that if the learner is struggling to solve the current problem, the problem can dynamically be made easier. Inter-problem adaptation means that the difficulty of the next problem is modified based on the learner's performance on the previous problem. This study provides evidence that solving intra-problem and inter-problem adaptive Parsons problems is a more efficient, but just as effective, form of practice as writing the equivalent code.

References

[1]
John R. Anderson, Frederick G. Conrad, and Albert T. Corbett . 1989. Skill Acquisition and the LISP Tutor. Cognitive Science Vol. 13 (1989), 467--505.
[2]
John R. Anderson, Albert T. Corbett, Kenneth R. Koedinger, and Ray Pelletier . 1995. Cognitive Tutors: Lessons Learned. Journal of the Learning Sciences Vol. 4, 2 (1995), 167--207.
[3]
Owen Astrachan, Jan Cuny, Chris Stephenson, and Cameron Wilson . 2011. The CS10K project: mobilizing the community to transform high school computing Proceedings of the 42nd ACM technical symposium on Computer science education. ACM, 85--86.
[4]
Ronald Baecker and Ian Small . 1990. Animation at the interface. The art of human-computer interface design (1990), 251--267.
[5]
Tim Bell, Peter Andreae, and Lynn Lambert . 2010. Computer science in New Zealand high schools. In Proceedings of the Twelfth Australasian Conference on Computing Education-Volume 103. Australian Computer Society, Inc., 15--22.
[6]
Klara Benda, Amy Bruckman, and Mark Guzdial . 2012. When Life and Learning Do Not Fit: Challenges of Workload and Communication in Introductory Computer Science Online. Trans. Comput. Educ. Vol. 12, 4 (2012), 1--38.
[7]
Jens Bennedsen and Michael E. Caspersen . 2007. Failure rates in introductory programming. SIGCSE Bull. Vol. 39, 2 (2007), 32--36.
[8]
Laura E. Berk and Adam Winsler . 1995. Scaffolding Children's Learning: Vygotsky and Early Childhood Education. National Association for the Education of Young Children.
[9]
Elizabeth L Bjork and Robert A. Bjork . 2011. Making things hard on yourself, but in a good way: Creating desirable difficulties to enhance learning. Psychology and the real world: Essays illustrating fundamental contributions to society (2011), 56--64.
[10]
Benedict Du Boulay . 1988. Some Difficulties of Learning to Program. Lawrence Erlbaum Associates, 283--299.
[11]
Neil C. C. Brown, Sue Sentance, Tom Crick, and Simon Humphreys . 2014. Restart: The Resurgence of Computer Science in UK Schools. Trans. Comput. Educ. Vol. 14, 2 (2014), 1--22.
[12]
Michael E Caspersen and Palle Nowack . 2013. Computational thinking and practice: A generic approach to computing in Danish high schools. In Proceedings of the Fifteenth Australasian Computing Education Conference-Volume 136. Australian Computer Society, Inc., 137--143.
[13]
Fanny Chevalier, Nathalie Henry Riche, Catherine Plaisant, Amira Chalbi, and Christophe Hurter . 2016. Animations 25 years later: new roles and opportunities Proceedings of the International Working Conference on Advanced Visual Interfaces. ACM, 280--287.
[14]
Jacob Cohen . 1988. Statistical power analysis for the behavioral sciences. 2nd.
[15]
Gemma Corbalan, Liesbeth Kester, and Jeroen JG Van Merriënboer . 2008. Selecting learning tasks: Effects of adaptation and shared control on learning efficiency and task involvement. Contemporary Educational Psychology Vol. 33, 4 (2008), 733--756.
[16]
Albert T Corbett, Kenneth R Koedinger, and John R Anderson . 1997. Intelligent tutoring systems. Handbook of human-computer interaction Vol. 5 (1997), 849--874.
[17]
Tom Crick and Sue Sentance . 2011. Computing at school: stimulating computing education in the UK Proceedings of the 11th Koli Calling International Conference on Computing Education Research. ACM, 122--123.
[18]
Jan Cuny, Diane A Baxter, Daniel D Garcia, Jeff Gray, and Ralph Morelli . 2014. CS principles professional development: only 9,500 to go! Proceedings of the 45th ACM technical symposium on Computer science education. ACM, 543--544.
[19]
Paul Denny, Andrew Luxton-Reilly, and Beth Simon . 2008. Evaluating a new exam question: Parsons problems. In Proceedings of the fourth international workshop on computing education research. ACM, 113--124.
[20]
Alireza Ebrahimi . 1994. Novice programmer errors: language constructs and plan composition. International Journal of Human-Computer Studies Vol. 41 (1994), 457--480.
[21]
Barbara J Ericson, Mark J Guzdial, and Briana B Morrison . 2015. Analysis of interactive features designed to enhance learning in an ebook Proceedings of the eleventh annual International Conference on International Computing Education Research. ACM, 169--178.
[22]
Barbara J Ericson, Lauren E Margulieux, and Jochen Rick . 2017. Solving parsons problems versus fixing and writing code Proceedings of the 17th Koli Calling Conference on Computing Education Research. ACM, 20--29.
[23]
Kathi Fisler . 2014. The recurring rainfall problem. In Proceedings of the tenth annual conference on International computing education research. ACM, 35--42.
[24]
Judith Gal-Ezer, Catriel Beeri, David Harel, and Amiram Yehudai . 1995. A high school program in computer science. Computer Vol. 28, 10 (1995), 73--80.
[25]
Stuart Garner . 2007. An Exploration of How a Technology-Facilitated Part-Complete Solution Method Supports the Learning of Computer Programming. Journal of Issues in Informing Science and Information Technology Vol. 4 (2007), 491--501.
[26]
Kyle James Harms, Jason Chen, and Caitlin L Kelleher . 2016. Distractors in Parsons Problems Decrease Learning Efficiency for Young Novice Programmers. In Proceedings of the 2016 ACM Conference on International Computing Education Research. ACM, 241--250.
[27]
Juha Helminen, Petri Ihantola, Ville Karavirta, and Lauri Malmi . 2012. How do students solve parsons programming problems?: an analysis of interaction traces. In Proceedings of the ninth annual international conference on International computing education research. ACM, 119--126.
[28]
Petri Ihantola and Ville Karavirta . 2011. Two-Dimensional Parson's Puzzles: The Concept, Tools, and First Observations. Journal of Information Technology Education Vol. 10 (2011), 119--132.
[29]
Paivi Kinnunen and Beth Simon . 2010. Experiencing programming assignments in CS1: the emotional toll Proceedings of the Sixth international workshop on Computing education research. ACM, 77--86.
[30]
Amruth N Kumar . 2018. Epplets: A Tool for Solving Parsons Puzzles. In Proceedings of the 49th ACM Technical Symposium on Computer Science Education. ACM, 527--532.
[31]
Karen Lang, Ria Galanos, Joanna Goode, Deborah Seehorn, Fran Trees, Pat Phillips, and Chris Stephenson . 2013. Bugs in the System: Computer Science Teacher Certification in the U.S. Report. The Computer Science Teachers Association The Association for Computing Machinery.
[32]
Raymond Lister, Elizabeth S Adams, Sue Fitzgerald, William Fone, John Hamer, Morten Lindholm, Robert McCartney, Jan Erik Moström, Kate Sanders, Otto Sepp"al"a, et almbox. . 2004. A multi-national study of reading and tracing skills in novice programmers ACM SIGCSE Bulletin, Vol. Vol. 36. ACM, 119--150.
[33]
Brad Miller and David Ranum . 2014. Runestone interactive: tools for creating interactive course materials Proceedings of the first ACM conference on Learning@ scale conference. ACM, 213--214.
[34]
Dale Parsons and Patricia Haden . 2006. Parson's programming puzzles: a fun and effective learning tool for first programming courses. In Proceedings of the 8th Australasian Conference on Computing Education-Volume 52. Australian Computer Society, Inc., 157--163.
[35]
Andrew Petersen, Michelle Craig, and Daniel Zingaro . 2011. Reviewing CS1 exam question content. In Proceedings of the 42nd ACM technical symposium on Computer science education. ACM, 631--636.
[36]
Barbara Rogoff . 1990. Apprenticeship in thinking: Cognitive development in sociocultural activity. Oxford University Press, New York, NY, USA.
[37]
Otto Sepp"al"a, Petri Ihantola, Essi Isohanni, Juha Sorva, and Arto Vihavainen . 2015. Do we know how difficult the rainfall problem is?. In Proceedings of the 15th Koli Calling Conference on Computing Education Research. ACM, 87--96.
[38]
Simon . 2013. Soloway's Rainfall Problem has become Harder. In Learning and Teaching in Computing and Engineering. IEEE Computer Society, Washington DC, USA, 130--135.
[39]
Elliot Soloway . 1986. Learning to program = learning to construct mechanisms and explanations. Commun. ACM Vol. 29, 9 (1986), 850--858. deftempurl%http://dl.acm.org/citation.cfm?doid=6592.6594 tempurl
[40]
John Sweller . 1988. Cognitive load during problem solving: Effects on learning. Cognitive science Vol. 12, 2 (1988), 257--285.
[41]
John Sweller and Graham Cooper . 1985. The Use of Worked Examples as a Substitute for Problem Solving in Learning Algebra. Cognition and Instruction Vol. 2, 1 (1985), 59--89.
[42]
John Gregory Trafton and Brian J. Reiser . 1993. The contributions of studying examples and solving problems to skill acquisition 15th Annual Conference of the Cognitive Science Society. Lawrence Erlbaum Associates, Inc., 1017--1022. deftempurl%http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.52.9933&rep=rep1&type=pdf tempurl
[43]
Jeroen JG Van Merriënboer and Marcel BM De Croock . 1992. Strategies for computer-based programming instruction: Program completion vs. program generation. Journal of Educational Computing Research Vol. 8, 3 (1992), 365--394.
[44]
Jeroen JG Van Merriënboer, Paul A Kirschner, and Liesbeth Kester . 2003. Taking the load off a learner's mind: Instructional design for complex learning. Educational psychologist Vol. 38, 1 (2003), 5--13.

Cited By

View all
  • (2025)Measuring the Impact of Distractors on Student Learning Gains while Using Proof BlocksProceedings of the 56th ACM Technical Symposium on Computer Science Education V. 110.1145/3641554.3701787(917-923)Online publication date: 12-Feb-2025
  • (2024)Evaluating Micro Parsons Problems as Exam QuestionsProceedings of the 2024 on Innovation and Technology in Computer Science Education V. 110.1145/3649217.3653583(674-680)Online publication date: 3-Jul-2024
  • (2024)Automating Personalized Parsons Problems with Customized Contexts and ConceptsProceedings of the 2024 on Innovation and Technology in Computer Science Education V. 110.1145/3649217.3653568(688-694)Online publication date: 3-Jul-2024
  • Show More Cited By

Index Terms

  1. Evaluating the Efficiency and Effectiveness of Adaptive Parsons Problems

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    ICER '18: Proceedings of the 2018 ACM Conference on International Computing Education Research
    August 2018
    307 pages
    ISBN:9781450356282
    DOI:10.1145/3230977
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 08 August 2018

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. adaptive parsons problems
    2. cognitive load
    3. parson's problems
    4. parsons problems
    5. parsons puzzles
    6. zone of proximal development

    Qualifiers

    • Research-article

    Funding Sources

    Conference

    ICER '18
    Sponsor:

    Acceptance Rates

    ICER '18 Paper Acceptance Rate 28 of 125 submissions, 22%;
    Overall Acceptance Rate 189 of 803 submissions, 24%

    Upcoming Conference

    ICER 2025
    ACM Conference on International Computing Education Research
    August 3 - 6, 2025
    Charlottesville , VA , USA

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)265
    • Downloads (Last 6 weeks)17
    Reflects downloads up to 18 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2025)Measuring the Impact of Distractors on Student Learning Gains while Using Proof BlocksProceedings of the 56th ACM Technical Symposium on Computer Science Education V. 110.1145/3641554.3701787(917-923)Online publication date: 12-Feb-2025
    • (2024)Evaluating Micro Parsons Problems as Exam QuestionsProceedings of the 2024 on Innovation and Technology in Computer Science Education V. 110.1145/3649217.3653583(674-680)Online publication date: 3-Jul-2024
    • (2024)Automating Personalized Parsons Problems with Customized Contexts and ConceptsProceedings of the 2024 on Innovation and Technology in Computer Science Education V. 110.1145/3649217.3653568(688-694)Online publication date: 3-Jul-2024
    • (2024)Distractors Make You Pay Attention: Investigating the Learning Outcomes of Including Distractor Blocks in Parsons ProblemsProceedings of the 2024 ACM Conference on International Computing Education Research - Volume 110.1145/3632620.3671114(177-191)Online publication date: 12-Aug-2024
    • (2024)Scaffolding Novices: Analyzing When and How Parsons Problems Impact Novice Programming in an Integrated Science AssignmentProceedings of the 2024 ACM Conference on International Computing Education Research - Volume 110.1145/3632620.3671110(42-54)Online publication date: 12-Aug-2024
    • (2024)Disentangling the Learning Gains from Reading a Book Chapter and Completing Proof Blocks ProblemsProceedings of the 55th ACM Technical Symposium on Computer Science Education V. 110.1145/3626252.3630831(1056-1062)Online publication date: 7-Mar-2024
    • (2024)Generating Multi-Part Autogradable Faded Parsons Problems From Code-Writing ExercisesProceedings of the 55th ACM Technical Symposium on Computer Science Education V. 110.1145/3626252.3630786(179-185)Online publication date: 7-Mar-2024
    • (2024)SQL Puzzles: Evaluating Micro Parsons Problems With Different Feedbacks as Practice for NovicesProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641910(1-15)Online publication date: 11-May-2024
    • (2023)Effect of Block-Based Python Programming Environment on Programming LearningApplied Sciences10.3390/app13191089813:19(10898)Online publication date: 30-Sep-2023
    • (2023)Understanding the Effects of Using Parsons Problems to Scaffold Code Writing for Students with Varying CS Self-Efficacy LevelsProceedings of the 23rd Koli Calling International Conference on Computing Education Research10.1145/3631802.3631832(1-12)Online publication date: 13-Nov-2023
    • Show More Cited By

    View Options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Login options

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media