Developing Assessments to Determine Mastery of Programming Fundamentals

ABSTRACT

Current learning outcomes for introductory programming courses are relatively general, specifying tasks such as designing, implementing, testing and debugging programs that use some fundamental programming constructs. These outcomes impact what we teach, our expectations, and our assessments. Although prior work has demonstrated the utility of single concept assessments, most assessments used in formal examinations combine numerous heterogeneous concepts, resulting in complex and difficult tasks.

As a consequence, teachers may not be able to diagnose the actual difficulties faced by students and students are not provided with accurate feedback about their achievements. Such limitations on the nature and quality of feedback to teachers and students alike may contribute to the perceived difficulty and high dropout rates commonly observed in introductory programming courses.

In this paper we review the concepts that Computer Science education researchers have identified as important for novice programming. We survey expected learning outcomes for introductory programming courses, and analyse assessments designed for such courses to determine the individual components of syntax and semantics required to complete them. Having recognized the implicit and explicit expectations of novice programming courses, we look at the relationships between components, and progression between concepts. Finally, we demonstrate how some complex assessments can be decomposed into atomic elements that can be assessed independently.

References

1. Alireza Ahadi and Raymond Lister Geek Genes, Prior Knowledge, Stumbling Points and Learning Edge Momentum: Parts of the One Elephant?. Proceedings of the Ninth International Computing Education Research Conference (ICER '13) 123--128.
2. Deborah J. Armstrong 2006 The Quarks of Object-oriented Development.journal Commun. ACMvolume49,number2 (\dateFebruaryyear2006),pages123--128. ISSN0001-0782
3. Association for Computing Machinery (ACM) and IEEE Computer Society, Joint Task Force on Computing Curricula2001}% taskforce.2001.acm.computing\bibfieldauthor\ Association for Computing Machinery (ACM) and IEEE Computer Society, Joint Task Force on Computing Curricula 2001\natexlab.booktitle Computing Curricula 2001: Computer Science. ACM, New York, NY, USA. ISBNx978--1--4503--2309--3
4. Association for Computing Machinery (ACM) and IEEE Computer Society, Joint Task Force on Computing Curricula2013}% acmCC.2013\bibfieldauthor\ Association for Computing Machinery (ACM) and IEEE Computer Society, Joint Task Force on Computing Curricula .booktitle Computer Science Curricula 2013: Curriculum Guidelines for Undergraduate Degree Programs in Computer Science. ACM, New York, NY, USA. ISBNx978--1--4503--2309--3
5. Mordechai Ben-Ari 1998 Constructivism in Computer Science Education. Proceedings of the 29th SIGCSE Technical Symposium on Computer Science Education (SIGCSE '98) 257--261. ISBNx0--89791--994--7
6. Jens Bennedsen and Michael E. Caspersen 2007 Failure Rates in Introductory Programming.journal SIGCSE Bull.volume39,number2 (\dateJuneyear2007),pages32--36. ISSN0097--8418
7. Marc Berges, Andreas Mühling, and Peter Hubwieser 2012 The Gap between Knowledge and Ability. Proceedings of the 12th Koli Calling International Conference on Computing Education Research (Koli Calling 2012) 126--134. ISBNx978--1--4503--1795--5
8. James H. Block and Robert B. Burns 1976 Mastery Learning.journal Review of Research in Educationvolume4,number1 (1976),pages3--49.
9. Benjamin S. Bloom 1968 Learning for Mastery.journal Evaluation Commentvolume1,number2 (1968).
10. Benjamin S. Bloom 1984 The 2 Sigma Problem: The Search for Methods of Group Instruction as Effective as One-to-One Tutoring.journal Educational Researchervolume13,number6 (1984),pages4--16. ISSN0013189X, 1935102X http://www.jstor.org/stable/1175554
11. Richard Bornat, Saeed Dehnadi, and Simon 2008 Mental Models, Consistency and Programming Aptitude. Proceedings of the Tenth Australasian Computing Education Conference (ACE '08) 53--61. ISBNx978--1--920682--59-0 http://dl.acm.org/citation.cfm?id=1379249.1379253
12. Pauli Byckling and Jorma Sajaniemi 2006 A Role-based Analysis Model for the Evaluation of Novices' Programming Knowledge Development. Proceedings of the Second International Computing Education Research Workshop (ICER '06) 85--96. ISBNx1--59593--494--4
13. Ricardo Caceffo, Steve Wolfman, Kellogg S. Booth, and Rodolfo Azevedo 2016 Developing a Computer Science Concept Inventory for Introductory Programming. Proceedings of the 47th SIGCSE Technical Symposium on Computing Science Education (SIGCSE '16) 364--369.
14. Dino Capovilla, Marc Berges, Andreas Mühling, and Peter Hubwieser 2015 Handling Heterogeneity in Programming Courses for Freshmen. Proceedings of the International Conference on Learning and Teaching in Computing and Engineering (LaTiCE 2015) 197--203.
15. Yuliya Cherenkova, Daniel Zingaro, and Andrew Petersen 2014 Identifying Challenging CS1 Concepts in a Large Problem Dataset. Proceedings of the 45th SIGCSE Technical Symposium on Computer Science Education (SIGCSE '14) 695--700. ISBNx978--1--4503--2605--6
16. Liberal Arts Computer Science Consortium 2007 A 2007 Model Curriculum for a Liberal Arts Degree in Computer Science.journal J. Educ. Resour. Comput.volume7,number2, Articlearticleno2 (Juneyear2007). ISSN1531--4278
17. Albert T. Corbett and John R. Anderson 1992 Student Modeling and Mastery Learning in a Computer-based Programming Tutor. Proceedings of the Intelligent Tutoring Systems: Second International Conference (ITS '92) 413--420. ISBNx978--3--540--47254--4
18. Malcolm Corney, Raymond Lister, and Donna Teague 2011 Early Relational Reasoning and the Novice Programmer: Swapping as the Hello World of Relational Reasoning. Proceedings of the 13th Australasian Computing Education Conference (ACE '11) 95--104. http://crpit.com/confpapers/CRPITV114Corney.pdf
19. Nell Dale 2005 Content and Emphasis in CS1.journal SIGCSE Bull.volume37,number4 (\dateDecemberyear2005),pages69--73. ISSN0097--8418
20. Stephen Davies, Jennifer A. Polack-Wahl, and Karen Anewalt 2011 A Snapshot of Current Practices in Teaching the Introductory Programming Sequence. Proceedings of the 42nd ACM Technical Symposium on Computer Science Education (SIGCSE '11) 625--630. ISBNx978--1--4503-0500--6
21. Ton de Jong 2010 Cognitive load theory, educational research, and instructional design: some food for thought.journal Instructional Sciencevolume38,number2 (\date01 Maryear2010),pages105--134. ISSN1573--1952
22. Michael de Raadt, Richard Watson, and Mark Toleman 2005 Textbooks: Under inspection.howpublishedphTechnical Report -- Department of Maths and Computing, University of Southern Queensland. Toowoomba, Queensland, Australia. https://eprints.usq.edu.au/167/1/TechReport_Draft_10.pdf note-- retrieved: September 2017.
23. Benedict du Boulay 1986 Some Difficulties of Learning to Program.journal J. of Educational Comput. Researchvolume2,number1 (1986),pages57--73.
24. Anna Eckerdal, Robert McCartney, Jan Erik Moström, Mark Ratcliffe, and Carol Zander 2006 Can Graduating Students Design Software Systems?. Proceedings of the 37th SIGCSE Technical Symposium on Computer Science Education (SIGCSE '06) 403--407. ISBNx1--59593--259--3
25. Allison Elliott Tew and Mark Guzdial 2010 Developing a Validated Assessment of Fundamental CS1 Concepts. Proceedings of the 41st SIGCSE Technical Symposium on Computer Science Education (SIGCSE '10) 97--101. ISBNx978--1--4503-0006--3
26. Allison Elliott Tew and Mark Guzdial 2011 The FCS1: A Language Independent Assessment of CS1 Knowledge. Proceedings of the 42nd SIGCSE Technical Symposium on Computer Science Education (SIGCSE '11) 111--116. ISBNx978--1--4503-0500--6
27. Sophie Engle and Sami Rollins Expert Code Review and Mastery Learning in a Software Development Course.journal J. Comput. Sci. Coll.volume28,number4 (2013),pages139--147.
28. Heidi L. Eyre 2007 Keller's Personalized System of Instruction: Was it a Fleeting Fancy or is there a Revival on the Horizon'journal The Behavior Analyst Todayvolume8,number3 (2007),pages317--324.
29. Mark Guzdial and Briana Morrison 2016 Growing Computer Science Education into a STEM Education Discipline.journal Commun. ACMvolume59,number11 (\dateOctoberyear2016),pages31--33. ISSN0001-0782
30. John Hattie and Helen Timperley 2007 The Power of Feedback.journal Review of Educational Researchvolume77,number1 (\dateMarchyear2007),pages81--112. ISSN0034--6543
31. Matthew Hertz 2010 What Do “CS1” and “CS2” Mean?: Investigating Differences in the Early Courses. Proceedings of the 41st SIGCSE Technical Symposium on Computer Science Education (SIGCSE '10) 199--203. ISBNx978--1--4503-0006--3
32. Cruz Izu, Amali Weerasinghe, and Cheryl Pope 2016 A Study of Code Design Skills in Novice Programmers Using the SOLO Taxonomy. Proceedings of the 12th International Computing Education Research Conference (ICER '16) 251--259. ISBNx978--1--4503--4449--4
33. Philip Johnson-Laird 1983\natexlab.booktitle Mental Models. Harvard University Press, Cambridge, USA. ISBNx0--674--56881--8
34. Fred. S. Keller 1968 Goodbye, Teacher ...journal Journal of Applied Behavior Analysisnumber1 (1968),pages79--89.
35. Chen-Lin C. Kulik, James A. Kulik, and Robert L. Bangert-Drowns 1990 Effectiveness of Mastery Learning Programs: A Meta-Analysis.journal Review of Educational Researchvolume60,number2 (1990),pages265--299.
36. Amruth N. Kumar 1996 Fork Diagrams for Teaching Selection in CS I. Proceedings of the 27th SIGCSE Technical Symposium on Computer Science Education (SIGCSE '96) 348--352. ISBNx0--89791--757-X
37. Noel LeJeune 2010 Contract Grading with Mastery Learning in CS 1.journal J. Comput. Sci. Coll.volume26,number2 (\dateDecemberyear2010),pages149--156. ISSN1937--4771 http://dl.acm.org/citation.cfm?id=1858583.1858604
38. Colleen M. Lewis 2014 Exploring Variation in Students' Correct Traces of Linear Recursion. Proceedings of the Tenth International Computing Education Research Conference (ICER '14) 67--74. ISBNx978--1--4503--2755--8
39. Raymond Lister, Elizabeth S. Adams, Sue Fitzgerald, William Fone, John Hamer, Morten Lindholm, Robert McCartney, Jan Erik Moström, Kate Sanders, Otto Seppälä, Beth Simon, and Lynda Thomas 2004 A Multi-national Study of Reading and Tracing Skills in Novice Programmers. 2004 ITiCSE Working Group Reports (ITiCSE-WGR '04) 119--150.
40. Raymond Lister, Beth Simon, Errol Thompson, Jacqueline L. Whalley, and Christine Prasad 2006 Not Seeing the Forest for the Trees: Novice Programmers and the SOLO Taxonomy. Proceedings of the 11th Conference on Innovation and Technology in Computer Science Education (ITICSE '06) 118--122. ISBNx1--59593-055--8
41. Mike Lopez, Jacqueline Whalley, Phil Robbins, and Raymond Lister 2008 Relationships between Reading, Tracing and Writing Skills in Introductory Programming. Proceedings of the Fourth International Computing Education Research Workshop (ICER '08) 101--112. ISBNx978--1--60558--216-0
42. Andrew Luxton-Reilly 2016 Learning to Program is Easy. Proceedings of the 21st Conference on Innovation and Technology in Computer Science Education (ITiCSE '16) 284--289. ISBNx978--1--4503--4231--5
43. Andrew Luxton-Reilly and Andrew Petersen 2017 The Compound Nature of Novice Programming Assessments. Proceedings of the 19th Australasian Computing Education Conference (ACE '17) 26--35. ISBNx978--1--4503--4823--2
44. Linxiao Ma, John Ferguson, Marc Roper, and Murray Wood 2007 Investigating the Viability of Mental Models Held by Novice Programmers. Proceedings of the 38th SIGCSE Technical Symposium on Computer Science Education (SIGCSE '07) 499--503. ISBNx1--59593--361--1
45. Philipp Mayring 2014 title Qualitative Content Analysis: Theoretical Foundation, Basic Procedures and Software Solution. Klagenfurt.
46. Brendan McCane, Claudia Ott, Nick Meek, and Anthony Robins 2017 Mastery Learning in Introductory Programming. Proceedings of the 19th Australasian Computing Education Conference (ACE '17) 1--10. ISBNx978--1--4503--4823--2
47. Robert McCartney, Jonas Boustedt, Anna Eckerdal, Kate Sanders, and Carol Zander Can First-year Students Program Yet?: A Study Revisited. Proceedings of the Ninth International Computing Education Research Conference (ICER '13) 91--98. ISBNx978--1--4503--2243-0
48. Michael McCracken, Vicki Almstrum, Danny Diaz, Mark Guzdial, Dianne Hagan, Yifat Ben-David Kolikant, Cary Laxer, Lynda Thomas, Ian Utting, and Tadeusz Wilusz 2001 A Multi-national, Multi-institutional Study of Assessment of Programming Skills of First-year CS Students.journal SIGCSE Bull.volume33,number4 (\dateDecemberyear2001),pages125--180. ISSN0097--8418
49. Iain Milne and Glenn Rowe 2002 Difficulties in Learning and Teaching Programming--Views of Students and Tutors.journal Education and Information Technologiesvolume7,number1 (March 2002), pages 55--66. ISSN1360--2357
50. Luiz Augusto de Macêdo Morais, Jorge C. A. Figueiredo, and Dalton D. S. Guerrero 2014 Students Satisfaction with Mastery Learning in an Introductory Programming Course. III Congresso Brasileiro de Informática na Educac cåo (CBIE 2014) 1048--1052.
51. Briana B. Morrison, Lauren E. Margulieux, and Mark Guzdial 2015 Subgoals, Context, and Worked Examples in Learning Computing Problem Solving. Proceedings of the 11th Annual International Computing Education Research Conference (ICER '15) 21--29. ISBNx978--1--4503--3630--7
52. Andreas Mühling, Alexander Ruf, and Peter Hubwieser 2015 Design and First Results of a Psychometric Test for Measuring Basic Programming Abilities. Proceedings of the Workshop in Primary and Secondary Computing Education (WiPSCE '15) 2--10. ISBNx978--1--4503--3753--3
53. Pavol Návrat 1994 Hierarchies of Programming Concepts: Abstraction, Generality, and Beyond.journal SIGCSE Bull.volume26,number3 (Septemberyear1994), pages17--21. ISSN0097--8418
54. Fred G. W. C. Paas and Jeroen J. G. van Merriënboer 1994 Instructional Control of Cognitive Load in the Training of Complex Cognitive Tasks.journal Educational Psychology Review volume6,number4 (1994),pages351--371.
55. Miranda C. Parker, Mark Guzdial, and Shelly Engleman 2016 Replication, Validation, and Use of a Language Independent CS1 Knowledge Assessment. Proceedings of the 12th International Computing Education Research Conference (ICER '16) 93--101.
56. Terence Parr and Sam Harwell 2012--2016 Java 1.7 grammar for ANTLR v4.howpublishedhttps://github.com/antlr/grammars-v4/blob/master/java/Java.g4. (2012--2016).
57. Elizabeth Patitsas, Jesse Berlin, Michelle Craig, and Steve Easterbrook 2016 Evidence That Computer Science Grades Are Not Bimodal. Proceedings of the 12th International Computing Education Research Conference (ICER '16) 113--121.
58. Michela Pedroni and Bertrand Meyer 2010 Object-Oriented Modeling of Object-Oriented Concepts.. Proceedings of the Fourth International Conference on Informatics in Secondary Schools - Evolution and Perspectives (ISSEP 2010). Springer,pages155--169.
59. Andrew Petersen, Michelle Craig, Jennifer Campbell, and Anya Tafliovich 2016 Revisiting Why Students Drop CS1. Proceedings of the 16th Koli Calling International Conference on Computing Education Research (Koli Calling 2016) 71--80. ISBNx978--1--4503--4770--9
60. Andrew Petersen, Michelle Craig, and Daniel Zingaro 2011 Reviewing CS1 Exam Question Content. Proceedings of the 42nd SIGCSE Technical Symposium on Computer Science Education (SIGCSE '11) 631--636. ISBNx978--1--4503-0500--6
61. Jean Piaget 1976 Piaget's Theory. Piaget and His School: A Reader in Developmental Psychology, editor Bärbel Inhelder, Harold H. Chipman, and Charles Zwingmann (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, pages11--23. ISBNx978--3--642--46323--5
62. roulx2000}% proulx.2000.sigcse.patterns\bibfieldauthor\ Viera K. Proulx 2000 Programming Patterns and Design Patterns in the Introductory Computer Science Course. Proceedings of the 31st SIGCSE Technical Symposium on Computer Science Education (SIGCSE '00) 80--84. ISBNx1--58113--213--1
63. obins2010}% robins.2010.cse.learning\bibfieldauthor\ Anthony Robins 2010 Learning Edge Momentum: A New Account of Outcomes in CS1.journal Computer Science Educationvolume20,number1 (\dateMarchyear2010),pages37--71. ISSN0899--3408, 1744--5175
64. obins et al\mbox.2003}% robins.2003.cse.learning\bibfieldauthor\ Anthony Robins, Janet Rountree, and Nathan Rountree 2003 Learning and Teaching Programming: A Review and Discussion.journal Computer Science Educationvolume13,number2 (\dateJuneyear2003),pages137--142. ISSN0899--3408
65. ajaniemi et al\mbox.2006}% sajaniemi.2006.compsciedu.roles\bibfieldauthor\ Jorma Sajaniemi, Mordechai Ben-Ari, Pauli Byckling, Petri Gerdt, and Yevgeniya Kulikova 2006 Roles of Variables in Three Programming Paradigms.journal Computer Science Educationvolume16,number4 (\dateDecemberyear2006),pages261--279. ISSN0737-0024
66. anders et al\mbox.2013}% sanders.2013.canterbury\bibfieldauthor\ Kate Sanders, Marzieh Ahmadzadeh, Tony Clear, Stephen H. Edwards, Mikey Goldweber, Chris Johnson, Raymond Lister, Robert McCartney, Elizabeth Patitsas, and Jaime Spacco The Canterbury QuestionBank: Building a Repository of Multiple-choice CS1 and CS2 Questions. 2013 ITiCSE Working Group Reports (ITiCSE-WGR '13) 33--52. ISBNx978--1--4503--2665-0
67. anders and Thomas2007}% sanders.2007.sigcse.checklists\bibfieldauthor\ Kate Sanders and Lynda Thomas 2007 Checklists for Grading Object-oriented CS1 Programs: Concepts and Misconceptions. Proceedings of the 12th Conference on Innovation and Technology in Computer Science Education (ITiCSE '07) 166--170. ISBNx978--1--59593--610--3
68. choltz and Sanders2010}% scholtz.2010.iticse.mental\bibfieldauthor\ Tamarisk Lurlyn Scholtz and Ian Sanders 2010 Mental Models of Recursion: Investigating Students' Understanding of Recursion. Proceedings of the 15th Conference on Innovation and Technology in Computer Science Education (ITiCSE '10) 103--107. ISBNx978--1--60558--820--9
69. chulte and Bennedsen2006}% schulte.2006.icer.teachers\bibfieldauthor\ Carsten Schulte and Jens Bennedsen 2006 What Do Teachers Teach in Introductory Programming?. Proceedings of the Second International Computing Education Research Workshop (ICER '06) 17--28. ISBNx1--59593--494--4
70. "a, Ihantola, Isohanni, Sorva, and VihavainenSepp\"al\"a et al\mbox.2015}% seppala.2015.koli.know\bibfieldauthor\ Otto Sepp\"al\"a, Petri Ihantola, Essi Isohanni, Juha Sorva, and Arto Vihavainen 2015 Do We Know How Difficult the Rainfall Problem Is?. Proceedings of the 15th Koli Calling Conference on Computing Education Research (Koli Calling 2015) 87--96. ISBNx978--1--4503--4020--5
71. heard et al\mbox.2008}% sheard.2008.iticse.solo\bibfieldauthor\ Judy Sheard, Angela Carbone, Raymond Lister, Beth Simon, Errol Thompson, and Jacqueline L. Whalley 2008 Going SOLO to Assess Novice Programmers. Proceedings of the 13th Conference on Innovation and Technology in Computer Science Education (ITiCSE '08) 209--213. ISBNx978--1--60558-078--4
72. Carbone, Chinn, Clear, Corney, D'Souza, Fenwick, Harland, Laakso, and TeagueSheard et al\mbox.2013}% sheard.2013.ace.difficult\bibfieldauthor\ Judy Sheard, Simon, Angela Carbone, Donald Chinn, Tony Clear, Malcolm Corney, Daryl D'Souza, Joel Fenwick, James Harland, Mikko-Jussi Laakso, and Donna Teague How Difficult Are Exams?: A Framework for Assessing the Complexity of Introductory Programming Exams. Proceedings of the 15th Australasian Computing Education Conference (ACE '13) 145--154. ISBNx978--1--921770--21-0 http://dl.acm.org/citation.cfm?id=2667199.2667215
73. huhidan et al\mbox.2009}% shuhidan.2009.ace.taxonomic\bibfieldauthor\ Shuhaida Shuhidan, Margaret Hamilton, and Daryl D'Souza A Taxonomic Study of Novice Programming Summative Assessment. Proceedings of the 11th Australasian Computing Education Conference (ACE '09) 147--156. ISBNx978--1--920682--76--7 http://dl.acm.org/citation.cfm?id=1862712.1862734
74. Robert M. Siegfried, Jason P. Siegfried, and Gina Alexandro 2016 A Longitudinal Analysis of the Reid List of First Programming Languages.journal Information Systems Education Journalvolume14,number6 pages47--54. ISSN1545--679X http://isedj.org/2016--14/
75. Simon Soloway's Rainfall Problem Has Become Harder. Proceedings of the 2013 Conference on Learning and Teaching in Computing and Engineering 130--135.
76. Simon, Donald Chinn, Michael de Raadt, Anne Philpott, Judy Sheard, Mikko-Jussi Laakso, Daryl D'Souza, James Skene, Angela Carbone, Tony Clear, Raymond Lister, and Geoff Warburton 2012 Introductory Programming: Examining the Exams. Proceedings of the 14th Australasian Computing Education Conference (ACE '12) 61--70. ISBNx978--1--921770-04--3 http://dl.acm.org/citation.cfm?id=2483716.2483724
77. Simon, Judy Sheard, Daryl D'Souza, Peter Klemperer, Leo Porter, Juha Sorva, Martijn Stegeman, and Daniel Zingaro 2016\natexlaba.showarticletitleBenchmarking Introductory Programming Exams: How and Why. Proceedings of the 21st Conference on Innovation and Technology in Computer Science Education 154--159. ISBNx978--1--4503--4231--5
78. Simon, Judy Sheard, Daryl D'Souza, Peter Klemperer, Leo Porter, Juha Sorva, Martijn Stegeman, and Daniel Zingaro 2016\natexlabb.showarticletitleBenchmarking Introductory Programming Exams: Some Preliminary Results. Proceedings of the 12th International Computing Education Research Conference (ICER '16) 103--111. ISBNx978--1--4503--4449--4
79. imon et al\mbox.2006}% simon.2006.icer.commonsense\bibfieldauthor\ Beth Simon, Tzu-Yi Chen, Gary Lewandowski, Robert McCartney, and Kate Sanders 2006 Commonsense Computing: What Students Know before We Teach (Episode 1: Sorting). Proceedings of the Second International Computing Education Research Workshop (ICER '06) 29--40. ISBNx1--59593--494--4
80. kudder and Luxton-Reilly2014}% skudder.2014.ace.worked\bibfieldauthor\ Ben Skudder and Andrew Luxton-Reilly 2014 Worked Examples in Computer Science. Proceedings of the 16th Australasian Computing Education Conference (ACE '14) 59--64. ISBNx978--1--921770--31--9 http://dl.acm.org/citation.cfm?id=2667490.2667497
81. oloway et al\mbox.1982}% soloway.1982.chi.tapping\bibfieldauthor\ Elliot Soloway, Kate Ehrlich, and Jeffrey Bonar 1982 Tapping into Tacit Programming Knowledge. Proceedings of the 1982 Conference on Human Factors in Computing Systems (CHI '82). ACM, New York, NY, USA,pages52--57.
82. Juha Sorva 2012 Visual Program Simulation in Introductory Programming Education.thesistype Ph.D. Dissertation.school Aalto University.
83. Juha Sorva, Ville Karavirta, and Lauri Malmi A Review of Generic Program Visualization Systems for Introductory Programming Education.journal Trans. Comput. Educ.volume13,number4, Articlearticleno15 (November 2013). ISSN1946--6226
84. Andreas Stefik and Susanna Siebert An Empirical Investigation into Programming Language Syntax.journal ACM Transactions on Computing Educationvolume13,number4 (2013),pages19:1. ISSN1946--6226
85. John Sweller Cognitive Load during Problem Solving: Effects on Learning.journal Cognitive Science volume12,number2 (1988),pages257--285. ISSN1551--6709
86. John Sweller 1994 Cognitive Load Theory, Learning Difficulty, and Instructional Design.journal Learning and Instruction volume4 (1994), pages295--312.
87. John Sweller and Paul Chandler 1994 Why Some Material Is Difficult to Learn.journal Cognition and Instructionvolume12,number3 (1994),pages185--233.
88. Cynthia Taylor, Daniel Zingaro, Leo Porter, Kevin Webb, Cynthia Bailey Lee, and Mike Clancy 2014 Computer Science Concept Inventories: Past and Future.journal Computer Science Educationvolume24,number4 (\dateOctoberyear2014),pages253--276.
89. Donna Teague and Raymond Lister 2014 Longitudinal Think Aloud Study of a Novice Programmer. Proceedings of the 16th Australasian Computing Education Conference (ACE '14) 41--50. ISBNx978--1--921770--31--9 http://dl.acm.org/citation.cfm?id=2667490.2667495
90. Mark Urban-Lurain and Donald J. Weinshank 1999 “I Do and I Understand”: Mastery Model Learning for a Large Non-major Course. Proceedings of the 30th SIGCSE Technical Symposium on Computer Science Education (SIGCSE '99) 150--154. ISBNx1--58113-085--6
91. Jeroen J. G. van Merriënboer, Otto Jelsma, and Fred G. W. C. Paas 1992 Training for Reflective Expertise: A Four-Component Instructional Design Model for Complex Cognitive Skills.journal Educational Technology Research and Developmentvolume40,number2 (01 Jun 1992), pages23--43. ISSN1556--6501
92. Anne Venables, Grace Tan, and Raymond Lister A Closer Look at Tracing, Explaining and Code Writing Skills in the Novice Programmer. Proceedings of the Fifth International Computing Education Research Workshop (ICER '09) 117--128. ISBNx978--1--60558--615--1
93. Christopher Watson and Frederick W.B. Li 2014 Failure Rates in Introductory Programming Revisited. Proceedings of the 19th Conference on Innovation and Technology in Computer Science Education (ITiCSE '14) 39--44. ISBNx978--1--4503--2833--3
94. Jacqueline Whalley and Nadia Kasto 2014 How Difficult Are Novice Code Writing Tasks?: A Software Metrics Approach. Proceedings of the 16th Australasian Computing Education Conference (ACE '14) 105--112. ISBNx978--1--921770--31--9 http://dl.acm.org/citation.cfm?id=2667490.2667503

Index Terms

• Social and professional topics

  • Professional topics

    • Computing education