Programming, Problem Solving, and Self-Awareness: Effects of Explicit Guidance

ABSTRACT

More people are learning to code than ever, but most learning opportunities do not explicitly teach the problem solving skills necessary to succeed at open-ended programming problems. In this paper, we present a new approach to impart these skills, consisting of: 1) explicit instruction on programming problem solving, which frames coding as a process of translating mental representations of problems and solutions into source code, 2) a method of visualizing and monitoring progression through six problem solving stages, 3) explicit, on-demand prompts for learners to reflect on their strategies when seeking help from instructors, and 4) context-sensitive help embedded in a code editor that reinforces the problem solving instruction. We experimentally evaluated the effects of our intervention across two 2-week web development summer camps with 48 high school students, finding that the intervention increased productivity, independence, programming self-efficacy, metacognitive awareness, and growth mindset. We discuss the implications of these results on learning technologies and classroom instruction.

References

1. Petek Askar and David Davenport. 2009. An investigation of factors related to self-efficacy for Java programming among engineering students. Online Submission 8, 1. http://eric.ed.gov/?id=ED503900.
2. Albert Bandura. 1977. Self-efficacy: Toward a unifying theory of behavioral change. Psychological Review 8, 2: 191--215.
3. Albert Bandura. 1986. Social Foundations of Thought and Action. Prentice Hall, Englewood Cliffs, NJ.
4. Henry Jay Becker. 1987. The importance of a methodology that maximizes falsifiability: Its applicability to research about Logo. Educational Researcher, 16, 5: 11--16.
5. Jens Bennedsen and Michael E. Caspersen. 2007. Failure rates in introductory programming. ACM SIGCSE Bulletin 39, 2: 32--36.
6. Susan Bergin, Ronan Reilly, and Desmond Traynor. 2005. Examining the role of self-regulated learning on introductory programming performance. In Proceedings of the International Workshop on Computing Education Research (ICER '05), 81--86.
7. Lisa S. Blackwell, Kali H. Trzesniewski, and Carol Sorich Dweck. 2007. Implicit theories of intelligence predict achievement across an adolescent transition: A longitudinal study and an intervention. Child development, 78, 1: 246--263.
8. Bureau of Labor Statistics. U.S. Department of Labor, Occupational Outlook Handbook, 2014--15 Edition, Software Developers. Retrieved August 11, 2015 from http://www.bls.gov/ooh/computer-and-informationtechnology/software-developers.htm.
9. Joel Brandt, Philip J. Guo, Joel Lewenstein, Mira Dontcheva, and Scott R. Klemmer. 2009. Two studies of opportunistic programming: interleaving web foraging, learning, and writing code. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '09), 1589--1598.
10. Jill Cao, Scott D. Fleming, and Margaret Burnett. 2011. An exploration of design opportunities for "gardening" end-user programmers' ideas. In Proceedings of the IEEE symposium on Visual Languages and Human-Centric Computing (VL/HCC '11), 35--42.
11. Jill Cao. 2012. An idea garden for end-user programmers. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Extended Abstracts on Human Factors in Computing Systems (CHI EA '12), 915--918.
12. Jill Cao, Irwin Kwan, Faezeh Bahmani, Margaret Burnett, Scott D. Fleming, Josh Jordahl, Amber Horvath, and Sherry Yang. 2013. End-user programmers in trouble: Can the Idea Garden help them to help themselves? In Proceedings of the IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC '13), 151--158.
13. Jill Cao, Scott D. Fleming, Margaret Burnett, and Christopher Scaffidi. 2014. Idea Garden: situated support for problem solving by end-user programmers. Interacting with Computers.
14. Parmit K. Chilana, Celena Alcock, Shruti Dembla, Anson Ho, Ada Hurst, Brett Armstrong, and Philip J. Guo. 2015. Perceptions of Non-CS Majors in Intro Programming: The Rise of the Conversational Programmer. In Proceedings of the IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC), to appear.
15. Douglas H. Clements and Dominic F. Gullo. 1984. Effects of computer programming on young children's cognition. Journal of Educational Psychology 76, 6: 1051.
16. David R. Cross and Scott G. Paris. 1988. Developmental and instructional analyses of children's metacognition and reading comprehension. Journal of Educational Psychology 80, 2: 131.
17. Savia Coutinho. 2008. Self-efficacy, metacognition, and performance. North American Journal of Psychology 10, 1: 165.
18. Carol Dweck. 2006. Mindset: The new psychology of success. Random House.
19. Anneli Eteläpelto. 1993. Metacognition and the expertise of computer program comprehension. Scandinavian Journal of Educational Research 37, 3: 243--254.
20. Allan Fisher and Jane Margolis. 2002. Unlocking the clubhouse: the Carnegie Mellon experience. ACM SIGCSE Bulletin 34, 2: 79--83.
21. Abraham E. Flanigan, Markeya S. Peteranetz, Duane F. Shell, and Leen-Kiat Soh. 2015. Exploring changes in computer science students' implicit theories of intelligence across the semester. In Proceedings of the International Conference on Computing Education Research (ICER '15), 161--168.
22. Forrest-Pressley, Donna-Lynn, and G. E. MacKinnon. 1985. Metacognition, Cognition, and Human Performance: Theoretical perspectives. Vol. 1. Academic Press.
23. James G. Greeno and Rogers P. Hall. 1997. Practicing representation. Phi Delta Kappan 78, 5: 361.
24. M. Havenga, 2011. Problem-solving processes in computer programming: a case study." In Southern African Computer Lecturers' Association (SACLA) Conference Proceedings, 91--99.
25. Jean-Michel Hoc and Anh Nguyen-Xuan. 1990. Language semantics, mental models and analogy. Psychology of programming, 10: 139--156.
26. William Jernigan, Amber Horvath, Michael J. Lee, Margaret M. Burnett, Taylor Cuilty, Sandeep Kuttal, Anicia Peters, Irwin Kwan, Faezeh Bahmani, Andrew J. Ko. 2015. A principled evaluation for a principled Idea Garden. In Proceedings IEEE Visual Languages and Human-Centric Computing (VL/HCC '15), to appear.
27. Katy Jordan. 2014. Initial trends in enrollment and completion of massive open online courses. The International Review Of Research In Open And Distributed Learning 15, 1.
28. Caitlin Kelleher, Randy Pausch, and Sara Kiesler. 2007. Storytelling Alice motivates middle school girls to learn computer programming. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 1455--1464.
29. Andrew J. Ko, Brad Myers, and Htet Htet Aung. 2004. Six learning barriers in end-user programming systems." In the Proceedings of the IEEE Symposium on Visual Languages and Human Centric Computing, 199--206.
30. Andrew J. Ko. 2009. Attitudes and self-efficacy in young adults' computing autobiographies. In IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC '09), 67--74.
31. Andrew J. Ko and Yann Riche. 2011. The role of conceptual knowledge in API usability. In the Proceedings of the IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC), 173--176.
32. Michael J. Lee, Faezeh Bahmani, Irwin Kwan, Jilian LaFerte, Polina Charters, Amber Horvath, Fanny Luor, Jill Cao, Catherine Law, Michael Beswetherick, Sheridan Long, Margaret M. Burnett, and Andrew J. Ko. 2014. Principles of a debugging-first puzzle game for computing education. In the IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC '14), 57--64.
33. Michael J. Lee and Amy J. Ko. 2015. Comparing the effectiveness of online learning approaches on CS1 learning outcomes. In Proceedings of the International Conference on Computing Education Research (ICER '15), 237--246.
34. Paul Luo Li, Andrew J. Ko, and Jiamin Zhu. 2015. What makes a great software engineer? In Proceedings of the International Conference on Software Engineering (ICSE '15), 700--710.
35. Raymond Lister. 2011. Geek genes and bimodal grades. ACM Inroads 1, 3: 16--17.
36. Charlie McDowell, Linda Werner, Heather Bullock, Julian Fernald. 2002. The effects of pair-programming on performance in an introductory programming course. ACM SIGCSE Bulletin, 34, 1: 38--42.
37. Janet Ed Metcalfe and Arthur P. Shimamura. 1994. Metacognition: Knowing about knowing. The MIT Press.
38. John L. Nietfeld and Gregory Schraw. 2002. The effect of knowledge and strategy training on monitoring accuracy. The Journal of Educational Research 95, 3: 131--142.
39. David B. Palumbo and W. Michael Reed. 1991. The effect of BASIC programming language instruction on high school students' problem solving ability and computer anxiety. Journal of Research on computing in Education 23, 3: 343--372.
40. Leo Porter and Beth Simon. 2013. Retaining nearly onethird more majors with a trio of instructional best practices in CS1. In Proceedings of the ACM Technical Symposium on Computer Science Education (SIGCSE '13), 165--170.
41. Mitchel Resnick, John Maloney, Andrés MonroyHernández, Natalie Rusk, Evelyn Eastmond, Karen Bren.nan, Amon Millner, Eric Rosenbaum, Jay Silver, Brian Silverman, and Yasmin Kafai. 2009. Scratch: programming for all. Communications of the ACM 52, 11: 60--67.
42. Martin P. Robillard, Wesley Coelho, and Gail C. Murphy. 2004. How effective developers investigate source code: An exploratory study. IEEE Transactions on Software Engineering, 30, 12: 889--903.
43. Ido Roll, Vincent Aleven, Bruce M. McLaren, and Kenneth R. Koedinger. 2007. Designing for metacognition-applying cognitive tutor principles to the tutoring of help seeking. Metacognition and Learning 2, 2--3: 125--140.
44. Ido Roll, Natasha G. Holmes, James Day, and Doug Bonn. Evaluating Metacognitive Scaffolding in Guided Invention Activities. Instructional Science 40, no. 4 (2012): 691--710.
45. Christopher Scaffidi, Mary Shaw, and Brad Myers. 2005. Estimating the numbers of end users and end user programmers. In Proceedings of the IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC '05), 207--214.
46. Chris Scaffidi and Chris Chambers. 2012. Skill progression demonstrated by users in the Scratch animation environment. International Journal of Human-Computer Interaction, 28, 6: 383--398.
47. Gregory Schraw. Promoting general metacognitive awareness. Instructional science 26, 1--2: 113--125.
48. Gregory Schraw, Kent J. Crippen, and Kendall Hartley. 2006. Promoting self-regulation in science education: Metacognition as part of a broader perspective on learning. Research in Science Education 36, 1--2: 111139.
49. Michael James Scott and Gheorghita Ghinea. 2014. On the domain-specificity of mindsets: The relationship between aptitude beliefs and programming practice. IEEE Transactions on Education, 57, 3: 169--174.
50. Teresa M. Shaft. 1995. Helping programmers understand computer programs: the use of metacognition. ACM SIGMIS Database 26, 4: 25--46.
51. Robert H. Sloan and Patrick Troy. 2008. CS 0.5: a better approach to introductory computer science for majors. ACM SIGCSE Bulletin, 40, 1: 271--275.
52. Abdrabo Moghazy Soliman and Elsayed Khaled Mathna. 2009. Metacognitive strategy training improves driving situation awareness. Social Behavior and Personality: An International Journal 37, 9: 1161--1170.
53. Rayne A. Sperling, Bruce C. Howard, Lee Ann Miller, and Cheryl Murphy. 2002. Measures of children's knowledge and regulation of cognition. Contemporary educational psychology 27, 1: 51--79.
54. Kentaro Toyama. 2011. Technology as amplifier in international development. In Proceedings of the iConference, 75--82.
55. David Whitebread, Penny Coltman, Deborah Pino Pasternak, Claire Sangster, Valeska Grau, Sue Bingham, Qais Almeqdad, and Demetra Demetriou. 2009. The development of two observational tools for assessing metacognition and self-regulated learning in young children. Metacognition and Learning 4, 1: 63--85.

Index Terms

• Human-centered computing

• Social and professional topics

  • Professional topics

    • Computing education

      • Computing education programs

        • Computer science education

        • Information science education