What Is Hard about Teaching Machine Learning to Non-Majors? Insights from Classifying Instructors’ Learning Goals

Abstract

Given its societal impacts and applications to numerous fields, machine learning (ML) is an important topic to understand for many students outside of computer science and statistics. However, machine-learning education research is nascent, and research on this subject for non-majors thus far has only focused on curricula and courseware. We interviewed 10 instructors of ML courses for non-majors, inquiring as to what their students find both easy and difficult about machine learning. While ML has a reputation for having algorithms that are difficult to understand, in practice our participating instructors reported that it was not the algorithms that were difficult to teach, but the higher-level design decisions. We found that the learning goals that participants described as hard to teach were consistent with higher levels of the Structure of Observed Learning Outcomes (SOLO) taxonomy, such as making design decisions and comparing/contrasting models. We also found that the learning goals that were described as easy to teach, such as following the steps of particular algorithms, were consistent with the lower levels of the SOLO taxonomy. Realizing that higher-SOLO learning goals are more difficult to teach is useful for informing course design, public outreach, and the design of educational tools for teaching ML.

References

1. Aziza Alsaadi. 2001. A comparison of primary mathematics curriculum in England and Qatar: The SOLO taxonomy. Res. Learn. Math. 21, 3 (2001), 1.
2. John Biggs and Catherine Tang. 2007. Teaching for Quality Learning at University. Open University Press.
3. John B. Biggs. 2011. Teaching for Quality Learning at University: What the Student Does. McGraw-Hill Education (UK).
4. John B. Biggs and Kevin F. Collis. 1982. Evaluating the Quality of Learning: The SOLO Taxonomy (Structure of the Observed Learning Outcome). Academic Press.
5. Benjamin S. Bloom et al. 1956. Taxonomy of Educational Objectives. Vol. 1: Cognitive Domain. McKay, New York, NY, 20--24.
6. Gillian M. Boulton-Lewis. 1995. The SOLO taxonomy as a means of shaping and assessing learning in higher education. High. Educ. Res. Dev. 14, 2 (1995), 143--154.
7. Claus Brabrand and Bettina Dahl. 2007. Constructive alignment and the SOLO taxonomy: A comparative study of university competences in computer science vs. mathematics. In Proceedings of the 7th Baltic Sea Conference on Computing Education Research, Volume 88. Australian Computer Society, Inc., 3--17.
8. Claus Brabrand and Bettina Dahl. 2009. Using the SOLO taxonomy to analyze competence progression of university science curricula. High. Educ. 58, 4 (2009), 531--549.
9. Francisco Enrique Vicente Castro and Kathi Fisler. 2017. Designing a multi-faceted SOLO taxonomy to track program design skills through an entire course. In Proceedings of the 17th Koli Calling Conference on Computing Education Research. ACM, 10--19.
10. Helen Chick. 1998. Cognition in the formal modes: Research mathematics and the SOLO taxonomy. Math. Educ. Res. J. 10, 2 (1998), 4--26.
11. Ursula Fuller, Colin G. Johnson, Tuukka Ahoniemi, Diana Cukierman, Isidoro Hernán-Losada, Jana Jackova, Essi Lahtinen, Tracy L. Lewis, Donna McGee Thompson, Charles Riedesel, et al. 2007. Developing a computer science-specific learning taxonomy. ACM SIGCSE Bull. 39, 4 (2007), 152--170.
12. Yolanda Gil. 2016. Teaching big data analytics skills with intelligent workflow systems. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI’16). 3997--4088.
13. David Ginat and Eti Menashe. 2015. SOLO taxonomy for assessing novices’ algorithmic design. In Proceedings of the 46th ACM Technical Symposium on Computer Science Education. ACM, 452--457.
14. Richard Gluga, Judy Kay, Raymond Lister, Sabina Kleitman, and Tim Lever. 2012. Coming to terms with Bloom: An online tutorial for teachers of programming fundamentals. In Proceedings of the 14th Australasian Computing Education Conference, Volume 123. Australian Computer Society, Inc., 147--156.
15. Richard Gluga, Judy Kay, Raymond Lister, and Donna Teague. 2012. On the reliability of classifying programming tasks using a neo-piagetian theory of cognitive development. In Proceedings of the 9th Annual International Conference on International Computing Education Research. ACM, 31--38.
16. Michael Guerzhoy and Renjie Liao. 2018. Understanding How Recurrent Neural Networks Model Text. Retrieved from http://modelai.gettysburg.edu/2018/rnntext/index.html.
17. Mark Guzdial. 2018. Teaching Two Programming Languages in the First CS Course. Retrieved from https://cacm.acm.org/blogs/blog-cacm/228006-teaching-two-programming-languages-in-the-first-cs-course/fulltext.
18. Cindy E. Hmelo-Silver and Merav Green Pfeffer. 2004. Comparing expert and novice understanding of a complex system from the perspective of structures, behaviors, and functions. Cogn. Sci. 28, 1 (2004), 127--138.
19. Cruz Izu, Amali Weerasinghe, and Cheryl Pope. 2016. A study of code design skills in novice programmers using the SOLO taxonomy. In Proceedings of the 2016 ACM Conference on International Computing Education Research. ACM, 251--259.
20. Peter Lam and Yoke-Yeen Foong. 1996. Rasch analysis of math SOLO taxonomy levels using hierarchical items in testlets. ERIC-ED398271 (1996).
21. Niklas Lavesson. 2010. Learning machine learning: A case study. IEEE Trans. Educ. 53, 4 (2010), 672--676.
22. Raymond Lister, Beth Simon, Errol Thompson, Jacqueline L. Whalley, and Christine Prasad. 2006. Not seeing the forest for the trees: Novice programmers and the SOLO taxonomy. ACM SIGCSE Bull. 38, 3 (2006), 118--122.
23. Usman Mulbar, Abdul Rahman, and Ansari Ahmar. 2017. Analysis of the ability in mathematical problem-solving based on SOLO taxonomy and cognitive style. World Transactions on Engineering and Technology Education 15, 1 (2017).
24. Dave Oliver, Tony Dobele, Myles Greber, and Tim Roberts. 2004. This course has a Bloom rating of 3.9. In Proceedings of the 6th Australasian Conference on Computing Education, Volume 30. Australian Computer Society, Inc., 227--231.
25. Judy Sheard, Angela Carbone, Raymond Lister, Beth Simon, Errol Thompson, and Jacqueline L. Whalley. 2008. Going SOLO to assess novice programmers. In ACM SIGCSE Bulletin, Vol. 40. ACM, 209--213.
26. Shuhaida Shuhidan, Margaret Hamilton, and Daryl D’Souza. 2009. A taxonomic study of novice programming summative assessment. In Proceedings of the 11th Australasian Conference on Computing Education, Volume 95. Australian Computer Society, Inc., 147--156.
27. Lee S. Shulman. 1986. Those who understand: Knowledge growth in teaching. Educ. Res. 15, 2 (1986), 4--14. arXiv:https://doi.org/10.3102/0013189X015002004
28. Christopher W. Starr, Bill Manaris, and RoxAnn H. Stalvey. 2008. Bloom’s taxonomy revisited: Specifying assessable learning objectives in computer science. In ACM SIGCSE Bulletin, Vol. 40. ACM, 261--265.
29. Elisabeth Sulmont, Elizabeth Patitsas, and Jeremy R. Cooperstock. 2019. Can you teach me to machine learn? In Proceedings of the 50th ACM Technical Symposium on Computer Science Education. ACM, 948--954.
30. Claudia Szabo and Katrina Falkner. 2014. Neo-piagetian theory as a guide to curriculum analysis. In Proceedings of the 45th ACM Technical Symposium on Computer Science Education. ACM, 115--120.
31. Douglas Turnbull. 2012. Music Genre Classification. Retrieved from http://modelai.gettysburg.edu/2012/music/.
32. Jane Watson and Jonathan Moritz. 1998. Longitudinal development of chance measurement. Math. Educ. Res. J. 10, 2 (1998), 103--27.
33. Thomas Way, Lillian Cassel, Paula Matuszek, Mary-Angela Papalaskari, Divya Bonagiri, and Aravinda Gaddam. 2016. Broader and earlier access to machine learning. In Proceedings of the 2016 ACM Conference on Innovation and Technology in Computer Science Education. ACM, 362--362.
34. Thomas Way, Mary-Angela Papalaskari, Lillian Cassel, Paula Matuszek, Carol Weiss, and Yamini Praveena Tella. 2017. Machine learning modules for all disciplines. In Proceedings of the 2017 ACM Conference on Innovation and Technology in Computer Science Education. ACM, 84--85.
35. Jacqueline L. Whalley, Raymond Lister, Errol Thompson, Tony Clear, Phil Robbins, P. K. Kumar, and Christine Prasad. 2006. An Australasian study of reading and comprehension skills in novice programmers, using the Bloom and SOLO taxonomies. In Proceedings of the 8th Australasian Conference on Computing Education, Volume 52. Australian Computer Society, Inc., 243--252.

Index Terms

• Computing methodologies

  • Machine learning

• Social and professional topics

  • Professional topics

    • Computing education