Ken Goldman
Lisa Kaczmarczyk
Michael C. Loui
Abstract
A Delphi process is a structured multi-step process that uses a group of experts to achieve a consensus opinion. We present the results of three Delphi processes to identify topics that are important and difficult in each of three introductory computing subjects: discrete math, programming fundamentals, and logic design. The topic rankings can be used to guide both the coverage of standardized tests of student learning (i.e., concept inventories) and can be used by instructors to identify what topics merit emphasis.
- M. J. Clayton. Delphi: A Technique to Harness Expert Opinion for Critical Decision-Making Task in Education. Educational Psychology, 17:373--386, 1997Google Scholar
Cross Ref
- N. Dalkey and O. Helmer. An experimental application of the delphi method to the use of experts. Management Science, 9:458--467, 1963.Google ScholarCross Ref
- D. Evans. Personal communication, January 2006.Google Scholar
- D. Evans et al. Progress on Concept Inventory Assessment Tools. In the Thirty-Third ASEE/IEEE Frontiers in Education, Nov 2003.Google ScholarCross Ref
- K. J. Goldman et al. Identifying Important and Difficult Concepts in Introductory Computing Courses using a Delphi Process. Technical Report UIUCDCS-R-2007-2917, University of Illinois Computer Science Department, November 2007.Google Scholar
- G. L. Gray, D. Evans, P. Cornwell, F. Costanzo, and Self. Toward a Nationwide Dynamics Concept Inventory Assessment Test. In American Society of Engineering Education, Annual Conference, June 2003.Google Scholar
- R. Hake. Interactive-engagement vs traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses. Am. J. Physics, 66, 1998.Google Scholar
- B. Marion and D. Baldwin. Sigcse commitee report: On the implementation of a discrete mathematics course. Inroads: ACM SIGCSE Bulletin, 39(2):109--126, 2007. Google ScholarDigital Library
- M. McCracken, V. Almstrum, D. Diaz, M. Guzdial, Hagan, Y. B.-D. Kolikant, C. Laxer, L. Thomas, Utting, and T. Wilusz. A multi-national, multi-institutional study of assessment of programming skills of first-year cs students. In ITiCSE-Working Group Reports (WGR), pages 125--180, 2001. Google ScholarDigital Library
Index Terms
- Identifying important and difficult concepts in introductory computing courses using a delphi process
Recommendations
Identifying student misconceptions of programming
SIGCSE '10: Proceedings of the 41st ACM technical symposium on Computer science educationComputing educators are often baffled by the misconceptions that their CS1 students hold. We need to understand these misconceptions more clearly in order to help students form correct conceptions. This paper describes one stage in the development of a ...
Identifying important and difficult concepts in introductory computing courses using a delphi process
SIGCSE '08: Proceedings of the 39th SIGCSE technical symposium on Computer science educationA Delphi process is a structured multi-step process that uses a group of experts to achieve a consensus opinion. We present the results of three Delphi processes to identify topics that are important and difficult in each of three introductory computing ...
Setting the Scope of Concept Inventories for Introductory Computing Subjects
A concept inventory is a standardized assessment tool intended to evaluate a student’s understanding of the core concepts of a topic. In order to create a concept inventory it is necessary to accurately identify these core concepts. A Delphi process is ...
Reviews
Frank Lawrence Friedman
A concept inventory (CI) is a measurement tool designed to evaluate student understanding of a particular topic or field. This paper discusses research in computer science (CS) education targeted at the formulation of a CI for 10 to 15 of the most important topics in introductory courses in each of three CS instructional domains: discrete math, programming fundamentals, and logic design. The authors point to the use of a methodically developed CI in a pre-test and a post-test as an effective tool for measuring student knowledge gain. Inspired by a similar effort that was claimed to have revolutionized the teaching of introductory physics, the authors carried out methodical and controlled research in CI identification as the first of a four-step process designed to aid in the development of good assessment tools for computer science education. There should be no argument that the development of a measured consensus among experts as to the important and difficult concepts in any course domain is a key element in developing curricula and pedagogic approaches that work within that domain. The problem lies not with the quality of, nor the assumptions underlying, the research reported on in this paper. The goal of this research, however, should be to improve student knowledge gain and retention. The four steps outlined in this research place far too great an emphasis on the CI, and far too little on student retention. If there is a link between retention and such a painstaking identification of a CI, this case needs to be demonstrated. I am skeptical. Online Computing Reviews Service
Access critical reviews of Computing literature here
Become a reviewer for Computing Reviews.
Comments