skip to main content
10.1145/3159450.3159608acmconferencesArticle/Chapter ViewAbstractPublication PagessigcseConference Proceedingsconference-collections
research-article

On the Use of Semantic-Based AIG to Automatically Generate Programming Exercises

Published: 21 February 2018 Publication History

Abstract

In introductory programming courses, proficiency is typically achieved through substantial practice in the form of relatively small assignments and quizzes. Unfortunately, creating programming assignments and quizzes is both, time-consuming and error-prone. We use Automatic Item Generation (AIG) in order to address the problem of creating numerous programming exercises that can be used for assignments or quizzes in introductory programming courses. AIG is based on the use of test-item templates with embedded variables and formulas which are resolved by a computer program with actual values to generate test-items. Thus, hundreds or even thousands of test-items can be generated with a single test-item template. We present a semantic-based AIG that uses linked open data (LOD) and automatically generates contextual programming exercises. The approach was incorporated into an existing self-assessment and practice tool for students learning computer programming. The tool has been used in different introductory programming courses to generate a set of practice exercises different for each student, but with the same difficulty and quality.

References

[1]
M. McCracken, V. Almstrum, D. Diaz, M. Guzdial, D. Hagen, Y. Kolikant, C. Laxer, L. Thomas, I. Utting, and T. Wilusz. 2001. A Multi-National, Multi-Institutional Study of Assessment of Programming Skills of First- year CS Students. SIGCSE Bulletin, 33, no. 4, 125--140.
[2]
L. Zavala. 2016. Read, Manipulate, and Write: A study of the role of these cumulative skills in learning computer programming. Proceedings of the ASEE NE 2016 Conference. April 28--30, University of Rhode Island, Kingston, RI.
[3]
L. Zavala and B. Mendoza. 2017. Precursor skills to writing code. Journal of Computing Science in Colleges, 32, 3, 149--156.
[4]
J. Sweller, and G. A. Cooper. 1985. The use of worked examples as a substitute for problem-solving in learning algebra. Cognition and Instruction, 2, no. 1(1985), 59--89.
[5]
M. Guzdial and J. Robertson. 2010. Too much programming too soon? Communications of the ACM, 53, no. 3 10--11, ACM, March.
[6]
M. Guzdial. 2015. What's the best way to teach computer science to beginners? Communications of the ACM, 58, no. 2,12--13, ACM, January.
[7]
D. Deb, M. M. Fuad, and M. Kanan. 2017. Creating Engaging Exercises with Mobile Response System (MRS). Proceedings of the 2017 ACM SIGCSE Technical Symposium on Computer Science Education. March 08--11, Seattle, Washington.
[8]
M. J. Scott, S. Counsell, S. Lauria, S. Swift, A. Tucker, M. Shepperd and G. Ghinea. 2015. Enhancing Practice and Achievement in Introductory Programming with a Robot Olympics. IEEE Transactions on Education, vol. 58, no. 4, 249--254.
[9]
M. J. Gierl and H. Lai. 2012. The Role of Item Models in Automatic Item Generation. International Journal of Testing, 12(3), 273--298.
[10]
H. Wainer, N. J. Dorans, D. Eignor, R. Flaugher, B. F. Green, R. J. Mislevy, L. Steinberg, and D. Thissen. 2000. Computerized adaptive testing: A primer, Second Edition. Hillsdale, NJ: Lawrence Erlbaum Associates.
[11]
J. R. Savery and T. M. Duffy. 1995. Problem based learning: An instructional model and its constructivist framework. Educational technology, 35, no. 5, 31--38.
[12]
W. W. Cobern. 1993. Contextual constructivism: The impact of culture on the learning and teaching of science. InK. G.Tobin (Ed.),The practice of constructivism in science education(pp. 51--69). Hillsdale, NJ: Lawrence Erlbaum Associates.
[13]
T. Im, S. Siva, J. Freeman, B. Magerko, G. Hendler, S. Engelman, M. Miller, B. Villa, and T. McKlin. 2017 Incorporating music into an introductory college level programming course for non-majors. Proceedings of the IEEE Integrated STEM Education Conference (ISEC), Princeton, NJ, 2017, pp. 43--48.
[14]
J. Stigall and S. Sharma. 2017. Virtual reality instructional modules for introductory programming courses. Proceedings of the 2017 IEEE Integrated STEM Education Conference (ISEC), Princeton, NJ, pp. 34--42.
[15]
M. A. Rubio, R. Romero-Zaliz, C. Mañoso and A. P. de Madrid. 2014. Enhancing an introductory programming course with physical computing modules. Proceedings of the 2014 IEEE Frontiers in Education Conference (FIE), Madrid, 2014, pp. 1--8.
[16]
F. I. Anfurrutia, A. Álvarez, M. Larrañaga and J. M. López-Gil. 2016. Incorporating educational robots and visual programming environments in introductory programming courses. Proceedings of the 2016 International Symposium on Computers in Education (SIIE), Salamanca, pp. 1--4.
[17]
E. Weilemann, P. Brune, and D. Meyer. 2016. Geek Toys for Non-techies? Using Robots in Introductory Programming Courses for Computer Science Non-majors. Proceedings of the 49th Hawaii International Conference on System Sciences (HICSS), Koloa, HI, pp. 31--40.
[18]
W. S. Yue and W. L. Wan. 2015. The effectiveness of digital game for introductory programming concepts. Proceedings of the 10th International Conference for Internet Technology and Secured Transactions (ICITST), London, pp. 421--425.
[19]
M. Paralić and E. Pietriková. 2014. Learning by game creation in introductory programming course: 5-Year-long study. Proceedings of the 12th IEEE International Conference on Emerging eLearning Technologies and Applications (ICETA), Stary Smokovec, pp. 391--396.
[20]
R. Rahul, A. Whitchurch and M. Rao. 2014. An open source graphical robot programming environment in introductory programming curriculum for undergraduates. Proceedings of the International Conference on MOOC, Innovation and Technology in Education (MITE), Patiala, pp. 96--100.
[21]
N. Adamo-Villani, T. Haley-Hermiz and R. Cutler. 2013. Using a Serious Game Approach to Teach 'Operator Precedence' to Introductory Programming Students. Proceedings of the 17th International Conference on Information Visualization, London, pp. 523--526.
[22]
B. Mendoza, J. Reyes-Alamo, H. Wu, A. Carranza, and L. Zavala. 2016. iPractice: A Self-assessment Tool for Students Learning Computer Programming in an Urban Campus. Journal of Computing Sciences in Colleges, 31, 3, 93--100.
[23]
F. Prados, I. Boada, J. Soler, and J. Poch. 2005. Automatic generation and correction of technical exercices. Proceedings of the International Conference on Engineering and Computer Education (ICECE 2005), Madrid, Spain.
[24]
Vinu E.V. and P. Sreenivasa Kumar. 2015. Improving Large-Scale Assessment Tests by Ontology Based Approach. Proceedings of the Twenty-Eighth International Florida Artificial Intelligence Research Society Conference. May 18--20, Hollywood, Florida, USA.
[25]
A. Papasalouros, K. Kotis and K. Kanaris. 2008. Automatic generation of multiple-choice questions from domain ontologies. Proceedings of the IADIS e-Learning 2008 conference, Amsterdam, Netherlands.
[26]
Al-Yahya M. 2014. Ontology-Based Multiple Choice Question Generation. The Scientific World Journal. Vol. 2014, Article ID 274949, 9 pages, 2014.
[27]
M. Cubric and M. Tosic. 2010 Towards automatic generation of eAssessment using semantic web technologies. In Proceedings of the 2010 International Computer Assisted Assessment Conference, Jul 2010. University of Southampton.
[28]
Foulonneau, M. 2012. Generating Educational Assessment Items from Linked Open Data: the Case of DBpedia. In: Garcia-Castro, R.e.a. (ed.): Extended Semantic Web Conference Workshops (at the 8th International conference on The Semantic Web), Vol. LNCS 7117. Springer, Heraklion, Greece, pp 16--27.
[29]
N. Karamanis, L. A. Ha, and R. Mitkov. 2006. Generating multiple-choice test items from medical text: A pilot study. Paper presented at the Fourth International Conference Natural Language Generation. Sydney, Australia.
[30]
B. Liu, H. Chen, and W. He. 2008. A framework of deriving adaptive feedback from educational ontologies. Proceedings of the 9th International Conference for Young Computer Scientists, Zhang Jia Jie Hunan, China.
[31]
T. Berners-Lee. 2006. Linked Data. Design Issues. W3C. Retrieved 2017-08--18.
[32]
C. Bizer, T. Heath, T. Berners-Lee. 2009. Linked Data--The Story So Far. International Journal on Semantic Web and Information Systems. 5 (3): 1--22. ISSN 1552--6283.

Cited By

View all
  • (2024)Leveraging Large Language Models to Support Authoring Gamified Programming ExercisesApplied Sciences10.3390/app1418834414:18(8344)Online publication date: 16-Sep-2024
  • (2024)Evaluating Contextually Personalized Programming Exercises Created with Generative AIProceedings of the 2024 ACM Conference on International Computing Education Research - Volume 110.1145/3632620.3671103(95-113)Online publication date: 12-Aug-2024
  • (2024)Comparing the Security of Three Proctoring Regimens for Bring-Your-Own-Device ExamsProceedings of the 55th ACM Technical Symposium on Computer Science Education V. 110.1145/3626252.3630809(429-435)Online publication date: 7-Mar-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
SIGCSE '18: Proceedings of the 49th ACM Technical Symposium on Computer Science Education
February 2018
1174 pages
ISBN:9781450351034
DOI:10.1145/3159450
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 February 2018

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. assessment tools
  2. automatic item generation
  3. contextual examples
  4. linked open data
  5. motivation
  6. programming exercises
  7. semantic web
  8. semantic-based aig

Qualifiers

  • Research-article

Conference

SIGCSE '18
Sponsor:

Acceptance Rates

SIGCSE '18 Paper Acceptance Rate 161 of 459 submissions, 35%;
Overall Acceptance Rate 1,787 of 5,146 submissions, 35%

Upcoming Conference

SIGCSE TS 2025
The 56th ACM Technical Symposium on Computer Science Education
February 26 - March 1, 2025
Pittsburgh , PA , USA

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)27
  • Downloads (Last 6 weeks)1
Reflects downloads up to 18 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2024)Leveraging Large Language Models to Support Authoring Gamified Programming ExercisesApplied Sciences10.3390/app1418834414:18(8344)Online publication date: 16-Sep-2024
  • (2024)Evaluating Contextually Personalized Programming Exercises Created with Generative AIProceedings of the 2024 ACM Conference on International Computing Education Research - Volume 110.1145/3632620.3671103(95-113)Online publication date: 12-Aug-2024
  • (2024)Comparing the Security of Three Proctoring Regimens for Bring-Your-Own-Device ExamsProceedings of the 55th ACM Technical Symposium on Computer Science Education V. 110.1145/3626252.3630809(429-435)Online publication date: 7-Mar-2024
  • (2024)Research and Design of Automatic Questioning System Based on Question Generation2024 12th International Conference on Information and Education Technology (ICIET)10.1109/ICIET60671.2024.10542827(182-186)Online publication date: 18-Mar-2024
  • (2024)ICBench: Benchmarking Knowledge Mastery in Introductory Computer Science EducationBenchmarking, Measuring, and Optimizing10.1007/978-981-97-0316-6_1(1-17)Online publication date: 14-Feb-2024
  • (2024)GAMAI, an AI-Powered Programming Exercise Gamifier ToolArtificial Intelligence in Education. Posters and Late Breaking Results, Workshops and Tutorials, Industry and Innovation Tracks, Practitioners, Doctoral Consortium and Blue Sky10.1007/978-3-031-64315-6_47(485-493)Online publication date: 2-Jul-2024
  • (2023)An LOD-based AIG Approach to Automatically Generating Object-Oriented Programming ProblemsJournal of Computing Sciences in Colleges10.5555/3636988.363702339:3(295-305)Online publication date: 1-Oct-2023
  • (2023)TAnnotator: Towards Annotating Programming E-textbooks with Facts and ExamplesSmart Learning Environments10.1186/s40561-023-00228-y10:1Online publication date: 25-Jan-2023
  • (2023)Mask and Cloze: Automatic Open Cloze Question Generation Using a Masked Language ModelIEEE Access10.1109/ACCESS.2023.323900511(9835-9850)Online publication date: 2023
  • (2023)Generating Pedagogical Questions to Help Students LearnAugmented Intelligence and Intelligent Tutoring Systems10.1007/978-3-031-32883-1_17(195-208)Online publication date: 22-May-2023
  • Show More Cited By

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media