skip to main content
10.1145/2702123.2702424acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article

Providing Adaptive Support in an Interactive Simulation for Learning: An Experimental Evaluation

Published: 18 April 2015 Publication History

Abstract

Recent rise of Massive Open Online Courses (MOOCs) with unlimited participants, makes employing learning tools such as interactive simulations all but inevitable. Interactive simulations give students the opportunity to experiment with concrete examples and develop better understanding of concepts they have learned. However, some students do not learn well from this relatively unstructured form of interaction, suggesting the provision of adaptive support as a way to address this issue. This paper presents a formal evaluation of providing support to facilitate open exploration. We describe the process of designing an intervention delivery mechanism for adding adaptive support to an exploratory interactive simulation. The experimental evaluation of the adaptive version of the simulation indicates that the adaptive support provided to students significantly improved their learning performance. Quantitative and qualitative evaluations of users' acceptance of the system are generally positive but pinpoint areas for improvement.

Supplementary Material

suppl.mov (pn1574.mp4)
Supplemental video

References

[1]
Amershi, S., Carenini, G., Conati, C., Mackworth, A.K., and Poole, D. Pedagogy and usability in interactive algorithm visualizations: Designing and evaluating CIspace. Interacting with Computers 20, 1 (2008), 64--96.
[2]
Borek, A., McLaren, B.M., Karabinos, M., and Yaron, D. How Much Assistance Is Helpful to Students in Discovery Learning? In U. Cress, V. Dimitrova and M. Specht, eds., Learning in the Synergy of Multiple Disciplines. Springer Berlin Heidelberg, 2009, 391--404.
[3]
Cohen, J. Statistical Power Analysis for the Behavioral Sciences. Lawrence Erlbaum, 1988.
[4]
Cohen, J., West, S.G., Aiken, L.S., and Cohen, P. Applied Multiple Regression/Correlation Analysis for the Behavioral Sciences. Lawrence Erlbaum, Mahwah, N.J, 2002.
[5]
Gluck, J., Bunt, A., and McGrenere, J. Matching attentional draw with utility in interruption. In Proc. CHI '07, (2007), 41.
[6]
Gobert, J.D., Montalvo, O., Toto, E., Sao Pedro, M., and Baker, R.S.J. d. The Science Assistments Project: Scaffolding Scientific Inquiry Skills. In Proc. Intelligent Tutoring Systems, (2010), 445--445.
[7]
Hussain, T.S., Roberts, B., Menaker, E.S., et al. Designing and developing effective training games for the US Navy. The Interservice/Industry Training, Simulation & Education Conference (I/ITSEC), NTSA (2009).
[8]
Jackson, G.T., Olney, A., Graesser, A.C., and Kim, H.J. AutoTutor 3-D Simulations: Analyzing user's actions and learning trends. In Proc. the 28th annual meeting of the Cognitive Science Society, (2006), 1557--1562.
[9]
De Jong, T. Technological Advances in Inquiry Learning. Science 312, 5773 (2006), 532--533.
[10]
Van Joolingen, W.R., De Jong, T., and Dimitrakopoulou, A. Issues in computer supported inquiry learning in science. J. of Computer Assisted Learning 23, 2 (2007), 111--119.
[11]
Kardan, S. Data Mining for Adding Adaptive Interventions to Exploratory and Open-Ended Environments. In Proc. UMAP 2012, Springer Berlin / Heidelberg, 2012, 365--368.
[12]
Kardan, S. and Conati, C. A Framework for Capturing Distinguishing User Interaction Behaviours in Novel Interfaces. In Proc. the 4th Int. Conf. on Educational Data Mining, (2011), 159--168.
[13]
Kardan, S. and Conati, C. Exploring Gaze Data for Determining User Learning with an Interactive Simulation. In Proc. UMAP 2012, Springer Berlin / Heidelberg, 2012, 126--138.
[14]
Lenth, R.V. Java Applets for Power and Sample Size. 2006. http://www.stat.uiowa.edu/~rlenth/Power
[15]
Maloy, R.W., Verock-O'Loughlin, R.-E.A., Edwards, S.A., and Woolf, B.P. Transforming Learning with New Technologies. Pearson, Boston, 2013.
[16]
Mavrikis, M., Gutierrez-Santos, S., Geraniou, E., and Noss, R. Design requirements, student perception indicators and validation metrics for intelligent exploratory learning environments. Personal and Ubiquitous Computing, 1--16.
[17]
O'brien, R.M. A Caution Regarding Rules of Thumb for Variance Inflation Factors. Quality & Quantity 41, 5 (2007), 673--690.
[18]
Perkins, K., Adams, W., Dubson, M., et al. PhET: Interactive Simulations for Teaching and Learning Physics. The Physics Teacher 44, 1 (2006), 18--23.
[19]
Ploetzner, R., Lippitsch, S., Galmbacher, M., Heuer, D., and Scherrer, S. Students' difficulties in learning from dynamic visualisations and how they may be overcome. Computers in Human Behavior 25, 1 (2009), 56--65.
[20]
Poole, D.L. and Mackworth, A.K. Artificial Intelligence: Foundations of Computational Agents. Cambridge University Press, 2010.
[21]
Salajan, F.D., Perschbacher, S., Cash, M., Talwar, R., ElBadrawy, W., and Mount, G.J. Learning with web-based interactive objects: An investigation into student perceptions of effectiveness. Computers & Education 53, 3 (2009), 632--643.
[22]
Stamper, J., Eagle, M., Barnes, T., and Croy, M. Experimental Evaluation of Automatic Hint Generation for a Logic Tutor. Int. J. Artif. Intell. Ed.22, 1 (2013), 3--17.
[23]
Ting, C.Y., Zadeh, M., and Chong, Y.K. A decisiontheoretic approach to scientific inquiry exploratory learning environment. In Proc. Intelligent Tutoring Systems, (2006), 85--94.
[24]
VanLehn, K. The Behavior of Tutoring Systems. Int. J. Artif. Intell. Ed. 16, 3 (2006), 227--265.
[25]
Westerfield, G., Mitrovic, A., and Billinghurst, M. Intelligent Augmented Reality Training for Assembly Tasks. In Proc. Artificial Intelligence in Education,(2013), 542--551.

Cited By

View all
  • (2024)AI-Driven Personalization to Support Human-AI CollaborationCompanion Proceedings of the 16th ACM SIGCHI Symposium on Engineering Interactive Computing Systems10.1145/3660515.3661324(5-6)Online publication date: 24-Jun-2024
  • (2024)An Intelligent Pedagogical Agent for In-The-Wild Interaction in an Open-Ended Learning Environment for Computational ThinkingProceedings of the 24th ACM International Conference on Intelligent Virtual Agents10.1145/3652988.3673948(1-9)Online publication date: 16-Sep-2024
  • (2024)Does Difficulty even Matter? Investigating Difficulty Adjustment and Practice Behavior in an Open-Ended Learning TaskProceedings of the 14th Learning Analytics and Knowledge Conference10.1145/3636555.3636876(253-262)Online publication date: 18-Mar-2024
  • Show More Cited By

Index Terms

  1. Providing Adaptive Support in an Interactive Simulation for Learning: An Experimental Evaluation

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    CHI '15: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems
    April 2015
    4290 pages
    ISBN:9781450331456
    DOI:10.1145/2702123
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 18 April 2015

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. intelligent learning environments
    2. interactive simulations
    3. user modeling
    4. user-adaptive interaction

    Qualifiers

    • Research-article

    Funding Sources

    • Institute for Computing Information and Cognitive Systems (ICICS) University of British Columbia

    Conference

    CHI '15
    Sponsor:
    CHI '15: CHI Conference on Human Factors in Computing Systems
    April 18 - 23, 2015
    Seoul, Republic of Korea

    Acceptance Rates

    CHI '15 Paper Acceptance Rate 486 of 2,120 submissions, 23%;
    Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

    Upcoming Conference

    CHI 2025
    ACM CHI Conference on Human Factors in Computing Systems
    April 26 - May 1, 2025
    Yokohama , Japan

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)55
    • Downloads (Last 6 weeks)8
    Reflects downloads up to 22 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)AI-Driven Personalization to Support Human-AI CollaborationCompanion Proceedings of the 16th ACM SIGCHI Symposium on Engineering Interactive Computing Systems10.1145/3660515.3661324(5-6)Online publication date: 24-Jun-2024
    • (2024)An Intelligent Pedagogical Agent for In-The-Wild Interaction in an Open-Ended Learning Environment for Computational ThinkingProceedings of the 24th ACM International Conference on Intelligent Virtual Agents10.1145/3652988.3673948(1-9)Online publication date: 16-Sep-2024
    • (2024)Does Difficulty even Matter? Investigating Difficulty Adjustment and Practice Behavior in an Open-Ended Learning TaskProceedings of the 14th Learning Analytics and Knowledge Conference10.1145/3636555.3636876(253-262)Online publication date: 18-Mar-2024
    • (2024)Initial results on personalizing explanations of AI hints in an ITSProceedings of the 32nd ACM Conference on User Modeling, Adaptation and Personalization10.1145/3627043.3659566(244-248)Online publication date: 22-Jun-2024
    • (2023)Do students like us because we teach well? The popularity of high school mathematics teachersAsian Journal for Mathematics Education10.1177/275272632211429061:4(383-407)Online publication date: 17-Jan-2023
    • (2023)The Impact of Intelligent Pedagogical Agents’ Interventions on Student Behavior and Performance in Open-Ended Game Design EnvironmentsACM Transactions on Interactive Intelligent Systems10.1145/357852313:3(1-29)Online publication date: 11-Sep-2023
    • (2023)Classification of Alzheimer's Disease with Deep Learning on Eye-tracking DataProceedings of the 25th International Conference on Multimodal Interaction10.1145/3577190.3614149(104-113)Online publication date: 9-Oct-2023
    • (2023)The L@St Eight Years: A Review of Papers and Authors at Learning @ ScaleProceedings of the Tenth ACM Conference on Learning @ Scale10.1145/3573051.3596192(383-387)Online publication date: 20-Jul-2023
    • (2023)Bildung durch Künstliche Intelligenz ermöglichen. Ein Beitrag aus bildungstheoretischer PerspektiveKünstliche Intelligenz in der Bildung10.1007/978-3-658-40079-8_1(3-29)Online publication date: 2-Nov-2023
    • (2023)How to Repeat Hints: Improving AI-Driven Help in Open-Ended Learning EnvironmentsArtificial Intelligence in Education10.1007/978-3-031-36272-9_68(760-766)Online publication date: 26-Jun-2023
    • Show More Cited By

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media