Classifying Learner Behavior from High Frequency Touchscreen Data Using Recurrent Neural Networks

ABSTRACT

Sensor stream data, particularly those collected at the millisecond of granularity, have been notoriously difficult to leverage classifiable signal out of. Adding to the challenge is the limited domain knowledge that exists at these biological sensor levels of interaction that prohibits a comprehensive manual feature engineering approach to classification of those streams. In this paper, we attempt to enhance the assessment capability of a touchscreen based ratio tutoring system by using Recurrent Neural Networks (RNNs) to predict the strategy being demonstrated by students from their 60hz data streams. We hypothesize that the ability of neural networks to learn representations automatically, instead of relying on human feature engineering, may benefit this classification task. Our RNN and baseline models were trained and cross-validated at several levels on historical data which had been human coded with the task strategy believed to be exhibited by the learner. Our RNN approach to this historically difficult high frequency data classification task moderately advances performance above baselines and we discuss what implication this level of assessment performance has on enabling greater adaptive supports in the tutoring system.

References

1. Ahsan Abdullah, Mohammad Adil, Leah Rosenbaum, Miranda Clemmons, Mansi Shah, Dor Abrahamson, and Michael Neff. 2017. Pedagogical Agents to Support Embodied, Discovery-Based Learning Intelligent Virtual Agents, Jonas Beskow, Christopher Peters, Ginevra Castellano, Carol O'Sullivan, Iolanda Leite, and Stefan Kopp (Eds.). Springer International Publishing, Cham, 1--14.
2. Dor Abrahamson and Arthur Bakker. 2016. Making sense of movement in embodied design for mathematics learning. Cognitive Research: Principles and Implications Vol. 1, 1 (12. 2016), 1--13.
3. Ivon Arroyo, David G Cooper, Winslow Burleson, Beverly Park Woolf, Kasia Muldner, and Robert Christopherson. 2009. Emotion Sensors Go To School.. In AIED, Vol. Vol. 200. 17--24.
4. Paulo Blikstein. 2013. Multimodal learning analytics. In Proceedings of the third international conference on learning analytics and knowledge. ACM, 102--106.
5. Anthony F Botelho, Ryan S Baker, and Neil T Heffernan. 2017. Improving Sensor-Free Affect Detection Using Deep Learning International Conference on Artificial Intelligence in Education. Springer, 40--51.
6. Franccois Chollet. 2015. keras. https://github.com/fchollet/keras.
7. Abrahamson D., Lee R. G., Negrete A. G., and Gutiérrez J. F.. 2014. Coordinating visualizations of polysemous action: Values added for grounding proportion. In ZDM Mathematics Education, Visualization as an epistemological learning tool {Special issue}, F. Rivera, H. Steinbring, and A. Arcavi (Eds.). Vol. Vol. 46. 79--93.
8. Abrahamson D., Shayan S., Bakker A., and Van der Schaaf M. F.. 2016. Eye-tracking Piaget: Capturing the emergence of attentional anchors in the coordination of proportional motor action. Human Development Vol. 58, 4--5 (2016), 218--244.
9. Robin Devooght and Hugues Bersini. 2017. Long and Short-Term Recommendations with Recurrent Neural Networks Proceedings of the 25th Conference on User Modeling, Adaptation and Personalization (UMAP '17). ACM, New York, NY, USA, 13--21.
10. Klaus Greff, Rupesh K Srivastava, Jan Koutn'ık, Bas R Steunebrink, and Jürgen Schmidhuber. 2017. LSTM: A search space odyssey. IEEE transactions on neural networks and learning systems Vol. 28, 10 (2017), 2222--2232.
11. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation Vol. 9, 8 (1997), 1735--1780.
12. Mark Howison, Dragan Trninic, Daniel Reinholz, and Dor Abrahamson. 2011. The Mathematical Imagery Trainer: From Embodied Interaction to Conceptual Learning Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11). ACM, New York, NY, USA, 1989--1998.
13. Stephen Hutt, Caitlin Mills, Nigel Bosch, Kristina Krasich, James Brockmole, and Sidney D'Mello. 2017. "Out of the Fr-Eye-ing Pan": Towards Gaze-Based Models of Attention during Learning with Technology in the Classroom. (07. 2017), 94--103.
14. Lamon S. J.. 2007. Rational numbers and proportional reasoning: Toward a theoretical framework. In Second handbook of research on mathematics teaching and learning, F. Lester (Ed.). Charlotte, NC: Information Age Publishing, 629--668.
15. Dietmar Jannach, Malte Ludewig, and Lukas Lerche. 2017. Session-based item recommendation in e-commerce: on short-term intents, reminders, trends and discounts. User Modeling and User-Adapted Interaction (2017), 1--42.
16. Antoine Lefebvre-Brossard, Alexandre Spaeth, and Michel C. Desmarais. 2017. Encoding User As More Than the Sum of Their Parts: Recurrent Neural Networks and Word Embedding for People-to-people Recommendation Proceedings of the 25th Conference on User Modeling, Adaptation and Personalization (UMAP '17). ACM, New York, NY, USA, 298--302.
17. LR Medsker and LC Jain. 2001. Recurrent neural networks. Design and Applications Vol. 5 (2001).
18. Daniel Neil, Michael Pfeiffer, and Shih-Chi Liu. 2016. Phased LSTM: Accelerating Recurrent Network Training for Long or Event-based Sequences. In Advances in Neural Information Processing Systems 29, D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett (Eds.). Curran Associates, Inc, 3882--3890.
19. Chris Piech, Jonathan Bassen, Jonathan Huang, Surya Ganguli, Mehran Sahami, Leonidas J Guibas, and Jascha Sohl-Dickstein. 2015. Deep knowledge tracing. In Advances in Neural Information Processing Systems. 505--513.
20. George E. Raptis, Christina Katsini, Marios Belk, Christos Fidas, George Samaras, and Nikolaos Avouris. 2017. Using Eye Gaze Data and Visual Activities to Infer Human Cognitive Styles: Method and Feasibility Studies. In Proceedings of the 25th Conference on User Modeling, Adaptation and Personalization (UMAP '17). ACM, New York, NY, USA, 164--173.
21. MA Rau, HE Bowman, and JW Moore. 2016. Intelligent technology-support for collaborative connection-making among multiple visual representations in chemistry. Structure-Function Relationships in the Gas-Sensing Heme-Dependent Transcription Factors RcoM and DNR Vol. 1001 (2016), 178.
22. Martina A Rau and Zachary A Pardos. 2016. Adding eye-tracking AOI data to models of representation skills does not improve prediction accuracy.. In EDM. 622--623.
23. Alessandro Suglia, Claudio Greco, Cataldo Musto, Marco de Gemmis, Pasquale Lops, and Giovanni Semeraro. 2017. A Deep Architecture for Content-based Recommendations Exploiting Recurrent Neural Networks. In Proceedings of the 25th Conference on User Modeling, Adaptation and Personalization (UMAP '17). ACM, New York, NY, USA, 202--211.
24. Steven Tang, Joshua C Peterson, and Zachary A Pardos. 2017. Modelling Student Behavior using Granular Large Scale Action Data from a MOOC. The Handbook of Learning Analytics (2017), 223--233.
25. Theano Development Team. 2016. Theano: A Python framework for fast computation of mathematical expressions. arXiv e-prints Vol. abs/1605.02688 (May. 2016). http://arxiv.org/abs/1605.02688
26. Boyer T.W. and Levine S.C.. 2015. Prompting children to reason proportionally: Processing discrete units as continuous amounts. Developmental psychology Vol. 51, 5 (2015), 615--620.

Index Terms

• Applied computing

  • Education

    • Computer-assisted instruction

• Computer systems organization

  • Architectures

    • Other architectures

      • Neural networks

• Computing methodologies

  • Machine learning

    • Learning paradigms

      • Supervised learning

• Hardware

  • Communication hardware, interfaces and storage

    • Sensors and actuators