The pulse of learning analytics understandings and expectations from the stakeholders

ABSTRACT

While there is currently much buzz about the new field of learning analytics [19] and the potential it holds for benefiting teaching and learning, the impression one currently gets is that there is also much uncertainty and hesitation, even extending to scepticism. A clear common understanding and vision for the domain has not yet formed among the educator and research community. To investigate this situation, we distributed a stakeholder survey in September 2011 to an international audience from different sectors of education. The findings provide some further insights into the current level of understanding and expectations toward learning analytics among stakeholders. The survey was scaffolded by a conceptual framework on learning analytics that was developed based on a recent literature review. It divides the domain of learning analytics into six critical dimensions. The preliminary survey among 156 educational practitioners and researchers mostly from the higher education sector reveals substantial uncertainties in learning analytics.

In this article, we first briefly introduce the learning analytics framework and its six domains that formed the backbone structure to our survey. Afterwards, we describe the method and key results of the learning analytics questionnaire and draw further conclusions for the field in research and practice. The article finishes with plans for future research on the questionnaire and the publication of both data and the questions for others to utilize.

References

1. Anrig, B., Browne, W., Gasson, M. (2008). The Role of algorithms in profiling. In M. Hildebrandt, S. Gutwirth, (Eds.) Profiling the European Citizen. Cross-Disciplinary Perspectives. Springer 2008.
2. Dawson, S., Heathcote, L. and Poole, G. (2010). Harnessing ICT potential: The adoption and analysis of ICT systems for enhancing the student learning experience, International Journal of Educational Management 24(2) pp. 116--128.
3. Drachsler, H., Bogers, T., Vuorikari, R., Verbert, K., Duval, E., Manouselis, N., Beham, G., Lindstaedt, S., Stern, H., Friedrich, M., Wolpers, M. (2010). Issues and Considerations regarding Sharable Data Sets for Recommender Systems in Technology Enhanced Learning. Elsevier Procedia Computer Science, 1, 2, pp. 2849--2858.
4. Govaerts, S., Verbert, K., Klerkx, J., Duval, E., (2010). Visualizing Activities for Self-reflection and Awareness, The 9th International Conference on Web-based Learning, ICWL, December 7--11, 2010, Shanghai University, China.
5. Greller, W. & Drachsler, H., (submitted). Turning Learning into Numbers -- A learning analytics Framework. International Journal of Educational Technology & Society.
6. Mödritscher, Felix; Krumay, Barbara; Kadlec, Edgar; Taferner, Wolfgang (2011): On reconstructing and analyzing personal learning environments of scientific artifacts. Proceedings of the Workshop on Data Sets for Technology Enhanced Learning (dataTEL 2011) at the STELLAR Alpine Rendez-Vous (ARV), La Clusaz, France, March 2011.
7. Norris, D., Baer, L., Leonard, J., Pugliese, L. and Lefrere, P. (2008). Action Analytics: Measuring and Improving Performance That Matters in Higher Education, EDUCAUSE Review 43(1). Retrieved October 20, 2011: http://www.educause.edu/EDUCAUSE+Review/EDUCAUSEReviewMagazineVolume43/ActionAnalyticsMeasuringandImp/162422.
8. Oblinger, D. G. and Campbell, J. P. (2007). Academic Analytics, EDUCAUSE White Paper. Retrieved October 20, 2011 from http://www.educause.edu/ir/library/pdf/PUB6101.pdf.
9. Reinhardt, W., Metzko, C., Drachsler, H., & Sloep, P. B. (2011). AWESOME: A widget-based dashboard for awareness-support in Research Networks. Proceedings of the PLE Conference 2011. July, 11--13, 2011, Southampton, UK.
10. Stoyanov, S., Hoogveld, B., Kirschner, P. A., (2010). Mapping Major Changes to Education and Training in 2025, in JRC Technical Note JRC59079., Publications Office of the European Union: Luxembourg.
11. Verbert, K., Drachsler, H., Manouselis, N., Wolpers, M., Vuorikari, R., & Duval, E. (2011). Dataset-driven Research for Improving Recommender Systems for Learning. 1st International Conference learning analytics & Knowledge. February, 27 - March, 1, 2011, Banff, Alberta, Canada.
12. Wild, F., Palmér, M., and Kalz, M. (2011). IJTEL: Special Issue on Mash-Up Personal Learning Environments. Special Issue of the International Journal of Technology Enhanced Learning. Inderscience publishers. March 2011.
13. Drachsler, H., Bogers, T., Vuorikari, R., Verbert, K., Duval, E., Manouselis, N., Beham, G., Lindstaedt, S., Stern, H., Friedrich, M., Wolpers, M. (2010). Issues and Considerations regarding Sharable Data Sets for Recommender Systems in Technology Enhanced Learning. Elsevier Procedia Computer Science, 1, 2, pp. 2849--2858.
14. Florian, B., Glahn, C., Drachsler, H., Specht, M., & Fabregat, R. (2011). Activity-based learner-models for Learner Monitoring and Recommendations in Moodle. In M. Wolpers, C. D. Kloos, & D. Gillet (Eds.), Towards Ubiquitous Learning (pp. 111--124). LNCS 6964; Heidelberg, Berlin: Springer.
15. Verbert, K., Manouselis, N., Drachsler, H., & Duval, E. (accepted). Dataset-driven Research to Support Learning and Knowledge Analytics. (Eds.) Siemens, George and Gašević, Dragan. Educational Technology & Society, xx (x), xx--xx.
16. Hildebrandt, M. (2006). 'Privacy and Identity', Privacy and the Criminal Law; E. Claes, A. Duff and S. Gutwirth (eds.), Antwerpen - Oxford: Intersentia 2006, p. 43--58. Available at: http://www.imbroglio.be/site/spip.php?article21, accessed 28 June 2011.
17. Stamper, J. C., Koedinger, K. R., Baker, R. S. J. D., Skogsholm, A., Leber, B., Rankin, J., & Demi, S. (2010) PSLC DataShop: A Data Analysis Service for the Learning Science Community. In Proceedings of Intelligent Tutoring Systems, LNCS, Volume 6095/2010:455--456.
18. Buckingham Shum, S. and Ferguson, R. (2011). Social Learning Analytics. Available as: Technical Report KMI-11-01, Knowledge Media Institute, The Open University, UK. http://kmi.open.ac.uk/publications/pdf/kmi-11-01.pdf
19. Siemens, G. (2010). What are Learning Analytics? Retrieved 9 August 2011 from http://www.elearnspace.org/blog/2010/08/25/what-are-learning-analytics/.
20. Romero, C., Ventura, S. Espejo, P. G., & Hervs, C. (2008). Data Mining Algorithms to Classify Students. In de Baker, R., Barnes, T. and Beck, J. (eds), Proceedings of the 1^st International Conference on Educational Data Mining, pages 8--17.
21. Reffay, C., & Chanier, T. (2003). How social network analysis can help to measure cohesion in collaborative distance learning. Computer Supported Collaborative Learning (pp. 1--10). Kluwer.
22. Mazza, R., & Milani, C. (2005). Exploring Usage Analysis in Learning Systems: Gaining Insights From Visualisations. In Workshop on Usage Analysis in Learning Systems at the 12th International Conference on AIED (6 pages). IOS Press, Amsterdam.
23. Govaerts, S., Verbert, K., Klerkx, J., & Duval, E. (2010). Visualizing Activities for Self-reflection and Awareness. In Luo, X., Spaniol, M., Wang, L., Li, Q., Nejdl, W. & Zhang, W. (eds), Proceedings of ICWL 2010, LNCS, Volume 6483/2010:91--100.
24. Butoianu, V., Vidal, P., Verbert, K., Duval, E. & Broisin, J. (2010). User Context and Personalized Learning: a Federation of Contextualized Attention Metadata. J. UCS, 16(16):2252--2271.
25. Butler, D. L., & Winne, P. H. (1995). Feedback and self-regulated learning: A theoretical synthesis. Review of Educational Research, 65, 245--281

Index Terms

• Applied computing

  • Education

    • Collaborative learning

    • Computer-assisted instruction

    • Computer-managed instruction

    • Distance learning

    • E-learning

  • Enterprise computing

  • Law, social and behavioral sciences

• Computing methodologies

  • Modeling and simulation

    • Simulation theory

      • Systems theory

• General and reference

  • Document types

    • Surveys and overviews

• Mathematics of computing

  • Information theory