Philip J. Guo
ABSTRACT
Videos are a widely-used kind of resource for online learning. This paper presents an empirical study of how video production decisions affect student engagement in online educational videos. To our knowledge, ours is the largest-scale study of video engagement to date, using data from 6.9 million video watching sessions across four courses on the edX MOOC platform. We measure engagement by how long students are watching each video, and whether they attempt to answer post-video assessment problems.
Our main findings are that shorter videos are much more engaging, that informal talking-head videos are more engaging, that Khan-style tablet drawings are more engaging, that even high-quality pre-recorded classroom lectures might not make for engaging online videos, and that students engage differently with lecture and tutorial videos.
Based upon these quantitative findings and qualitative insights from interviews with edX staff, we developed a set of recommendations to help instructors and video producers take better advantage of the online video format. Finally, to enable researchers to reproduce and build upon our findings, we have made our anonymized video watching data set and analysis scripts public. To our knowledge, ours is one of the first public data sets on MOOC resource usage.
- Khan Academy YouTube Channel. http://www.youtube.com/user/khanacademy/about.Google Scholar
- Breslow, L., Pritchard, D. E., DeBoer, J., Stump, G. S., Ho, A. D., and Seaton, D. T. Studying learning in the worldwide classroom: Research into edX's first MOOC. Research and Practice in Assessment 8 (Summer 2013).Google Scholar
- Coetzee, D., Fox, A., Hearst, M. A., and Hartmann, B. Should Your MOOC Forum Use a Reputation System? CSCW'14, ACM (New York, NY, USA, 2014). Google ScholarDigital Library
- Cross, A., Bayyapunedi, M., Cutrell, E., Agarwal, A., and Thies, W. TypeRighting: Combining the Benefits of Handwriting and Typeface in Online Educational Videos. CHI'13, ACM (New York, NY, USA, 2013). Google ScholarDigital Library
- Google. How Visits are calculated in Analytics. https://support.google.com/analytics/answer/2731565?hl=en.Google Scholar
- Google. YouTube Analytics. http://www.youtube.com/yt/playbook/yt-analytics.html#details.Google Scholar
- Haber, J. xMOOC vs. cMOOC. http://degreeoffreedom.org/xmooc-vs-cmooc/, 2013.Google Scholar
- Ilioudi, C., Giannakos, M. N., and Chorianopoulos, K. Investigating Differences among the Commonly Used Video Lecture Styles. In Proceedings of the Workshop on Analytics on Video-based Learning, WAVe'13 (2013).Google Scholar
- Kizilcec, R. F., Piech, C., and Schneider, E. Deconstructing disengagement: analyzing learner subpopulations in massive open online courses. In Proceedings of the Third International Conference on Learning Analytics and Knowledge, LAK '13, ACM (New York, NY, USA, 2013), 170--179. Google ScholarDigital Library
- Levasseur, D. G., and Sawyer, J. K. Pedagogy Meets PowerPoint: A Research Review of the Effects of Computer-Generated Slides in the Classroom. Review of Communication 6, 1 (2006), 101--123.Google Scholar
- Mayer, R. E. Multimedia Learning. Cambridge University Press, 2001. Google ScholarDigital Library
- Roam, D. The Back of the Napkin (Expanded Edition): Solving Problems and Selling Ideas with Pictures. Portfolio Hardcover, 2009.Google Scholar
- Seaton, D. T., Bergner, Y., Chuang, I., Mitros, P., and Pritchard, D. E. Who does what in a massive open online course? Communications of the ACM (2013).Google Scholar
- Wade, A., and Koutoumanou, E. Non-parametric tests: Confidence intervals for a single median. https://epilab.ich.ucl.ac.uk/coursematerial/statistics/non_parametric/confidence_interval.html.Google Scholar
- Williams, J. R. Guidelines for the use of multimedia in instruction. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 42, 20 (1998), 1447--1451.Google ScholarCross Ref
- Wistia. Does length matter? It does for video! http://wistia.com/blog/does-length-matter-it-does-for-video, Sept. 2013.Google Scholar
Index Terms
- How video production affects student engagement: an empirical study of MOOC videos
Recommendations
Criteria for Video Engagement in a Biology MOOC
L@S '17: Proceedings of the Fourth (2017) ACM Conference on Learning @ ScaleWe have designed a Molecular Biology massive open online course (MOOC) for a global audience. Among the learning aids offered are two types of short video segments: lecture videos (delivered unscripted by a professor) and deep dives (fully scripted and ...
Understanding the Effect of In-Video Prompting on Learners and Instructors
CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing SystemsOnline instructional videos are ubiquitous, but it is difficult for instructors to gauge learners' experience and their level of comprehension or confusion regarding the lecture video. Moreover, learners watching the videos may become disengaged or fail ...
Blended Learning in Indian Colleges with Massively Empowered Classroom
L@S '15: Proceedings of the Second (2015) ACM Conference on Learning @ ScaleStudents in the developing world are frequently cited as being among the most important beneficiaries of online education initiatives such as massive open online courses (MOOCs). While some predict that online classrooms will replace physical classrooms,...
Comments