The engage taxonomy: SDT-based measurable engagement indicators for MOOCs and their evaluation

Cristea, Alexandra I.; Alamri, Ahmed; Alshehri, Mohammed; Dwan Pereira, Filipe; Toda, Armando M.; Harada T. de Oliveira, Elaine; Stewart, Craig

Abstract

Massive Online Open Course (MOOC) platforms are considered a distinctive way to deliver a modern educational experience, open to a worldwide public. However, student engagement in MOOCs is a less explored area, although it is known that MOOCs suffer from one of the highest dropout rates within learning environments in general, and in e-learning in particular. A special challenge in this area is finding early, measurable indicators of engagement. This paper tackles this issue with a unique blend of data analytics and NLP and machine learning techniques together with a solid foundation in psychological theories. Importantly, we show for the first time how Self-Determination Theory (SDT) can be mapped onto concrete features extracted from tracking student behaviour on MOOCs. We map the dimensions of Autonomy, Relatedness and Competence, leading to methods to characterise engaged and disengaged MOOC student behaviours, and exploring what triggers and promotes MOOC students’ interest and engagement. The paper further contributes by building the Engage Taxonomy, the first taxonomy of MOOC engagement tracking parameters, mapped over 4 engagement theories: SDT, Drive, ET, Process of Engagement. Moreover, we define and analyse students’ engagement tracking, with a larger than usual body of content (6 MOOC courses from two different universities with 26 runs spanning between 2013 and 2018) and students (initially around 218.235). Importantly, the paper also serves as the first large-scale evaluation of the SDT theory itself, providing a blueprint for large-scale theory evaluation. It also provides for the first-time metrics for measurable engagement in MOOCs, including specific measures for Autonomy, Relatedness and Competence; it evaluates these based on existing (and expanded) measures of success in MOOCs: Completion rate, Correct Answer ratio and Reply ratio. In addition, to further illustrate the use of the proposed SDT metrics, this study is the first to use SDT constructs extracted from the first week, to predict active and non-active students in the following week.

Citation

Cristea, A. I., Alamri, A., Alshehri, M., Dwan Pereira, F., Toda, A. M., Harada T. de Oliveira, E., & Stewart, C. (2024). The engage taxonomy: SDT-based measurable engagement indicators for MOOCs and their evaluation. User Modeling and User-Adapted Interaction, 34(2), 323-374. https://doi.org/10.1007/s11257-023-09374-x

• Journal Article Type: Article
• Acceptance Date: May 18, 2023
• Online Publication Date: Aug 12, 2023
• Publication Date: Apr 1, 2024
• Deposit Date: Feb 15, 2023
• Publicly Available Date: Aug 14, 2023
• Journal: User Modeling and User-Adapted Interaction
• Print ISSN: 0924-1868
• Electronic ISSN: 1573-1391
• Publisher: Springer
• Peer Reviewed: Peer Reviewed
• Volume: 34
• Issue: 2
• Pages: 323-374
• DOI: https://doi.org/10.1007/s11257-023-09374-x
• Keywords: MOOCs, Student engagement, Learning analytics, SDT
• Public URL: https://durham-repository.worktribe.com/output/1180972