Skip to main content

Research Repository

Advanced Search

Serendipitous Gains of Explaining a Classifier - Artificial versus Human Performance and Annotator Support in an Urgent Instructor-Intervention Model for MOOCs

Alrajhi, Laila; Pereira, Filipe Dwan; Cristea, Alexandra I.; Alamri, Ahmed

Serendipitous Gains of Explaining a Classifier - Artificial versus Human Performance and Annotator Support in an Urgent Instructor-Intervention Model for MOOCs Thumbnail


Authors

Laila Alrajhi laila.m.alrajhi@durham.ac.uk
PGR Student Doctor of Philosophy

Filipe Dwan Pereira



Abstract

Determining when instructor intervention is needed, based on learners’ comments and their urgency in massive open online course (MOOC) environments, is a known challenge. To solve this challenge, prior art used autonomous machine learning (ML) models. These models are described as having a "black-box" nature, and their output is incomprehensible to humans. This paper shows how to apply eXplainable Artificial Intelligence (XAI) techniques to interpret a MOOC intervention model for urgent comments detection. As comments were selected from the MOOC course and annotated using human experts, we additionally study the confidence between annotators (annotator agreement confidence), versus an estimate of the class score of making a decision via ML, to support intervention decision. Serendipitously, we show, for the first time, that XAI can be further used to support annotators creating high-quality, gold standard datasets for urgent intervention.

Citation

Alrajhi, L., Pereira, F. D., Cristea, A. I., & Alamri, A. (2023, September). Serendipitous Gains of Explaining a Classifier - Artificial versus Human Performance and Annotator Support in an Urgent Instructor-Intervention Model for MOOCs. Paper presented at HT '23: 34th ACM Conference on Hypertext and Social Media, Rome Italy

Presentation Conference Type Conference Paper (unpublished)
Conference Name HT '23: 34th ACM Conference on Hypertext and Social Media
Start Date Sep 4, 2023
End Date Sep 8, 2023
Online Publication Date Sep 21, 2023
Publication Date Sep 4, 2023
Deposit Date Nov 7, 2023
Publicly Available Date Nov 7, 2023
DOI https://doi.org/10.1145/3603607.3613480
Public URL https://durham-repository.worktribe.com/output/1899258

Files





You might also like



Downloadable Citations