Toward Supporting CS1 Instructors and Learners With Fine-Grained Topic Detection in Online Judges

Pereira, Filipe Dwan; Fonseca, Samuel C.; Wiktor, Sandra; Oliveira, David B.F.; Cristea, Alexandra I.; Benedict, Aileen; Fallahian, Mohammadali; Dorodchi, Mohsen; Carvalho, Leandro S.G.; Mello, Rafael Ferreira; Oliveira, Elaine H.T.

Abstract

Online judges (OJ) are a popular tool to support programming learning. However, one major issue with OJs is that problems are often put together without any associated meta-information that could, for example, be used to help classify problems. This meta-information could be extremely valuable to help users quickly find what types of problems they need most. To face this problem, several OJ administrators have recently begun manually annotating the topics of problems based on computer science-related subjects, such as dynamic programming, graphs, and data structures. Initially, these topics were used to support programming competitions and experienced learners. However, with OJs being increasingly used to support CS1 classes, such topic annotation needs to be extended to suit CS1 learners and instructors. In this work, for the first time, to the best of our knowledge, we propose and validate a predictive model that can automatically detect fine-grained topics of problems based on the CS1 syllabus. After experimenting with many shallow and deep learning models with different word representations based on cutting-edge NLP techniques, our best model is a CNN, achieving an F1-score of 88.9%. We then present how our model can be used for various applications, including (i) facilitating the search process of problems for CS1 learners and instructors and (ii) how it can be integrated into a system to recommend problems in OJs.

Citation

Pereira, F. D., Fonseca, S. C., Wiktor, S., Oliveira, D. B., Cristea, A. I., Benedict, A., Fallahian, M., Dorodchi, M., Carvalho, L. S., Mello, R. F., & Oliveira, E. H. (2023). Toward Supporting CS1 Instructors and Learners With Fine-Grained Topic Detection in Online Judges. IEEE Access, 11, https://doi.org/10.1109/access.2023.3247189

• Journal Article Type: Article
• Acceptance Date: Feb 11, 2023
• Online Publication Date: Feb 22, 2023
• Publication Date: 2023
• Deposit Date: May 22, 2023
• Publicly Available Date: May 22, 2023
• Journal: IEEE Access
• Publisher: Institute of Electrical and Electronics Engineers
• Peer Reviewed: Peer Reviewed
• Volume: 11
• DOI: https://doi.org/10.1109/access.2023.3247189
• Public URL: https://durham-repository.worktribe.com/output/1173389