Deep learning for early performance prediction of introductory programming students: a comparative and explanatory study

Pereira, Filipe Dwan; Fonseca, Samuel C.; Oliveira, Elaine H.T.; Oliveira, David B.F.; Cristea, Alexandra I.; Carvalho, Leandro S.G.

Abstract

Introductory programming may be complex for many students. Moreover, there is a high failure and dropout rate in these courses. A potential way to tackle this problem is to predict student performance at an early stage, as it facilitates human-AI collaboration towards prescriptive analytics, where the instructors/monitors will be told how to intervene and support students - where early intervention is crucial. However, the literature states that there is no reliable predictor yet for programming students’ performance, since even large-scale analysis of multiple features have resulted in only limited predictive power. Notice that Deep Learning (DL) can provide high-quality results for huge amount of data and complex problems. In this sense, we employed DL for early prediction of students’ performance using data collected in the very first two weeks from introductory programming courses offered for a total of 2058 students during 6 semesters (longitudinal study). We compared our results with the state-of-the-art, an Evolutionary Algorithm (EA) that automatic creates and optimises machine learning pipelines. Our DL model achieved an average accuracy of 82.5%, which is statistically superior to the model constructed and optimised by the EA (p-value << 0.05 even with Bonferroni correction). In addition, we also adapted the DL model in a stacking ensemble for continuous prediction purposes. As a result, our regression model explained ~62% of the final grade variance. In closing, we also provide results on the interpretation of our regression model to understand the leading factors of success and failure in introductory programming.

Citation

Pereira, F. D., Fonseca, S. C., Oliveira, E. H., Oliveira, D. B., Cristea, A. I., & Carvalho, L. S. (2020). Deep learning for early performance prediction of introductory programming students: a comparative and explanatory study. Revista Brasileira de Informática na Educação, 28, 723-749. https://doi.org/10.5753/rbie.2020.28.0.723

• Journal Article Type: Article
• Online Publication Date: Oct 12, 2020
• Publication Date: 2020
• Deposit Date: Oct 13, 2020
• Publicly Available Date: Oct 13, 2020
• Journal: Revista brasileira de informaÌ tica na educação
• Print ISSN: 1414-5685
• Electronic ISSN: 2317-6121
• Publisher: RBIE Editors
• Peer Reviewed: Peer Reviewed
• Volume: 28
• Pages: 723-749
• DOI: https://doi.org/10.5753/rbie.2020.28.0.723
• Public URL: https://durham-repository.worktribe.com/output/1290280