No Tests Required: Comparing Traditional and Dynamic Predictors of Programming Success

Watson, Christopher; Li, Frederick W.B.; Godwin, Jamie L.

Abstract

Research over the past fifty years into predictors of programming performance has yielded little improvement in the identification of at-risk students. This is possibly because research to date is based upon using static tests, which fail to reflect changes in a student's learning progress over time. In this paper, the effectiveness of 38 traditional predictors of programming performance are compared to 12 new data-driven predictors, that are based upon analyzing directly logged data, describing the programming behavior of students. Whilst few strong correlations were found between the traditional predictors and performance, an abundance of strong significant correlations based upon programming behavior were found. A model based upon two of these metrics (Watwin score and percentage of lab time spent resolving errors) could explain 56.3% of the variance in coursework results. The implication of this study is that a student's programming behavior is one of the strongest indicators of their performance, and future work should continue to explore such predictors in different teaching contexts.

Citation

Watson, C., Li, F. W., & Godwin, J. L. (2014, December). No Tests Required: Comparing Traditional and Dynamic Predictors of Programming Success. Presented at 45th ACM Technical Symposium on Computer Science Education (SIGCSE '14), Atlanta GA

• Presentation Conference Type: Conference Paper (published)
• Conference Name: 45th ACM Technical Symposium on Computer Science Education (SIGCSE '14)
• Acceptance Date: Nov 30, 2014
• Publication Date: Jan 1, 2014
• Deposit Date: Sep 6, 2014
• Publicly Available Date: Jul 13, 2016
• Publisher: Association for Computing Machinery (ACM)
• Pages: 469-474
• Book Title: Proceedings of the 45th ACM Technical Symposium on Computer Science Education.
• DOI: https://doi.org/10.1145/2538862.2538930
• Public URL: https://durham-repository.worktribe.com/output/1155073
• Publisher URL: http://doi.acm.org/10.1145/2538862.2538930