Skip to main content

Research Repository

Advanced Search

Prediction of default probability by using statistical models for rare events

Ogundimu, Emmanuel O.

Prediction of default probability by using statistical models for rare events Thumbnail


Authors



Abstract

Prediction models in credit scoring usually involve the use of data sets with highly imbalanced distributions of the event of interest (default). Logistic regression, which is widely used to estimate the probability of default, PD, often suffers from the problem of separation when the event of interest is rare and consequently poor predictive performance of the minority class in small samples. A common solution is to discard majority class examples, to duplicate minority class examples or to use a combination of both to balance the data. These methods may overfit data. It is unclear how penalized regression models such as Firth's estimator, which reduces bias and mean-square error relative to classical logistic regression, performs in modelling PD. We review some methods for class imbalanced data and compare them in a simulation study using the Taiwan credit card data. We emphasize the effect of events per variable for developing an accurate model—an often neglected concept in PD-modelling. The data balancing techniques that are considered are the random oversampling examples and synthetic minority oversampling technique methods. The results indicate that the synthetic minority oversampling technique improved predictive accuracy of PD regardless of sample size. Among the penalized regression models that are analysed, the log-F prior and ridge regression methods are preferred.

Citation

Ogundimu, E. O. (2019). Prediction of default probability by using statistical models for rare events. Journal of the Royal Statistical Society: Series A, 182(4), 1143-1162. https://doi.org/10.1111/rssa.12467

Journal Article Type Article
Online Publication Date Apr 22, 2019
Publication Date 2019-10
Deposit Date Oct 11, 2020
Publicly Available Date Oct 15, 2021
Journal Journal of the Royal Statistical Society: Series A (Statistics in Society)
Print ISSN 0964-1998
Electronic ISSN 1467-985X
Publisher Royal Statistical Society
Peer Reviewed Peer Reviewed
Volume 182
Issue 4
Pages 1143-1162
DOI https://doi.org/10.1111/rssa.12467

Files

Accepted Journal Article (30.6 Mb)
PDF

Copyright Statement
This is the peer reviewed version of the following article: Ogundimu, Emmanuel O. (2019). Prediction of default probability by using statistical models for rare events. Journal of the Royal Statistical Society: Series A (Statistics in Society) 182(4): 1143-1162., which has been published in final form at https://doi.org/10.1111/rssa.12467. This article may be used for non-commercial purposes in accordance with Wiley Terms and Conditions for Use of Self-Archived Versions.





You might also like



Downloadable Citations