Imprecise weighted extensions of random forests for classification and regression

Utkin, L.V.; Kovalev, M.S.; Coolen, F.P.A.

Abstract

One of the main problems of using the random forests (RF) in classification and regression tasks is a lack of sufficient data which fall into certain leaves of trees in order to estimate the tree predicted values. To cope with this problem, robust imprecise classification and regression RF models, called the imprecise RF, are proposed. They are based on the following ideas. First, imprecision of the tree estimates is taken into account by means of imprecise statistical inference models and confidence interval models. Secondly, we introduce weights assigned to trees or to groups of trees, which are computed in order to correct the RF estimates under condition of imprecise tree predicted values. In fact, the weights can be regarded as a robust meta-learner controlling the imprecision of estimates. Special modifications of loss functions to compute optimal weights for the classification and regression tasks are proposed in order to simplify maximin optimization problems. As a result, simple linear and quadratic optimization problems are obtained, whose solution does not meet any difficulties. Various numerical examples with real datasets illustrate the proposed robust models and show outperforming results when datasets are rather small or noisy.

Citation

Utkin, L., Kovalev, M., & Coolen, F. (2020). Imprecise weighted extensions of random forests for classification and regression. Applied Soft Computing, 92, Article 106324. https://doi.org/10.1016/j.asoc.2020.106324

• Journal Article Type: Article
• Acceptance Date: Apr 16, 2020
• Online Publication Date: Apr 21, 2020
• Publication Date: Jul 31, 2020
• Deposit Date: Apr 18, 2020
• Publicly Available Date: Apr 21, 2021
• Journal: Applied Soft Computing
• Print ISSN: 1568-4946
• Publisher: Elsevier
• Peer Reviewed: Peer Reviewed
• Volume: 92
• Article Number: 106324
• DOI: https://doi.org/10.1016/j.asoc.2020.106324
• Public URL: https://durham-repository.worktribe.com/output/1266038