FedBoosting: Federated learning with gradient protected boosting for text recognition

Ren, Hanchi; Deng, Jingjing; Xie, Xianghua; Ma, Xiaoke; Wang, Yichuan

Abstract

Conventional machine learning methodologies require the centralization of data for model training, which may be infeasible in situations where data sharing limitations are imposed due to concerns such as privacy and gradient protection. The Federated Learning (FL) framework enables the collaborative learning of a shared model without necessitating the centralization or sharing of data among the data proprietors. Nonetheless, in this paper, we demonstrate that the generalization capability of the joint model is suboptimal for Non-Independent and Non-Identically Distributed (Non-IID) data, particularly when employing the Federated Averaging (FedAvg) strategy as a result of the weight divergence phenomenon. Consequently, we present a novel boosting algorithm for FL to address both the generalization and gradient leakage challenges, as well as to facilitate accelerated convergence in gradient-based optimization. Furthermore, we introduce a secure gradient sharing protocol that incorporates Homomorphic Encryption (HE) and Differential Privacy (DP) to safeguard against gradient leakage attacks. Our empirical evaluation demonstrates that the proposed Federated Boosting (FedBoosting) technique yields significant enhancements in both prediction accuracy and computational efficiency in the visual text recognition task on publicly available benchmarks.

Citation

Ren, H., Deng, J., Xie, X., Ma, X., & Wang, Y. (2024). FedBoosting: Federated learning with gradient protected boosting for text recognition. Neurocomputing, 569, Article 127126. https://doi.org/10.1016/j.neucom.2023.127126

• Journal Article Type: Article
• Acceptance Date: Dec 6, 2023
• Online Publication Date: Dec 12, 2023
• Publication Date: Feb 7, 2024
• Deposit Date: Dec 13, 2023
• Publicly Available Date: Dec 19, 2023
• Journal: Neurocomputing
• Print ISSN: 0925-2312
• Publisher: Elsevier
• Peer Reviewed: Peer Reviewed
• Volume: 569
• Article Number: 127126
• DOI: https://doi.org/10.1016/j.neucom.2023.127126
• Keywords: Artificial Intelligence; Cognitive Neuroscience; Computer Science Applications
• Public URL: https://durham-repository.worktribe.com/output/2024402