Multi-Armed Bandit Approaches for Location Planning with Dynamic Relief Supplies Allocation Under Disaster Uncertainty

Liang, Jun; Zhang, Zongjia; Zhi, Yanpeng

Abstract

Highlights: What are the main findings? We design a three-layer emergency logistics network to manage the flow of disaster relief materials and develop a bi-objective, multi-period stochastic integer programming model to support post-disaster decision-making under uncertainty. Multi-armed bandit approaches are innovatively applied to solve the problem. A newly developed multi-armed bandit (reinforcement learning) algorithm called the Geometric Greedy algorithm, achieves overall higher performance than the traditional ϵ-Greedy algorithm and the Upper Confidence Bound (UCB) algorithm. What is the implication of the main finding? The key advantage of using reinforcement learning to solve our problem is that agents can dynamically adjust their strategies through interaction with the uncertain environment to minimize action costs. Abstract: Natural disasters (e.g., floods, earthquakes) significantly impact citizens, economies, and the environment worldwide. Due to their sudden onset, devastating effects, and high uncertainty, it is crucial for emergency departments to take swift action to minimize losses. Among these actions, planning the locations of relief supply distribution centers and dynamically allocating supplies is paramount, as governments must prioritize citizens’ safety and basic living needs following disasters. To address this challenge, this paper develops a three-layer emergency logistics network to manage the flow of emergency materials, from warehouses to transfer stations to disaster sites. A bi-objective, multi-period stochastic integer programming model is proposed to solve the emergency location, distribution, and allocation problem under uncertainty, focusing on three key decisions: transfer station selection, upstream emergency material distribution, and downstream emergency material allocation. We introduce a multi-armed bandit algorithm, named the Geometric Greedy algorithm, to optimize transfer station planning while accounting for subsequent dynamic relief supply distribution and allocation in a stochastic environment. The new algorithm is compared with two widely used multi-armed bandit algorithms: the ϵ-Greedy algorithm and the Upper Confidence Bound (UCB) algorithm. A case study in the Futian District of Shenzhen, China, demonstrates the practicality of our model and algorithms. The results show that the Geometric Greedy algorithm excels in both computational efficiency and convergence stability. This research offers valuable guidelines for emergency departments in optimizing the layout and flow of emergency logistics networks.

Citation

Liang, J., Zhang, Z., & Zhi, Y. (2025). Multi-Armed Bandit Approaches for Location Planning with Dynamic Relief Supplies Allocation Under Disaster Uncertainty. Smart Cities, 8(1), Article 5. https://doi.org/10.3390/smartcities8010005

• Journal Article Type: Article
• Acceptance Date: Dec 23, 2024
• Online Publication Date: Dec 25, 2024
• Publication Date: 2025-02
• Deposit Date: Jan 17, 2025
• Publicly Available Date: Jan 17, 2025
• Journal: Smart Cities
• Electronic ISSN: 2624-6511
• Publisher: MDPI
• Peer Reviewed: Peer Reviewed
• Volume: 8
• Issue: 1
• Article Number: 5
• DOI: https://doi.org/10.3390/smartcities8010005
• Keywords: location-allocation model, emergency logistics network, disaster uncertainty, multi-armed bandit approaches
• Public URL: https://durham-repository.worktribe.com/output/3332419