Optimal Control of Probability on A Target Set for Continuous-Time Markov Chains

Ma, Chenglin; Zhao, Huaizhong

Abstract

In this paper, a stochastic optimal control problem is considered for a continuous-time Markov chain taking values in a denumerable state space over a fixed finite horizon. The optimality criterion is the probability that the process remains in a target set before and at a certain time. The optimal value is a superadditive capacity of target sets. Under some minor assumptions for the controlled Markov process, we establish the dynamic programming principle, based on which we prove that the value function is a classical solution of the Hamilton-Jacobi-Bellman equation on a discrete lattice space. We then prove that there exists an optimal deterministic Markov control under the compactness assumption of control domain. We further prove that the value function is the unique solution of the HJB equation. We also consider the case starting from the outside of target set and give the corresponding results. Finally, we apply our results to two examples.

Citation

Ma, C., & Zhao, H. (online). Optimal Control of Probability on A Target Set for Continuous-Time Markov Chains. IEEE Transactions on Automatic Control, 69(2), 1202-1209. https://doi.org/10.1109/tac.2023.3278789

• Journal Article Type: Article
• Acceptance Date: May 13, 2023
• Online Publication Date: May 22, 2023
• Deposit Date: Jul 12, 2023
• Publicly Available Date: Jul 12, 2023
• Journal: IEEE Transactions on Automatic Control
• Print ISSN: 0018-9286
• Electronic ISSN: 1558-2523
• Publisher: Institute of Electrical and Electronics Engineers
• Peer Reviewed: Peer Reviewed
• Volume: 69
• Issue: 2
• Pages: 1202-1209
• DOI: https://doi.org/10.1109/tac.2023.3278789
• Public URL: https://durham-repository.worktribe.com/output/1170113