Smart Multimodal In-Bed Pose Estimation Framework Incorporating Generative Adversarial Neural Network

Singh, Sumit; Anisi, Mohammad Hossein; Jindal, Anish; Jarchi, Delaram

Abstract

Monitoring in-bed pose estimation based on the Internet of Medical Things (IoMT) and ambient technology has a significant impact on many applications such as sleep-related disorders including obstructive sleep apnea syndrome, assessment of sleep quality, and health risk of pressure ulcers. In this research, a new multimodal in-bed pose estimation has been proposed using a deep learning framework. The Simultaneously-collected multimodal Lying Pose (SLP) dataset has been used for performance evaluation of the proposed framework where two modalities including long wave infrared (LWIR) and depth images are used to train the proposed model. The main contribution of this research is the feature fusion network and the use of a generative model to generate RGB images having similar poses to other modalities (LWIR/depth). The inclusion of a generative model helps to improve the overall accuracy of the pose estimation algorithm. Moreover, the method can be generalized for situations to recover human pose both in home and hospital settings under various cover thickness levels. The proposed model is compared with other fusion-based models and shows an improved performance of 97.8% at PCKh@0.5. In addition, performance has been evaluated for different cover conditions, and under home and hospital environments which present improvements using our proposed model.

Citation

Singh, S., Anisi, M. H., Jindal, A., & Jarchi, D. (2024). Smart Multimodal In-Bed Pose Estimation Framework Incorporating Generative Adversarial Neural Network. IEEE Journal of Biomedical and Health Informatics, https://doi.org/10.1109/jbhi.2024.3384453

• Journal Article Type: Article
• Acceptance Date: Mar 29, 2024
• Online Publication Date: Apr 4, 2024
• Publication Date: Apr 4, 2024
• Deposit Date: Apr 22, 2024
• Publicly Available Date: Apr 22, 2024
• Journal: IEEE Journal of Biomedical and Health Informatics
• Print ISSN: 2168-2194
• Electronic ISSN: 2168-2208
• Publisher: Institute of Electrical and Electronics Engineers
• Peer Reviewed: Peer Reviewed
• DOI: https://doi.org/10.1109/jbhi.2024.3384453
• Keywords: Health Information Management; Electrical and Electronic Engineering; Computer Science Applications; Health Informatics
• Public URL: https://durham-repository.worktribe.com/output/2392841