Skip to main content

Research Repository

Advanced Search

A Video-Based Augmented Reality System for Human-in-the-Loop Muscle Strength Assessment of Juvenile Dermatomyositis

Zhou, Kanglei; Cai, Ruizhi; Ma, Yue; Tan, Qingqing; Wang, Xinning; Li, Jianguo; Shum, Hubert P.H.; Li, Frederick W.B.; Jin, Song; Liang, Xiaohui

A Video-Based Augmented Reality System for Human-in-the-Loop Muscle Strength Assessment of Juvenile Dermatomyositis Thumbnail


Authors

Kanglei Zhou

Ruizhi Cai

Yue Ma

Qingqing Tan

Xinning Wang

Song Jin

Xiaohui Liang



Abstract

As the most common idiopathic inflammatory myopathy in children, juvenile dermatomyositis (JDM) is characterized by skin rashes and muscle weakness. The childhood myositis assessment scale (CMAS) is commonly used to measure the degree of muscle involvement for diagnosis or rehabilitation monitoring. On the one hand, human diagnosis is not scalable and may be subject to personal bias. On the other hand, automatic action quality assessment (AQA) algorithms cannot guarantee 100% accuracy, making them not suitable for biomedical applications. As a solution, we propose a video-based augmented reality system for human-in-the-loop muscle strength assessment of children with JDM. We first propose an AQA algorithm for muscle strength assessment of JDM using contrastive regression trained by a JDM dataset. Our core insight is to visualize the AQA results as a virtual character facilitated by a 3D animation dataset, so that users can compare the real-world patient and the virtual character to understand and verify the AQA results. To allow effective comparisons, we propose a video-based augmented reality system. Given a feed, we adapt computer vision algorithms for scene understanding, evaluate the optimal way of augmenting the virtual character into the scene, and highlight important parts for effective human verification. The experimental results confirm the effectiveness of our AQA algorithm, and the results of the user study demonstrate that humans can more accurately and quickly assess the muscle strength of children using our system.

Citation

Zhou, K., Cai, R., Ma, Y., Tan, Q., Wang, X., Li, J., …Liang, X. (2023). A Video-Based Augmented Reality System for Human-in-the-Loop Muscle Strength Assessment of Juvenile Dermatomyositis. IEEE Transactions on Visualization and Computer Graphics, 29(5), 2456-2466. https://doi.org/10.1109/tvcg.2023.3247092

Journal Article Type Article
Acceptance Date Jan 30, 2023
Online Publication Date Feb 22, 2023
Publication Date 2023-05
Deposit Date Feb 3, 2023
Publicly Available Date Feb 22, 2023
Journal IEEE Transactions on Visualization and Computer Graphics
Print ISSN 1077-2626
Electronic ISSN 1941-0506
Publisher Institute of Electrical and Electronics Engineers
Peer Reviewed Peer Reviewed
Volume 29
Issue 5
Pages 2456-2466
DOI https://doi.org/10.1109/tvcg.2023.3247092
Public URL https://durham-repository.worktribe.com/output/1180140

Files

Accepted Journal Article (15.1 Mb)
PDF

Copyright Statement
© 2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.





You might also like



Downloadable Citations