Skip to main content

Research Repository

Advanced Search

Adaptive Graph Learning from Spatial Information for Surgical Workflow Anticipation

Zhang, Francis Xiatian; Deng, Jingjing; Lieck, Robert; Shum, Hubert P.H.

Adaptive Graph Learning from Spatial Information for Surgical Workflow Anticipation Thumbnail


Authors



Abstract

Surgical workflow anticipation is the task of predicting the timing of relevant surgical events from live video data, which is critical in Robotic-Assisted Surgery (RAS). Accurate predictions require the use of spatial information to model surgical interactions. However, current methods focus solely on surgical instruments, assume static interactions between instruments, and only anticipate surgical events within a fixed time horizon. To address these challenges, we propose an adaptive graph learning framework for surgical workflow anticipation based on a novel spatial representation, featuring three key innovations. First, we introduce a new representation of spatial information based on bounding boxes of surgical instruments and targets, including their detection confidence levels. These are trained on additional annotations we provide for two benchmark datasets. Second, we design an adaptive graph learning method to capture dynamic interactions. Third, we develop a multi-horizon objective that balances learning objectives for different time horizons, allowing for unconstrained predictions. Evaluations on two benchmarks reveal superior performance in short-to-midterm anticipation, with an error reduction of approximately 3% for surgical phase anticipation and 9% for remaining surgical duration anticipation. These performance improvements demonstrate the effectiveness of our method and highlight its potential for enhancing preparation and coordination within the RAS team. This can improve surgical safety and the efficiency of operating room usage.

Citation

Zhang, F. X., Deng, J., Lieck, R., & Shum, H. P. (2025). Adaptive Graph Learning from Spatial Information for Surgical Workflow Anticipation. IEEE Transactions on Medical Robotics and Bionics, 7(1), 266-280. https://doi.org/10.1109/TMRB.2024.3517137

Journal Article Type Article
Acceptance Date Dec 5, 2024
Online Publication Date Dec 13, 2024
Publication Date 2025-02
Deposit Date Dec 6, 2024
Publicly Available Date Dec 20, 2024
Journal IEEE Transactions on Medical Robotics and Bionics
Electronic ISSN 2576-3202
Publisher Institute of Electrical and Electronics Engineers
Peer Reviewed Peer Reviewed
Volume 7
Issue 1
Pages 266-280
DOI https://doi.org/10.1109/TMRB.2024.3517137
Public URL https://durham-repository.worktribe.com/output/3202367

Files





You might also like



Downloadable Citations