Francis Xiatian Zhang xiatian.zhang@durham.ac.uk
PGR Student Doctor of Philosophy
Francis Xiatian Zhang xiatian.zhang@durham.ac.uk
PGR Student Doctor of Philosophy
Dr Jingjing Deng jingjing.deng@durham.ac.uk
Assistant Professor
Dr Robert Lieck robert.lieck@durham.ac.uk
Assistant Professor
Professor Hubert Shum hubert.shum@durham.ac.uk
Professor
Surgical workflow anticipation is the task of predicting the timing of relevant surgical events from live video data, which is critical in Robotic-Assisted Surgery (RAS). Accurate predictions require the use of spatial information to model surgical interactions. However, current methods focus solely on surgical instruments, assume static interactions between instruments, and only anticipate surgical events within a fixed time horizon. To address these challenges, we propose an adaptive graph learning framework for surgical workflow anticipation based on a novel spatial representation, featuring three key innovations. First, we introduce a new representation of spatial information based on bounding boxes of surgical instruments and targets, including their detection confidence levels. These are trained on additional annotations we provide for two benchmark datasets. Second, we design an adaptive graph learning method to capture dynamic interactions. Third, we develop a multi-horizon objective that balances learning objectives for different time horizons, allowing for unconstrained predictions. Evaluations on two benchmarks reveal superior performance in short-to-midterm anticipation, with an error reduction of approximately 3% for surgical phase anticipation and 9% for remaining surgical duration anticipation. These performance improvements demonstrate the effectiveness of our method and highlight its potential for enhancing preparation and coordination within the RAS team. This can improve surgical safety and the efficiency of operating room usage.
Zhang, F. X., Deng, J., Lieck, R., & Shum, H. P. (2025). Adaptive Graph Learning from Spatial Information for Surgical Workflow Anticipation. IEEE Transactions on Medical Robotics and Bionics, 7(1), 266-280. https://doi.org/10.1109/TMRB.2024.3517137
Journal Article Type | Article |
---|---|
Acceptance Date | Dec 5, 2024 |
Online Publication Date | Dec 13, 2024 |
Publication Date | 2025-02 |
Deposit Date | Dec 6, 2024 |
Publicly Available Date | Dec 20, 2024 |
Journal | IEEE Transactions on Medical Robotics and Bionics |
Electronic ISSN | 2576-3202 |
Publisher | Institute of Electrical and Electronics Engineers |
Peer Reviewed | Peer Reviewed |
Volume | 7 |
Issue | 1 |
Pages | 266-280 |
DOI | https://doi.org/10.1109/TMRB.2024.3517137 |
Public URL | https://durham-repository.worktribe.com/output/3202367 |
Accepted Journal Article
(1.1 Mb)
PDF
Depth-Aware Endoscopic Video Inpainting
(2024)
Presentation / Conference Contribution
Pose-based tremor type and level analysis for Parkinson’s disease from video
(2024)
Journal Article
Correlation-Distance Graph Learning for Treatment Response Prediction from rs-fMRI
(2023)
Presentation / Conference Contribution
About Durham Research Online (DRO)
Administrator e-mail: dro.admin@durham.ac.uk
This application uses the following open-source libraries:
Apache License Version 2.0 (http://www.apache.org/licenses/)
Apache License Version 2.0 (http://www.apache.org/licenses/)
SIL OFL 1.1 (http://scripts.sil.org/OFL)
MIT License (http://opensource.org/licenses/mit-license.html)
CC BY 3.0 ( http://creativecommons.org/licenses/by/3.0/)
Powered by Worktribe © 2025
Advanced Search