Xiatian Zhang xiatian.zhang@durham.ac.uk
Research Assistant/Associate (Casual)
Xiatian Zhang xiatian.zhang@durham.ac.uk
Research Assistant/Associate (Casual)
Dr Noura Al Moubayed noura.al-moubayed@durham.ac.uk
Associate Professor
Professor Hubert Shum hubert.shum@durham.ac.uk
Professor
Surgical workflow anticipation can give predictions on what steps to conduct or what instruments to use next, which is an essential part of the computer-assisted intervention system for surgery, e.g. workflow reasoning in robotic surgery. However, current approaches are limited to their insufficient expressive power for relationships between instruments. Hence, we propose a graph representation learning framework to comprehensively represent instrument motions in the surgical workflow anticipation problem. In our proposed graph representation, we maps the bounding box information of instruments to the graph nodes in the consecutive frames and build inter-frame/inter-instrument graph edges to represent the trajectory and interaction of the instruments over time. This design enhances the ability of our network on modeling both the spatial and temporal patterns of surgical instruments and their interactions. In addition, we design a multi-horizon learning strategy to balance the understanding of various horizons indifferent anticipation tasks, which significantly improves the model performance in anticipation with various horizons. Experiments on the Cholec80 dataset demonstrate the performance of our proposed method can exceed the state-of-the-art method based on richer backbones, especially in instrument anticipation (1.27 v.s. 1.48 for inMAE; 1.48 v.s. 2.68 for eMAE). To the best of our knowledge, we are the first to introduce a spatial-temporal graph representation into surgical workflow anticipation.
Zhang, X., Al Moubayed, N., & Shum, H. P. (2022). Towards Graph Representation Learning Based Surgical Workflow Anticipation. . https://doi.org/10.1109/bhi56158.2022.9926801
Presentation Conference Type | Conference Paper (Published) |
---|---|
Conference Name | 2022 IEEE-EMBS International Conference on Biomedical and Health Informatics (BHI) |
Start Date | Sep 27, 2022 |
End Date | Sep 30, 2022 |
Acceptance Date | Jul 19, 2022 |
Online Publication Date | Nov 4, 2022 |
Publication Date | 2022 |
Deposit Date | Aug 9, 2022 |
Publicly Available Date | Oct 1, 2022 |
Series ISSN | 2641-3590,2641-3604 |
DOI | https://doi.org/10.1109/bhi56158.2022.9926801 |
Public URL | https://durham-repository.worktribe.com/output/1135760 |
Accepted Conference Proceeding
(1.3 Mb)
PDF
Copyright Statement
© 2022 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
Is Unimodal Bias Always Bad for Visual Question Answering? A Medical Domain Study with Dynamic Attention
(2022)
Presentation / Conference Contribution
Efficient Uncertainty Quantification for Multilabel Text Classification
(2022)
Presentation / Conference Contribution
Contrastive Learning with Heterogeneous Graph Attention Networks on Short Text Classification
(2022)
Presentation / Conference Contribution
INTERACTION: A Generative XAI Framework for Natural Language Inference Explanations
(2022)
Presentation / Conference Contribution
Does lossy image compression affect racial bias within face recognition?
(2022)
Presentation / Conference Contribution
About Durham Research Online (DRO)
Administrator e-mail: dro.admin@durham.ac.uk
This application uses the following open-source libraries:
Apache License Version 2.0 (http://www.apache.org/licenses/)
Apache License Version 2.0 (http://www.apache.org/licenses/)
SIL OFL 1.1 (http://scripts.sil.org/OFL)
MIT License (http://opensource.org/licenses/mit-license.html)
CC BY 3.0 ( http://creativecommons.org/licenses/by/3.0/)
Powered by Worktribe © 2024
Advanced Search