A Skeleton-aware Graph Convolutional Network for Human-Object Interaction Detection

Zhu, Manli; Ho, Edmund S.L.; Shum, Hubert P.H.

Abstract

Detecting human-object interactions is essential for comprehensive understanding of visual scenes. In particular, spatial connections between humans and objects are important cues for reasoning interactions. To this end, we propose a skeleton-aware graph convolutional network for human-object interaction detection, named SGCN4HOI. Our network exploits the spatial connections between human keypoints and object keypoints to capture their fine-grained structural interactions via graph convolutions. It fuses such geometric features with visual features and spatial configuration features obtained from human-object pairs. Furthermore, to better preserve the object structural information and facilitate human-object interaction detection, we propose a novel skeleton-based object keypoints representation. The performance of SGCN4HOI is evaluated in the public benchmark V-COCO dataset. Experimental results show that the proposed approach outperforms the state-of-theart pose-based models and achieves competitive performance against other models.

Citation

Zhu, M., Ho, E. S., & Shum, H. P. (2022, October). A Skeleton-aware Graph Convolutional Network for Human-Object Interaction Detection. Presented at IEEE SMC 2022: International Conference on Systems, Man, and Cybernetics, Prague, Czech Republic

• Presentation Conference Type: Conference Paper (published)
• Conference Name: IEEE SMC 2022: International Conference on Systems, Man, and Cybernetics
• Start Date: Oct 9, 2022
• End Date: Oct 12, 2022
• Acceptance Date: Jul 6, 2022
• Online Publication Date: Nov 18, 2022
• Publication Date: 2022
• Deposit Date: Jul 11, 2022
• Publicly Available Date: Oct 13, 2022
• Series ISSN: 1062-922X,2577-1655
• DOI: https://doi.org/10.1109/smc53654.2022.9945149
• Public URL: https://durham-repository.worktribe.com/output/1135977