Skip to main content

Research Repository

Advanced Search

From Category to Scenery: An End-to-End Framework for Multi-Person Human-Object Interaction Recognition in Videos

Qiao, Tanqiu; Li, Ruochen; Li, Frederick W B; Shum, Hubert P H

Authors

Tanqiu Qiao tanqiu.qiao@durham.ac.uk
PGR Student Doctor of Philosophy

Ruochen Li ruochen.li@durham.ac.uk
PGR Student Doctor of Philosophy



Abstract

Video-based Human-Object Interaction (HOI) recognition explores the intricate dynamics between humans and objects, which are essential for a comprehensive understanding of human behavior and intentions. While previous work has made significant strides, effectively integrating geometric and visual features to model dynamic relationships between humans and objects in a graph framework remains a challenge. In this work, we propose a novel end-to-end category to scenery framework, CATS, starting by generating geometric features for various categories through graphs respectively, then fusing them with corresponding visual features. Subsequently, we construct a scenery interactive graph with these enhanced geometric-visual features as nodes to learn the relationships among human and object categories. This methodological advance facilitates a deeper, more structured comprehension of interactions, bridging category-specific insights with broad scenery dynamics. Our method demonstrates state-of-the-art performance on two pivotal HOI benchmarks , including the MPHOI-72 dataset for multi-person HOIs and the single-person HOI CAD-120 dataset.

Citation

Qiao, T., Li, R., Li, F. W. B., & Shum, H. P. H. (2024, December). From Category to Scenery: An End-to-End Framework for Multi-Person Human-Object Interaction Recognition in Videos. Presented at Proceedings of the 2024 International Conference on Pattern Recognition, Kolkata, India, 2024., Kolkata, India

Presentation Conference Type Conference Paper (published)
Conference Name Proceedings of the 2024 International Conference on Pattern Recognition, Kolkata, India, 2024.
Start Date Dec 1, 2024
End Date Dec 5, 2024
Acceptance Date Jul 1, 2024
Deposit Date Jul 2, 2024
Journal International Conference on Pattern Recognition
Print ISSN 1051-4651
Electronic ISSN 2831-7475
Publisher Institute of Electrical and Electronics Engineers
Peer Reviewed Peer Reviewed
Public URL https://durham-repository.worktribe.com/output/2514413
Publisher URL https://ieeexplore.ieee.org/xpl/conhome/1000545/all-proceedings