Skip to main content

Research Repository

Advanced Search

From Category to Scenery: An End-to-End Framework for Multi-Person Human-Object Interaction Recognition in Videos

Qiao, Tanqiu; Li, Ruochen; Li, Frederick W B; Shum, Hubert P H

Authors

Tanqiu Qiao tanqiu.qiao@durham.ac.uk
PGR Student Doctor of Philosophy

Ruochen Li ruochen.li@durham.ac.uk
PGR Student Doctor of Philosophy



Abstract

Video-based Human-Object Interaction (HOI) recognition explores the intricate dynamics between humans and objects, which are essential for a comprehensive understanding of human behavior and intentions. While previous work has made significant strides, effectively integrating geometric and visual features to model dynamic relationships between humans and objects in a graph framework remains a challenge. In this work, we propose a novel end-to-end category to scenery framework, CATS, starting by generating geometric features for various categories through graphs respectively, then fusing them with corresponding visual features. Subsequently, we construct a scenery interactive graph with these enhanced geometric-visual features as nodes to learn the relationships among human and object categories. This methodological advance facilitates a deeper, more structured comprehension of interactions, bridging category-specific insights with broad scenery dynamics. Our method demonstrates state-of-the-art performance on two pivotal HOI benchmarks , including the MPHOI-72 dataset for multi-person HOIs and the single-person HOI CAD-120 dataset.

Citation

Qiao, T., Li, R., Li, F. W. B., & Shum, H. P. H. (in press). From Category to Scenery: An End-to-End Framework for Multi-Person Human-Object Interaction Recognition in Videos. International Conference on Pattern Recognition,

Presentation Conference Type Conference Paper (published)
Conference Name Proceedings of the 2024 International Conference on Pattern Recognition, Kolkata, India, 2024.
Acceptance Date Jul 1, 2024
Deposit Date Jul 2, 2024
Journal International Conference on Pattern Recognition
Print ISSN 1051-4651
Publisher Institute of Electrical and Electronics Engineers
Peer Reviewed Peer Reviewed
Public URL https://durham-repository.worktribe.com/output/2514413

This file is under embargo due to copyright reasons.





You might also like



Downloadable Citations