ACR: Attention Collaboration-based Regressor for Arbitrary Two-Hand Reconstruction
Yu, Z.; Haung, S.; Fang, C.; Breckon, T.P.; Wang, J.
Professor Toby Breckon firstname.lastname@example.org
Reconstructing two hands from monocular RGB images is challenging due to frequent occlusion and mutual confusion. Existing methods mainly learn an entangled representation to encode two interacting hands, which are incredibly fragile to impaired interaction, such as truncated hands, separate hands, or external occlusion. This paper presents ACR (Attention Collaboration-based Regressor), which makes the first attempt to reconstruct hands in arbitrary scenarios. To achieve this, ACR explicitly mitigates interdependencies between hands and between parts by leveraging center and part-based attention for feature extraction. However, reducing interdependence helps release the input constraint while weakening the mutual reasoning about reconstructing the interacting hands. Thus, based on center attention, ACR also learns cross-hand prior that handle the interacting hands better. We evaluate our method on various types of hand reconstruction datasets. Our method significantly outperforms the best interacting-hand approaches on the InterHand2.6M dataset while yielding comparable performance with the state-ofthe-art single-hand methods on the FreiHand dataset. More qualitative results on in-the-wild and hand-object interaction datasets and web images/videos further demonstrate the effectiveness of our approach for arbitrary hand reconstruction.
Yu, Z., Haung, S., Fang, C., Breckon, T., & Wang, J. (2023). ACR: Attention Collaboration-based Regressor for Arbitrary Two-Hand Reconstruction. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). https://doi.org/10.1109/CVPR52729.2023.01245
|Conference Name||IEEE/CVF Conference on Computer Vision and Pattern Recognition 2023|
|Conference Location||Vancouver, BC|
|Start Date||Jun 17, 2023|
|End Date||Jun 24, 2023|
|Acceptance Date||Feb 27, 2023|
|Online Publication Date||Aug 22, 2023|
|Deposit Date||Apr 18, 2023|
|Publicly Available Date||Sep 7, 2023|
|Publisher||Institute of Electrical and Electronics Engineers|
|Book Title||2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)|
|Related Public URLs||https://breckon.org/toby/publications/papers/yu23hands.pdf|
Accepted Conference Proceeding
© 2023 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for all other uses, in any current or future media, including reprinting/republishing this material for advertising or promotional purposes, creating new collective works, for resale or redistribution to servers or lists, or reuse of any copyrighted component of this work in other works.
You might also like
Robust Semi-Supervised Anomaly Detection via Adversarially Learned Continuous Noise Corruption
Less is More: Reducing Task and Model Complexity for 3D Point Cloud Semantic Segmentation
Region-based Appearance and Flow Characteristics for Anomaly Detection in Infrared Surveillance Imagery
Unaligned 2D to 3D Translation with Conditional Vector-Quantized Code Diffusion using Transformers