Imaginary-Connected Embedding in Complex Space for Unseen Attribute-Object Discrimination

Jiang, Chenyi; Wang, Shidong; Long, Yang; Li, Zechao; Zhang, Haofeng; Shao, Ling

Abstract

Compositional Zero-Shot Learning (CZSL) aims to recognize novel compositions of seen primitives. Prior studies have attempted to either learn primitives individually (non-connected) or establish dependencies among them in the composition (fully-connected). In contrast, human comprehension of composition diverges from the aforementioned methods as humans possess the ability to make composition-aware adaptation for these primitives, instead of inferring them rigidly through the aforementioned methods. However, developing a comprehension of compositions akin to human cognition proves challenging within the confines of real space. This arises from the limitation of real-space-based methods, which often categorize attributes, objects, and compositions using three independent measures, without establishing a direct dynamic connection. To tackle this challenge, we expand the CZSL distance metric scheme to encompass complex spaces to unify the independent measures, and we establish an imaginary-connected embedding in complex space to model human understanding of attributes. To achieve this representation, we introduce an innovative visual bias-based attribute extraction module that selectively extracts attributes based on object prototypes. As a result, we are able to incorporate phase information in training and inference, serving as a metric for attribute-object dependencies while preserving the independent acquisition of primitives. We evaluate the effectiveness of our proposed approach on three benchmark datasets, illustrating its superiority compared to baseline methods. Our code is available at https://github.com/LanchJL/IMAX.

Citation

Jiang, C., Wang, S., Long, Y., Li, Z., Zhang, H., & Shao, L. (online). Imaginary-Connected Embedding in Complex Space for Unseen Attribute-Object Discrimination. IEEE Transactions on Pattern Analysis and Machine Intelligence, https://doi.org/10.1109/tpami.2024.3487631

• Journal Article Type: Article
• Acceptance Date: Oct 26, 2024
• Online Publication Date: Oct 29, 2024
• Deposit Date: Oct 31, 2024
• Publicly Available Date: Nov 1, 2024
• Journal: IEEE Transactions on Pattern Analysis and Machine Intelligence
• Print ISSN: 0162-8828
• Electronic ISSN: 1939-3539
• Publisher: Institute of Electrical and Electronics Engineers
• Peer Reviewed: Peer Reviewed
• DOI: https://doi.org/10.1109/tpami.2024.3487631
• Keywords: Index Terms-Compositional Zero-Shot Learning; Compositionality; Visual-Attribute; Complex Space; Open-World Classification ✦
• Public URL: https://durham-repository.worktribe.com/output/2994465