Towards affordable semantic searching: Zero-shot retrieval via dominant attributes

Long, Yang; Liu, Li; Shen, Yuming; Shao, Ling

Abstract

Instance-level retrieval has become an essential paradigm to index and retrieves images from large-scale databases. Conventional instance search requires at least an example of the query image to retrieve images that contain the same object instance. Existing semantic retrieval can only search semantically-related images, such as those sharing the same category or a set of tags, not the exact instances. Meanwhile, the unrealistic assumption is that all categories or tags are known beforehand. Training models for these semantic concepts highly rely on instance-level attributes or human captions which are expensive to acquire. Given the above challenges, this paper studies the Zero-shot Retrieval problem that aims for instance-level image search using only a few dominant attributes. The contributions are: 1) we utilise automatic word embedding to infer class-level attributes to circumvent expensive human labelling; 2) the inferred class-attributes can be extended into discriminative instance attributes through our proposed Latent Instance Attributes Discovery (LIAD) algorithm; 3) our method is not restricted to complete attribute signatures, query of dominant attributes can also be dealt with. On two benchmarks, CUB and SUN, extensive experiments demonstrate that our method can achieve promising performance for the problem. Moreover, our approach can also benefit conventional ZSL tasks.

Citation

Long, Y., Liu, L., Shen, Y., & Shao, L. (2018, December). Towards affordable semantic searching: Zero-shot retrieval via dominant attributes. Presented at Thirty-Second AAAI Conference on Artificial Intelligence

• Presentation Conference Type: Conference Paper (published)
• Conference Name: Thirty-Second AAAI Conference on Artificial Intelligence
• Acceptance Date: Nov 8, 2017
• Online Publication Date: Apr 27, 2018
• Publication Date: 2018
• Deposit Date: Sep 1, 2019
• Pages: 7210-7217
• Series ISSN: 2374-3468
• Book Title: Thirty-Second AAAI Conference on Artificial Intelligence ; proceedings.
• Public URL: https://durham-repository.worktribe.com/output/1141489
• Publisher URL: https://www.aaai.org/ocs/index.php/AAAI/AAAI18/paper/view/16626