Interactive Video Asset Retrieval Using Sketched Queries

James, Stuart; Collomosse, John

Abstract

We present a new algorithm for searching video repositories using free-hand sketches. Our queries express both appearance (color, shape) and motion attributes, as well as semantic properties (object labels) enabling hybrid queries to be specified. Unlike existing sketch based video retrieval (SBVR) systems that enable hybrid queries of this form, we do not adopt a model fitting/optimization approach to match at query-time. Rather, we create an efficiently searchable index via a novel space-time descriptor that encapsulates all these properties. The real-time performance yielded by our indexing approach enables interactive refinement of search results within a relevance feedback (RF) framework; a unique contribution to SBVR. We evaluate our system over 700 sports footage clips exhibiting a variety of clutter and motion conditions, demonstrating significant accuracy and speed gains over the state of the art.

Citation

James, S., & Collomosse, J. (2014, November). Interactive Video Asset Retrieval Using Sketched Queries. Presented at CVMP '14: 11th European Conference on Visual Media Production, London

• Presentation Conference Type: Conference Paper (published)
• Conference Name: CVMP '14: 11th European Conference on Visual Media Production
• Start Date: Nov 13, 2014
• End Date: Nov 14, 2014
• Online Publication Date: Nov 13, 2014
• Publication Date: 2014
• Deposit Date: Dec 13, 2023
• Publisher: Association for Computing Machinery (ACM)
• Pages: 1-8
• Book Title: Proceedings of Conference on Visual Media Production (CVMP)
• ISBN: 9781450331852
• DOI: https://doi.org/10.1145/2668904.2668940
• Public URL: https://durham-repository.worktribe.com/output/2024585