ReEnact: Sketch Based Choreographic Design from Archival Dance Footage

James, Stuart; Fonseca, Manuel J.; Collomosse, John

Abstract

We describe a novel system for synthesising video choreography using sketched visual storyboards comprising human poses (stick men) and action labels. First, we describe an algorithm for searching archival dance footage using sketched pose. We match using an implicit representation of pose parsed from a mix of challenging low and high fidelity footage. In a training pre-process we learn a mapping between a set of exemplar sketches and corresponding pose representations parsed from the video, which are generalized at query-time to enable retrieval over previously unseen frames, and over additional unseen videos. Second, we describe how a storyboard of sketched poses, interspersed with labels indicating connecting actions, may be used to drive the synthesis of novel video choreography from the archival footage.

We demonstrate both our retrieval and synthesis algorithms over both low fidelity PAL footage from the UK Digital Dance Archives (DDA) repository of contemporary dance, circa 1970, and over higher-definition studio captured footage.

Citation

James, S., Fonseca, M. J., & Collomosse, J. (2014, April). ReEnact: Sketch Based Choreographic Design from Archival Dance Footage. Presented at ICMR '14: International Conference on Multimedia Retrieval, Glasgow

• Presentation Conference Type: Conference Paper (published)
• Conference Name: ICMR '14: International Conference on Multimedia Retrieval
• Start Date: Apr 1, 2014
• End Date: Apr 4, 2014
• Online Publication Date: Apr 1, 2014
• Publication Date: 2014
• Deposit Date: Dec 13, 2023
• Publisher: Association for Computing Machinery (ACM)
• Pages: 313-320
• Book Title: ICMR '14: Proceedings of International Conference on Multimedia Retrieval
• ISBN: 9781450327824
• DOI: https://doi.org/10.1145/2578726.2578766
• Public URL: https://durham-repository.worktribe.com/output/2024583