Visual Graphs from Motion (VGfM): Scene understanding with object geometry reasoning

Gay, Paul; James, Stuart; Del Bue, Alessio

Abstract

Recent approaches on visual scene understanding attempt to build a scene graph – a computational representation of objects and their pairwise relationships. Such rich semantic representation is very appealing, yet difficult to obtain from a single image, especially when considering complex spatial arrangements in the scene. Differently, an image sequence conveys useful information using the multi-view geometric relations arising from camera motions. Indeed, object relationships are naturally related to the 3D scene structure. To this end, this paper proposes a system that first computes the geometrical location of objects in a generic scene and then efficiently constructs scene graphs from video by embedding such geometrical reasoning. Such compelling representation is obtained using a new model where geometric and visual features are merged using an RNN framework. We report results on a dataset we created for the task of 3D scene graph generation in multiple views.

Citation

Gay, P., James, S., & Del Bue, A. (2018, December). Visual Graphs from Motion (VGfM): Scene understanding with object geometry reasoning. Presented at ACCV 2018: Computer Vision – ACCV 2018, Perth, Australia

• Presentation Conference Type: Conference Paper (Published)
• Conference Name: ACCV 2018: Computer Vision – ACCV 2018
• Start Date: Dec 2, 2018
• End Date: Dec 6, 2018
• Online Publication Date: May 29, 2019
• Publication Date: 2018
• Deposit Date: Dec 13, 2023
• Print ISSN: 0302-9743
• Publisher: Springer
• Series Title: Lecture Notes in Computer Science
• Series Number: 11363
• Book Title: Computer Vision – ACCV 2018 14th Asian Conference on Computer Vision, Perth, Australia, December 2–6, 2018, Revised Selected Papers, Part III
• ISBN: 9783030208929
• DOI: https://doi.org/10.1007/978-3-030-20893-6_21
• Public URL: https://durham-repository.worktribe.com/output/2024591