Skip to main content

Research Repository

Advanced Search

Generative Adversarial Framework for Depth Filling via Wasserstein Metric, Cosine Transform and Domain Transfer

Atapour-Abarghouei, Amir; Akcay, Samet; de La Garanderie, Grégoire Payen; Breckon, Toby P.

Authors

Amir Atapour-Abarghouei

Samet Akcay samet.akcay@durham.ac.uk
PGR Student Doctor of Philosophy

Grégoire Payen de La Garanderie



Contributors

Abstract

In this work, the issue of depth filling is addressed using a self-supervised feature learning model that predicts missing depth pixel values based on the context and structure of the scene. A fully-convolutional generative model is conditioned on the available depth information and full RGB colour information from the scene and trained in an adversarial fashion to complete scene depth. Since ground truth depth is not readily available, synthetic data is instead used with a separate model developed to predict where holes would appear in a sensed (non-synthetic) depth image based on the contents of the RGB image. The resulting synthetic data with realistic holes is utilized in training the depth filling model which makes joint use of a reconstruction loss which employs the Discrete Cosine Transform for more realistic outputs, an adversarial loss which measures the distribution distances via the Wasserstein metric and a bottleneck feature loss that aids in better contextual feature execration. Additionally, the model is adversarially adapted to perform well on naturally-obtained data with no available ground truth. Qualitative and quantitative evaluations demonstrate the efficacy of the approach compared to contemporary depth filling techniques. The strength of the feature learning capabilities of the resulting deep network model is also demonstrated by performing the task of monocular depth estimation using our pre-trained depth hole filling model as the initialization for subsequent transfer learning.

Citation

Atapour-Abarghouei, A., Akcay, S., de La Garanderie, G. P., & Breckon, T. P. (2019). Generative Adversarial Framework for Depth Filling via Wasserstein Metric, Cosine Transform and Domain Transfer. Pattern Recognition, 91, 232-244. https://doi.org/10.1016/j.patcog.2019.02.010

Journal Article Type Article
Acceptance Date Feb 17, 2019
Online Publication Date Feb 27, 2019
Publication Date Jul 31, 2019
Deposit Date Feb 28, 2019
Publicly Available Date Feb 27, 2020
Journal Pattern Recognition
Print ISSN 0031-3203
Publisher Elsevier
Peer Reviewed Peer Reviewed
Volume 91
Pages 232-244
DOI https://doi.org/10.1016/j.patcog.2019.02.010
Related Public URLs http://atapour.co.uk/papers/atapour19gan.pdf

Files







You might also like



Downloadable Citations