Back to Butterworth - a Fourier Basis for 3D Surface Relief Hole Filling within RGB-D Imagery

Atapour-Abarghouei, Amir; de La Garanderie, Gregoire Payen; Breckon, Toby P.

Abstract

We address the problem of hole filling in RGB-D (color and depth) images, obtained from either active or stereo based sensing, for the purposes of object removal and missing depth estimation. This is performed independently on the low frequency depth information (surface shape) and the high frequency depth detail (relief) by way of a Fourier space transform and classical Butterworth high/low pass filtering. The high frequency detail is then filled using a texture synthesis method, whilst the low frequency shape information is inpainted using structural inpainting. Here, a classical non-parametric sampling approach is extended, using the concept of query expansion, to perform high frequency depth synthesis with the final output then recombined in Fourier space. In order to improve the overall depth relief (D) and edge detail accuracy, color information (RGB) is also used to constrain the sampling process within high frequency component completion. Experimental results demonstrate the efficacy of the proposed method outperforming prior work for generalized depth filling in the presence of high frequency surface relief detail.

Citation

Atapour-Abarghouei, A., de La Garanderie, G. P., & Breckon, T. P. (2016, December). Back to Butterworth - a Fourier Basis for 3D Surface Relief Hole Filling within RGB-D Imagery. Presented at 2016 23rd International Conference on Pattern Recognition (ICPR), Cancun

• Presentation Conference Type: Conference Paper (published)
• Conference Name: 2016 23rd International Conference on Pattern Recognition (ICPR)
• Start Date: Dec 4, 2016
• End Date: Dec 8, 2016
• Online Publication Date: Apr 24, 2017
• Publication Date: 2016
• Deposit Date: Oct 12, 2018
• Publicly Available Date: Nov 29, 2021
• Publisher: Institute of Electrical and Electronics Engineers
• Pages: 2813-2818
• Book Title: Proc. Int. Conf. on Pattern Recognition
• DOI: https://doi.org/10.1109/ICPR.2016.7900062
• Keywords: depth filling, RGB-D, surface relief, Fourier, DFT, hole filling, surface completion, frequency domain, 3D texture, depth completion, query expansion, depth map, texture synthesis, disparity hole filling, Butterworth filtering
• Public URL: https://durham-repository.worktribe.com/output/1143773