6DGS: 6D Pose Estimation from a Single Image and a 3D Gaussian Splatting Model

Matteo, Bortolon; Tsesmelis, Theodore; James, Stuart; Poiesi, Fabio; Del Bue, Alessio

Abstract

We propose 6DGS to estimate the camera pose of a target RGB image given a 3D Gaussian Splatting (3DGS) model representing the scene. 6DGS avoids the iterative process typical of analysis-by-synthesis methods (e. g.iNeRF) that also require an initialization of the camera pose in order to converge. Instead, our method estimates a 6DoF pose by inverting the 3DGS rendering process. Starting from the object surface, we define a radiant Ellicell that uniformly generates rays departing from each ellipsoid that parameterize the 3DGS model. Each Ellicell ray is associated with the rendering parameters of each ellipsoid, which in turn is used to obtain the best bindings between the target image pixels and the cast rays. These pixel-ray bindings are then ranked to select the best scoring bundle of rays, which their intersection provides the camera center and, in turn, the camera rotation. The proposed solution obviates the necessity of an “a priori” pose for initialization, and it solves 6DoF pose estimation in closed form, without the need for iterations. Moreover, compared to the existing Novel View Synthesis (NVS) baselines for pose estimation, 6DGS can improve the overall average rotational accuracy by 12% and translation accuracy by 22% on real scenes, despite not requiring any initialization pose. At the same time, our method operates near real-time, reaching 15 fps on consumer hardware.

Citation

Matteo, B., Tsesmelis, T., James, S., Poiesi, F., & Del Bue, A. (2024, September). 6DGS: 6D Pose Estimation from a Single Image and a 3D Gaussian Splatting Model. Presented at Computer Vision – ECCV 2024 18th European Conference, Milan, Italy

• Presentation Conference Type: Conference Paper (published)
• Conference Name: Computer Vision – ECCV 2024 18th European Conference
• Start Date: Sep 29, 2024
• End Date: Oct 4, 2024
• Acceptance Date: Sep 30, 2024
• Online Publication Date: Nov 29, 2024
• Publication Date: Jan 1, 2025
• Deposit Date: Mar 26, 2025
• Publicly Available Date: Mar 26, 2025
• Print ISSN: 0302-9743
• Electronic ISSN: 1611-3349
• Peer Reviewed: Peer Reviewed
• Volume: 15110
• Pages: 420-436
• Series Title: Lecture Notes on Computer Science
• Series Number: 15110
• Series ISSN: 1611-3349
• Book Title: Computer Vision – ECCV 2024
• DOI: https://doi.org/10.1007/978-3-031-72943-0_24
• Public URL: https://durham-repository.worktribe.com/output/3741914