U3DS3 : Unsupervised 3D Semantic Scene Segmentation

Liu, Jiaxu; Yu, Zhengdi; Breckon, Toby P; Shum, Hubert P H

Abstract

Contemporary point cloud segmentation approaches largely rely on richly annotated 3D training data. However , it is both time-consuming and challenging to obtain consistently accurate annotations for such 3D scene data. Moreover, there is still a lack of investigation into fully un-supervised scene segmentation for point clouds, especially for holistic 3D scenes. This paper presents U3DS 3 , as a step towards completely unsupervised point cloud segmen-tation for any holistic 3D scenes. To achieve this, U3DS 3 leverages a generalized unsupervised segmentation method for both object and background across both indoor and outdoor static 3D point clouds with no requirement for model pre-training, by leveraging only the inherent information of the point cloud to achieve full 3D scene segmentation. The initial step of our proposed approach involves generating superpoints based on the geometric characteristics of each scene. Subsequently, it undergoes a learning process through a spatial clustering-based methodology, followed by iterative training using pseudo-labels generated in accordance with the cluster centroids. Moreover, by leverag-ing the invariance and equivariance of the volumetric representations , we apply the geometric transformation on vox-elized features to provide two sets of descriptors for robust representation learning. Finally, our evaluation provides state-of-the-art results on the ScanNet and SemanticKITTI, and competitive results on the S3DIS, benchmark datasets.

Citation

Liu, J., Yu, Z., Breckon, T. P., & Shum, H. P. H. (2024). U3DS3 : Unsupervised 3D Semantic Scene Segmentation. In 2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV) (3747-3756). https://doi.org/10.1109/WACV57701.2024.00372

• Conference Name: 2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)
• Conference Location: Waikoloa, Hawaii, USA
• Start Date: Jan 3, 2024
• End Date: Jan 8, 2024
• Acceptance Date: Oct 24, 2023
• Online Publication Date: Apr 9, 2024
• Publication Date: Apr 9, 2024
• Deposit Date: Nov 7, 2023
• Publicly Available Date: Apr 9, 2024
• Publisher: Institute of Electrical and Electronics Engineers
• Pages: 3747-3756
• Series ISSN: 2472-6737
• Book Title: 2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)
• ISBN: 9798350318937
• DOI: https://doi.org/10.1109/WACV57701.2024.00372
• Public URL: https://durham-repository.worktribe.com/output/1900650