Neural-code PIFu: High-fidelity Single Image 3D Human Reconstruction via Neural Code Integration

Liu, Ruizhi; Remagnino, Paolo; Shum, Hubert P.H.

Abstract

We introduce neural-code PIFu, a novel implicit function for 3D human reconstruction, leveraging neural codebooks, our approach learns recurrent patterns in the feature space and reuses them to improve current features. Many existing methods predict normal maps from image feature space which easily overlook non-trivial features. Moreover, neglecting global geometric correlations restricted the use of repetitive features to improve the expressive power of current features. In this work, we propose neural-code PIFu, a novel framework that enhances initial features by fusing them with neural codes that are learned from the input features and geometric prior. It also models the global geometric correlation to facilitate the use of neural codes. Extensive experiments demonstrate that our method outperforms state-of-the-art (SoTA) PIFubased approaches by a large margin, and achieves comparable results to parametric-models-based methods without the use of auxiliary data.

Citation

Liu, R., Remagnino, P., & Shum, H. P. (2024, December). Neural-code PIFu: High-fidelity Single Image 3D Human Reconstruction via Neural Code Integration. Presented at 2024 International Conference on Pattern Recognition, Kolkata, India

• Presentation Conference Type: Conference Paper (published)
• Conference Name: 2024 International Conference on Pattern Recognition
• Start Date: Dec 1, 2024
• End Date: Dec 5, 2024
• Acceptance Date: Aug 6, 2024
• Deposit Date: Nov 7, 2024
• Publisher: Institute of Electrical and Electronics Engineers
• Peer Reviewed: Peer Reviewed
• Book Title: Proceedings of the 2024 International Conference on Pattern Recognition
• Public URL: https://durham-repository.worktribe.com/output/3084242