Multi-Feature Fusion Enhanced Monocular Depth Estimation With Boundary Awareness

Song, Chao; Chen, Qingjie; Li, Frederick W. B.; Jiang, Zhaoyi; Zheng, Dong; Shen, Yuliang; Yang, Bailin

Abstract

Self-supervised monocular depth estimation has opened up exciting possibilities for practical applications, including scene understanding, object detection, and autonomous driving, without the need for expensive depth annotations. However, traditional methods for single-image depth estimation encounter limitations in photometric loss due to a lack of geometric constraints, reliance on pixel-level intensity or color differences, and the assumption of perfect photometric consistency, leading to errors in challenging conditions and resulting in overly smooth depth maps with insufficient capture of object boundaries and depth transitions. To tackle these challenges, we propose MFFENet, which leverages multi-level semantic and boundary-aware features to improve depth estimation accuracy. MFFENet extracts multi-level semantic features using our modified HRFormer approach. These features are then fed into our decoder and enhanced using attention mechanisms to enrich the boundary information generated by Laplacian pyramid residuals. To mitigate the weakening of semantic features during convolution processes, we introduce a feature-enhanced combination strategy. We also integrate the DeconvUp module to improve the restoration of depth map boundaries. We introduce a boundary loss that enforces constraints between object boundaries. We propose an extended evaluation method that utilizes Laplacian pyramid residuals to evaluate boundary depth. Extensive evaluations on the KITTI, Cityscape, and Make3D datasets demonstrate the superior performance of MFFENet compared to state-of-the-art models in monocular depth estimation.

Citation

Song, C., Chen, Q., Li, F. W. B., Jiang, Z., Zheng, D., Shen, Y., & Yang, B. (2024). Multi-Feature Fusion Enhanced Monocular Depth Estimation With Boundary Awareness. Visual Computer, 40, 4955–4967. https://doi.org/10.1007/s00371-024-03498-w

• Journal Article Type: Article
• Acceptance Date: May 15, 2024
• Online Publication Date: Jun 22, 2024
• Publication Date: Jun 22, 2024
• Deposit Date: Apr 29, 2024
• Publicly Available Date: Jun 25, 2024
• Journal: Visual Computer
• Print ISSN: 0178-2789
• Electronic ISSN: 1432-2315
• Publisher: Springer
• Peer Reviewed: Peer Reviewed
• Volume: 40
• Pages: 4955–4967
• DOI: https://doi.org/10.1007/s00371-024-03498-w
• Public URL: https://durham-repository.worktribe.com/output/2408008
• Publisher URL: https://link.springer.com/article/10.1007/s00371-024-03498-w
• Additional Information: Accepted to publish in a Special Issue for Computer Graphics International (CGI) 2024; https://www.cgs-network.org/cgi24/