Depth Embedded Recurrent Predictive Parsing Network for Video Scenes

Zhou, Lingli; Zhang, Haofeng; Long, Yang; Shao, Ling; Yang, Jingyu

Abstract

Semantic segmentation-based scene parsing plays an important role in automatic driving and autonomous navigation. However, most of the previous models only consider static images, and fail to parse sequential images because they do not take the spatial-temporal continuity between consecutive frames in a video into account. In this paper, we propose a depth embedded recurrent predictive parsing network (RPPNet), which analyzes preceding consecutive stereo pairs for parsing result. In this way, RPPNet effectively learns the dynamic information from historical stereo pairs, so as to correctly predict the representations of the next frame. The other contribution of this paper is to systematically study the video scene parsing (VSP) task, in which we use the RPPNet to facilitate conventional image paring features by adding spatial-temporal information. The experimental results show that our proposed method RPPNet can achieve fine predictive parsing results on cityscapes and the predictive features of RPPNet can significantly improve conventional image parsing networks in VSP task.

Citation

Zhou, L., Zhang, H., Long, Y., Shao, L., & Yang, J. (2019). Depth Embedded Recurrent Predictive Parsing Network for Video Scenes. IEEE Transactions on Intelligent Transportation Systems, 20(12), 4643-4654. https://doi.org/10.1109/tits.2019.2909053

• Journal Article Type: Article
• Acceptance Date: Apr 1, 2019
• Online Publication Date: Apr 15, 2019
• Publication Date: Dec 31, 2019
• Deposit Date: Sep 1, 2019
• Publicly Available Date: Sep 3, 2019
• Journal: IEEE Transactions on Intelligent Transportation Systems
• Print ISSN: 1524-9050
• Electronic ISSN: 1558-0016
• Publisher: Institute of Electrical and Electronics Engineers
• Peer Reviewed: Peer Reviewed
• Volume: 20
• Issue: 12
• Pages: 4643-4654
• DOI: https://doi.org/10.1109/tits.2019.2909053
• Public URL: https://durham-repository.worktribe.com/output/1294785