Multi-task Deep Learning with Optical Flow Features for Self-Driving Cars

Hu, Yuan; Shum, Hubert P.H.; Ho, Edmond S.L.

Abstract

The control of self-driving cars has received growing attention recently. Although existing research shows promising results in the vehicle control using video from a monocular dash camera, there has been very limited work on directly learning vehicle control from motion-based cues. Such cues are powerful features for visual representations, as they encode the per-pixel movement between two consecutive images, allowing a system to effectively map the features into the control signal. The authors propose a new framework that exploits the use of a motion-based feature known as optical flow extracted from the dash camera and demonstrates that such a feature is effective in significantly improving the accuracy of the control signals. The proposed framework involves two main components. The flow predictor, as a self-supervised deep network, models the underlying scene structure from consecutive frames and generates the optical flow. The controller, as a supervised multi-task deep network, predicts both steer angle and speed. The authors demonstrate that the proposed framework using the optical flow features can effectively predict control signals from a dash camera video. Using the Cityscapes data set, the authors validate that the system prediction has errors as low as 0.0130 rad/s on steer angle and 0.0615 m/s on speed, outperforming existing research.

Citation

Hu, Y., Shum, H. P., & Ho, E. S. (2020). Multi-task Deep Learning with Optical Flow Features for Self-Driving Cars. IET Intelligent Transport Systems, 14(13), 1845-1854. https://doi.org/10.1049/iet-its.2020.0439

• Journal Article Type: Article
• Acceptance Date: Nov 16, 2020
• Online Publication Date: Jan 6, 2021
• Publication Date: 2020-12
• Deposit Date: Nov 17, 2020
• Publicly Available Date: Nov 17, 2020
• Journal: IET intelligent transport systems
• Print ISSN: 1751-956X
• Electronic ISSN: 1751-9578
• Publisher: Institution of Engineering and Technology (IET)
• Peer Reviewed: Peer Reviewed
• Volume: 14
• Issue: 13
• Pages: 1845-1854
• DOI: https://doi.org/10.1049/iet-its.2020.0439
• Public URL: https://durham-repository.worktribe.com/output/1257132