Stanford AA289/ENGR319 - Robotics and Autonomous Systems Seminar - Stanford Seminar - Self-Supervised Pseudo-Lidar Networks

Adrien Gaidon Toyota Research Institute October 11, 2019 Although cameras are ubiquitous, robotic platforms typically rely on active sensors like LiDAR for direct 3D perception, especially in safety critical contexts like Automated Driving. Nonetheless, recent progress in combining deep learning and geometry suggests that cameras may become a competitive source of reliable 3D information. In this talk, we will present our latest developments in self-supervised monocular depth and pose estimation for urban environments. Particularly, we show that with the proper network architecture, large-scale training, and computational power it is possible to outperform fully supervised methods while still operating on the much more challenging self-supervised setting, where the only source of input information are video sequences. Furthermore, we discuss how other sources of information (i.e. camera velocity, sparse LiDAR data, and semantic predictions) can be leveraged at training time to further improve pseudo-lidar accuracy and overcome some of the inherent limitations of self-supervised learning. View the full playlist: https://www.youtube.com/playlist?list=PLoROMvodv4rMeercb-kvGLUrOq4HR6BZD 0:00 Introduction 2:46 Pseudo-LiDAR / Monocular Depth Estimation 6:00 Machine Learning at Scale: Beyond Supervised Learning 9:33 Self-Supervised Monocular Depth 10:36 Self-Supervised Depth Learning Objective 11:24 Photometric Loss ++ 13:30 Depth Super-Resolution 14:26 Dense Monocular 3D Reconstruction 23:41 Packing/Unpacking blocks 27:09 Velocity Scaling 37:25 Two Stream Nets for Self-Supervised VO 39:55 Why Semi-Supervised? 40:32 Semi = Self + Reprojected(Fully) 44:21 Semi-Supervised PackNet 46:43 Semantically-Guided Depth Network 49:54 Class-Specific Depth Evaluation 50:21 Two-Stage Training for Infinite Depth