Deep Learning for Computer Vision - Lecture 18: Videos

Lecture 18 discusses techniques for processing videos with neural networks. We see how single-frame CNNs are a strong baseline for video classification tasks, and how they can be improved with late fusion or early fusion. We contrast early and late fusion approaches with 3D CNNs that perform 3D convolution. We discuss optical flow, and see how two-stream networks use separate pathways for spatial and temporal information. We see how recurrent networks and self-attention can be utilized to capture long-term temporal information in videos. We discuss common architectures for video recognition, including C3D, I3D, and SlowFast networks. Slides: http://myumi.ch/51vGd _________________________________________________________________________________________________ Computer Vision has become ubiquitous in our society, with applications in search, image understanding, apps, mapping, medicine, drones, and self-driving cars. Core to many of these applications are visual recognition tasks such as image classification and object detection. Recent developments in neural network approaches have greatly advanced the performance of these state-of-the-art visual recognition systems. This course is a deep dive into details of neural-network based deep learning methods for computer vision. During this course, students will learn to implement, train and debug their own neural networks and gain a detailed understanding of cutting-edge research in computer vision. We will cover learning algorithms, neural network architectures, and practical engineering tricks for training and fine-tuning networks for visual recognition tasks. Course Website: http://myumi.ch/Bo9Ng Instructor: Justin Johnson http://myumi.ch/QA8Pg