Deep Learning Course (NYU, Spring 2020) - Week 10 – Lecture: Self-supervised learning (SSL) in computer vision (CV)

Course website: http://bit.ly/DLSP20-web Playlist: http://bit.ly/pDL-YouTube Speaker: Ishan Misra Week 10: http://bit.ly/DLSP20-10 0:00:00 – Week 10 – Lecture LECTURE Part A: http://bit.ly/DLSP20-10-1 In this section, we understand the motivation behind Self-Supervised Learning (SSL), define what it is and see some of its applications in NLP and Computer Vision. We understand how pretext tasks aid with SSL and see some example pretext tasks in images, videos and videos with sound. Finally, we try to get an intuition behind the representation learned by pretext tasks. 0:01:15 – Challenges of supervised learning and how self-supervised learning differs from supervised and unsupervised, with examples in NLP and Relative positions for vision 0:12:39 – Examples of pretext tasks in images, videos and videos with sound 0:40:26 – Understanding what the "pretext" task learns LECTURE Part B: http://bit.ly/DLSP20-10-2 In this section, we discuss the shortcomings of pretext tasks, define characteristics that make a good pre-trained feature, and how we can achieve this using Clustering and Contrastive Learning. We then learn about ClusterFit, its steps and performance. We further dive into a specific simple framework for Contrastive Learning known as PIRL. We discuss its working as well as its evaluation in different contexts. 1:01:50 – Generalization of pretext task and ClusterFit 1:19:08 – Basic idea of PIRL 1:38:09 – Evaluating PIRL on different tasks and questions