Deep Learning Course (NYU, Spring 2020) - Week 8 – Lecture: Contrastive methods and regularised latent variable models

Course website: http://bit.ly/DLSP20-web Playlist: http://bit.ly/pDL-YouTube Speaker: Yann LeCun Week 8: http://bit.ly/DLSP20-08 0:00:00 – Week 8 – Lecture LECTURE Part A: http://bit.ly/DLSP20-08-1 In this section, we focused on the introduction of contrastive methods in Energy-Based Models in several aspects. First, we discuss the advantage brought by applying contrastive methods in self-supervised learning. Second, we discussed the architecture of denoising autoencoders and their weakness in image reconstruction tasks. We also talked about other contrastive methods, like contrastive divergence and persistent contrastive divergence. 0:00:05 – Recap on EBM and Characteristics of Different Contrastive Methods 0:10:13 – Contrastive Methods in Self-Supervised Learning 0:23:04 – Denoising Autoencoder and other Contrastive methods LECTURE Part B: http://bit.ly/DLSP20-08-2 In this section, we discussed regularized latent variable EBMs in detail covering concepts of conditional and unconditional versions of these models. We then discussed the algorithms of ISTA, FISTA and LISTA and look at examples of sparse coding and filters learned from convolutional sparse encoders. Finally we talked about Variational Auto-Encoders and the underlying concepts involved. 0:37:13 – Overview of Regularized Latent Variable Energy Based Models and Sparse Coding 1:07:46 – Convolutional Sparse Auto-Encoders 1:12:51 – Variational Auto-Encoders