NYU Deep Learning SP21 - 03L – Parameter sharing: recurrent and convolutional nets

Course website: http://bit.ly/DLSP21-web Playlist: http://bit.ly/DLSP21-YouTube Speaker: Yann LeCun Chapters 00:00:00 – Welcome to class 00:00:49 – Hypernetworks 00:02:24 – Shared weights 00:06:10 – Parameter sharing ⇒ adding the gradients 00:09:33 – Max and sum reductions 00:11:46 – Recurrent nets 00:14:20 – Unrolling in time 00:16:17 – Vanishing and exploding gradients 00:19:48 – Math on the whiteboard 00:23:18 – RNN tricks 00:24:29 – RNN for differential equations 00:27:18 – GRU 00:28:23 – What is a memory 00:41:26 – LSTM – Long Short-Term Memory net 00:43:11 – Multilayer LSTM 00:46:01 – Attention for sequence to sequence mapping 00:48:41 – Convolutional nets 00:50:50 – Detecting motifs in images 00:56:57 – Convolution definition(s) 00:59:43 – Backprop through convolutions 01:03:42 – Stride and skip: subsampling and convolution “à trous” 01:06:56 – Convolutional net architecture 01:19:08 – Multiple convolutions 01:20:37 – Vintage ConvNets 01:32:32 – How does the brain interpret images? 01:37:18 – Hubel & Wiesel's model of the visual cortex 01:42:51 – Invariance and equivariance of ConvNets 01:49:23 – In the next episode… 01:52:54 – Training time, iteration cycle, and historical remarks