Stanford CS224N: Natural Language Processing with Deep Learning Course | Winter 2019 - Stanford CS224N: NLP with Deep Learning | Winter 2019 | Lecture 6 – Language Models and RNNs

For more information about Stanford’s Artificial Intelligence professional and graduate programs, visit: https://stanford.io/3n7saLk Professor Christopher Manning & PhD Candidate Abigail See, Stanford University http://onlinehub.stanford.edu/ Professor Christopher Manning Thomas M. Siebel Professor in Machine Learning, Professor of Linguistics and of Computer Science Director, Stanford Artificial Intelligence Laboratory (SAIL) To follow along with the course schedule and syllabus, visit: http://web.stanford.edu/class/cs224n/index.html#schedule 0:00 Introduction 0:33 Overview 2:50 You use Language Models every day! 5:36 n-gram Language Models: Example 10:12 Sparsity Problems with n-gram Language Models 10:58 Storage Problems with n-gram Language Models 11:34 n-gram Language Models in practice 12:53 Generating text with a n-gram Language Model 15:08 How to build a neural Language Model? 16:03 A fixed-window neural Language Model 20:57 Recurrent Neural Networks (RNN) 22:39 ARNN Language Model 32:51 Training a RNN Language Model 36:35 Multivariable Chain Rule 37:10 Backpropagation for RNNs: Proof sketch 41:23 Generating text with a RNN Language Model 51:39 Evaluating Language Models 53:30 RNNs have greatly improved perplexity 54:09 Why should we care about Language Modeling? 58:30 Recap 59:21 RNNs can be used for tagging