Deep Learning Course (NYU, Spring 2020) - Week 12 – Practicum: Attention and the Transformer

5.0 (0)

8 learners

What you'll learn

This course includes

42.5 hours of video
Certificate of completion
Access on mobile and TV

Summary

Keywords

Deep Learning Yann LeCun PyTorch NYU Neural Machine Translation NMT Natural Language Processing NLP attention transformer BERT OpenAI

Full Transcript

Course website: http://bit.ly/DLSP20-web Playlist: http://bit.ly/pDL-YouTube Speaker: Alfredo Canziani Week 12: http://bit.ly/DLSP20-12 0:00:00 – Week 12 – Practicum PRACTICUM: http://bit.ly/DLSP20-12-3 We introduce attention, focusing on self-attention and its hidden layer representations of the inputs. Then, we introduce the key-value store paradigm and discuss how to represent queries, keys, and values as rotations of an input. Finally, we use attention to interpret the transformer architecture, taking a forward pass through a basic transformer, and comparing the encoder-decoder paradigm to sequential architectures. 0:01:09 – Attention 0:17:36 – Key-value store 0:35:14 – Transformer and PyTorch implementation 0:54:00 – Q&A

Continue this lesson in the app

Install CourseHive on Android or iOS to keep learning while you move.

Related Courses

30-Day Beginner Guitar Challenge for New Players

Master the Guitar in 30 Days: Your Ultimate Beginner Challenge! Unleash your inner guitarist with step-by-step lessons designed to transform you from novice to confident player. Join Your Guitar Academy and kickstart your musical journey today!

⭐ 4.3

36 ratings

7 hours