Intro to Deep Learning and Generative Models Course - L19.4.1 Using Attention Without the RNN -- A Basic Form of Self-Attention

Sebastian's books: https://sebastianraschka.com/books/ Slides: https://sebastianraschka.com/pdf/lecture-notes/stat453ss21/L19_seq2seq_rnn-transformers__slides.pdf 00:00 Introducing self attention and transformer networks. 02:05 Introduction to RNNs with an Attention Mechanism 04:08 Attention Mechanism is a foundational concept in transformer architecture. 06:07 Introduction to self attention mechanism in transformers 08:04 RNNs with Attention Mechanism use weighted sum to compute attention value 10:32 RNNs with Attention Mechanism involve computing normalized attention weights using softmax function. 12:24 RNNs with attention use dot product to compute similarity. 14:29 Word embeddings in RNNs provide consistent values regardless of word position. Crafted by Merlin AI. ------- This video is part of my Introduction of Deep Learning course. Next video: https://youtu.be/0PjHri8tc1c The complete playlist: https://www.youtube.com/playlist?list=PLTKMiZHVd_2KJtIXOW0zFhFfBaJJilH51 A handy overview page with links to the materials: https://sebastianraschka.com/blog/2021/dl-course.html ------- If you want to be notified about future videos, please consider subscribing to my channel: https://youtube.com/c/SebastianRaschka