Intro to Deep Learning and Generative Models Course - L19.5.2.6 BART: Combining Bidirectional and Auto-Regressive Transformers

Sebastian's books: https://sebastianraschka.com/books/ Slides: https://sebastianraschka.com/pdf/lecture-notes/stat453ss21/L19_seq2seq_rnn-transformers__slides.pdf 0:00 Introduction 0:33 BART. Combining Bidirectional and Auto-Regressive Transformers 2:14 BART. BERT Encoder + GPT Decoder - Noise Transformations 4:39 Noise Transformations in BART for Pre-Training on Unlabeled Data 6:19 BART Performance Under Different Noise Transformations 7:04 Fine-Tuning on Labeled Data 8:21 BART Performance for Discriminative Tasks 9:26 BART Performance for Generative Tasks ------- This video is part of my Introduction of Deep Learning course. Next video: https://youtu.be/OyqIuxMmLRg The complete playlist: https://www.youtube.com/playlist?list=PLTKMiZHVd_2KJtIXOW0zFhFfBaJJilH51 A handy overview page with links to the materials: https://sebastianraschka.com/blog/2021/dl-course.html ------- If you want to be notified about future videos, please consider subscribing to my channel: https://youtube.com/c/SebastianRaschka