Building LLMs from scratch - Loading pre-trained weights from OpenAI GPT-2

In this lecture, we learn how to load pretrained weights from OpenAI GPT-2 into our LLM architecture. We also test the model with pre-trained weights! This is a very comprehensive lecture which uses detailed whiteboard notes, Python code and develops your theory, intuition and also LLM code understanding. The key reference book which this video series very closely follows is Build a Large Language Model from Scratch by Manning Publications. All schematics and their descriptions are borrowed from this incredible book! This book serves as a comprehensive guide to understanding and building large language models, covering key concepts, techniques, and implementations. Affiliate links for purchasing the book will be added soon. Stay tuned for updates! 0:00 Need for retrained weights 2:26 Open AI GPT-2 weights 3:38 Loading libraries 5:12 Gpt2 weights downloading introduction 7:57 Understanding the gpt downloaded files 12:05 Understanding the gpt-2 download code 14:23 Understanding the gpt-2 parameter dictionary 28:40 Downloading gpt-2 weights into Python code 34:34 Integrating the gpt-2 weights with our LLM architecture 43:21 Testing the pre-trained GPT2 model 45:25 Research explorations and next steps Link to code file: https://drive.google.com/file/d/1k--iNpy0ynmiHCyn4YC8yiZFbzDNT-nO/view?usp=sharing gpt_download3.py code file: https://drive.google.com/file/d/1JX_DNJ9Xvty_ItciHhl_3TyOfahkGvvI/view?usp=sharing Kaggle OpenAI gpt-2 weights: https://www.kaggle.com/datasets/xhlulu/openai-gpt2-weights ================================================= ✉️ Join our FREE Newsletter: https://vizuara.ai/our-newsletter/ ================================================= Vizuara philosophy: As we learn AI/ML/DL the material, we will share thoughts on what is actually useful in industry and what has become irrelevant. We will also share a lot of information on which subject contains open areas of research. Interested students can also start their research journey there. Students who are confused or stuck in their ML journey, maybe courses and offline videos are not inspiring enough. What might inspire you is if you see someone else learning and implementing machine learning from scratch. No cost. No hidden charges. Pure old school teaching and learning. ================================================= 🌟 Meet Our Team: 🌟 🎓 Dr. Raj Dandekar (MIT PhD, IIT Madras department topper) 🔗 LinkedIn: https://www.linkedin.com/in/raj-abhijit-dandekar-67a33118a/ 🎓 Dr. Rajat Dandekar (Purdue PhD, IIT Madras department gold medalist) 🔗 LinkedIn: https://www.linkedin.com/in/rajat-dandekar-901324b1/ 🎓 Dr. Sreedath Panat (MIT PhD, IIT Madras department gold medalist) 🔗 LinkedIn: https://www.linkedin.com/in/sreedath-panat-8a03b69a/ 🎓 Sahil Pocker (Machine Learning Engineer at Vizuara) 🔗 LinkedIn: https://www.linkedin.com/in/sahil-p-a7a30a8b/ 🎓 Abhijeet Singh (Software Developer at Vizuara, GSOC 24, SOB 23) 🔗 LinkedIn: https://www.linkedin.com/in/abhijeet-singh-9a1881192/ 🎓 Sourav Jana (Software Developer at Vizuara) 🔗 LinkedIn: https://www.linkedin.com/in/souravjana131/