Building LLMs from scratch
5.0
(1)
12 learners
What you'll learn
This course includes
- 30.5 hours of video
- Certificate of completion
- Access on mobile and TV
Course content
1 modules • 43 lessons • 30.5 hours of video
Building LLMs from scratch
43 lessons
• 30.5 hours
Building LLMs from scratch
43 lessons
• 30.5 hours
- Lecture 1: Building LLMs from scratch: Series introduction 16:08
- Lecture 2: Large Language Models (LLM) Basics 33:42
- Lecture 3: Pretraining LLMs vs Finetuning LLMs 28:12
- Lecture 4: What are transformers? 40:39
- Lecture 5: How does GPT-3 really work? 48:05
- Lecture 6: Stages of building an LLM from Scratch 20:15
- Lecture 7: Code an LLM Tokenizer from Scratch in Python 01:09:44
- Lecture 8: The GPT Tokenizer: Byte Pair Encoding 53:35
- Lecture 9: Creating Input-Target data pairs using Python DataLoader 55:45
- Lecture 10: What are token embeddings? 01:00:52
- Lecture 11: The importance of Positional Embeddings 48:52
- Lecture 12: The entire Data Preprocessing Pipeline of Large Language Models (LLMs) 01:34:15
- Lecture 13: Introduction to the Attention Mechanism in Large Language Models (LLMs) 51:25
- Lecture 14: Simplified Attention Mechanism - Coded from scratch in Python | No trainable weights 01:19:22
- Lecture 15: Coding the self attention mechanism with key, query and value matrices 01:19:08
- Lecture 16: Causal Self Attention Mechanism | Coded from scratch in Python 55:55
- Lecture 17: Multi Head Attention Part 1 - Basics and Python code 32:19
- Lecture 18: Multi Head Attention Part 2 - Entire mathematics explained 01:01:13
- Lecture 19: Birds Eye View of the LLM Architecture 48:51
- Lecture 20: Layer Normalization in the LLM Architecture 38:57
- GELU Activation Function in the LLM Architecture 27:57
- Shortcut connections in the LLM Architecture 32:46
- Coding the entire LLM Transformer Block 45:06
- Coding the 124 million parameter GPT-2 model 01:01:33
- Coding GPT-2 to predict the next token 40:59
- Measuring the LLM loss function 56:14
- Evaluating LLM performance on real dataset | Hands on project | Book data 58:36
- Coding the entire LLM Pre-training Loop 43:21
- Temperature Scaling in Large Language Models (LLMs) 26:32
- Top-k sampling in Large Language Models 23:34
- Saving and loading LLM model weights using PyTorch 12:26
- Loading pre-trained weights from OpenAI GPT-2 50:21
- Introduction to LLM Finetuning | Python Coding with hands-on-example 27:24
- Dataloaders in LLM Classification Finetuning | Python Coding | Hands on LLM project 31:03
- Coding the model architecture for LLM classification fine-tuning 34:44
- Coding a fine-tuned LLM spam classification model | From Scratch 49:39
- Introduction to LLM Instruction Fine-tuning | Loading Dataset | Alpaca Prompt format 25:32
- Data Batching in LLM instruction fine-tuning | Hands on project | Live Python coding 52:02
- Dataloaders in Instruction Fine-tuning 24:25
- Instruction fine-tuning: Loading pre-trained LLM weights 19:29
- LLM fine-tuning training loop | Coded from scratch 23:47
- Evaluating fine-tuned LLM using Ollama 52:58
- Build LLMs from scratch 20 minutes summary 19:18
Take this course with you
Download the app to continue lessons from your phone anytime.
