100 Days of Deep Learning - Layer Normalization in Transformers | Layer Norm Vs Batch Norm

Layer Normalization is a technique used to stabilize and accelerate the training of transformers by normalizing the inputs across the features. It adjusts and scales the activations, ensuring consistent output distributions. This helps in reducing training time and improving model performance, making it a key component in transformer architectures. Notes: https://learnwith.campusx.in/s/store/courses/YouTube%20Notes ============================ Did you like my teaching style? Check my affordable mentorship program at : https://learnwith.campusx.in DSMP FAQ: https://docs.google.com/document/d/1OsMe9jGHoZS67FH8TdIzcUaDWuu5RAbCbBKk2cNq6Dk/edit#heading=h.gvv0r2jo3vjw ============================ 📱 Grow with us: CampusX' LinkedIn: https://www.linkedin.com/company/campusx-official CampusX on Instagram for daily tips: https://www.instagram.com/campusx.official My LinkedIn: https://www.linkedin.com/in/nitish-singh-03412789 Discord: https://discord.gg/PsWu8R87Z8 E-mail us at [email protected] ✨ Hashtags✨ #deeplearning #campusx #transformers #transformerarchitechture ⌚Time Stamps⌚ 00:00 - Intro 02:20 - What is Normalization 03:50 - What do we normalize? 05:30 - Benefits of Normalization in DL 07:10 - Internal Covariate Shift 12:49 - Batch Normalization Revision 22:56 - Why don't we use Batch Norm in Transformers? 38:25 - How does Layer Normalization works? 43:00 - Layer Normalization in Transformer