100 Days of Deep Learning - SGD with Momentum Explained in Detail with Animations | Optimizers in Deep Learning Part 2

In this video, we will understand in detail what is Momentum Optimizer in Deep Learning. Momentum Optimizer in Deep Learning is a technique that reduces the time taken to train a model. The path of learning in mini-batch gradient descent is zig-zag, and not straight. Thus, some time gets wasted in moving in a zig-zag direction. Momentum Optimizer in Deep Learning smooths out the zig-zag path and makes it much straighter, thus reducing the time taken to train the model. Momentum Optimizer uses Exponentially Weighted Moving Average, which averages out the vertical movement and the net movement is mostly in the horizontal direction. Thus zig-zag path becomes straighter. Notes: https://learnwith.campusx.in/s/store/courses/YouTube%20Notes Convex Vs Non-convex Cost Function: https://www.youtube.com/watch?v=TXVtbgaEyms&t=1s&ab_channel=CampusX ============================ Do you want to learn from me? Check my affordable mentorship program at : https://learnwith.campusx.in ============================ 📱 Grow with us: CampusX' LinkedIn: https://www.linkedin.com/company/campusx-official CampusX on Instagram for daily tips: https://www.instagram.com/campusx.official My LinkedIn: https://www.linkedin.com/in/nitish-singh-03412789 Discord: https://discord.gg/PsWu8R87Z8 👍If you find this video helpful, consider giving it a thumbs up and subscribing for more educational videos on data science! 💭Share your thoughts, experiences, or questions in the comments below. I love hearing from you! ⌚Time Stamps⌚ 00:00 - Intro 01:24 - Understanding Graphs 07:13 - Image Representation 10:41 - Convex vs Non-Convex Optimization 18:13 - Momentum Optimization 20:49 - The What? 24:23 - How to implement the concept Mathematically 28:41 - Effect of Beta 34:33 - Problems with Momentum Optimization 35:54 - Visualization 38:04 - Outro