Learn AI with KodeKloud - Understanding vLLM with a Hands On Demo

5.0 (2)

19 learners

What you'll learn

This course includes

17.5 hours of video
Certificate of completion
Access on mobile and TV

Summary

Keywords

KodeKloud vllm vllm tutorial llm inference llm serving paged attention kv cache llm inference engine serve llm llm production vllm api server llm throughput

Full Transcript

🧪 vLLMs Labs for FREE — https://kode.wiki/4toLSl7 Most people can use an LLM. Very few know how to serve one at scale. This video breaks down vLLM, the inference engine transforming production AI deployments, and shows you exactly why it dominates when it comes to throughput, concurrency, and KV cache efficiency. No fluff. No theory overload. Just clear, hands-on learning starting from why your LLM is slow, all the way to launching a production-ready API server with a live monitoring dashboard. ───────────────────────────────────────── 📌 WHAT YOU'LL LEARN IN THIS VIDEO ───────────────────────────────────────── ✅ What LLM inference is and why tokens per second varies across platforms like ChatGPT & Gemini ✅ Comparison of different inference engines ✅ The KV Cache problem ✅ How PagedAttention works — inspired by OS virtual memory paging ✅ Demo - Build a monitoring dashboard to track throughput, latency & concurrency live 🧪 FREE HANDS-ON LABS INCLUDED — https://kode.wiki/4toLSl7 Practice everything in a real sandbox environment with no local setup, no credit card, no surprises. GPU environment, model weights, and all dependencies are already configured and ready to go. ⏱️ TIMESTAMPS 00:00 – Overview of LLM Inference Engines 00:52 – What Makes vLLM Stand Out 01:48 – How PagedAttention Works 02:31 – Other Inference Engine 03:44 – Lab Intro & Environment Setup 05:21 – Task 1 - Naive HuggingFace Inference 05:58 – Task 2 - vLLM Offline Interference 07:04 – Task 3 - The K Cache problem 07:52 – Task 4 - PageAttention 09:11 – Task 5 - Launch vLLM as an OpenAI-compatible API server 10:08 – Task 6 - Multi-user Throughput under load 11:29 – Task 7 - Tuning vLLM Parameters for Production 12:21 – Task 8 - Capstone (Building a Monitoring Dashboard) 13:54 – Key Takeaways & When to Use vLLM vs Other Engines #vLLM #LLMInference #PagedAttention #KVCache #LLMDeployment #LLMServing #AIEngineering #MLOps #LLMPerformance #HuggingFace #GPUOptimization #LLMTuning #GenAI #AIInfrastructure #LargeLanguageModels #DeepLearning #AIProduction #KodeKloud #LLMOps #MachineLearning #DevOps #CloudAI #AIDevelopment #OpenAI