Data Engineering on Microsoft Azure - Azure for DE: 49 - Introduction to streaming, Event Hubs

Hey Data Engineers! Tired of the same old batch processing? Let's dive into something more exciting—streaming and real-time data processing! In the 49th episode of my free DP-203 course, I'm exploring streaming, sharing real-world examples, outlining a high-level architecture, and taking a deep dive into the ingestion phase using Azure Event Hubs. Enjoy! ▬▬▬▬▬▬ IMPORTANT LINKS ▬▬▬▬▬▬ My LinkedIn profile: https://www.linkedin.com/in/piotr-tybulewicz-81a8793/ GitHub with my drawings: https://github.com/TybulOnAzure/DP-203 Rasberry PI simulator: https://azure-samples.github.io/raspberry-pi-web-simulator/ Real Time Data Simulator by Kamil Nowiński: https://github.com/Azure-Player/Real-Time-Data-Simulator ▬▬▬▬▬▬ MEMBERSHIP ▬▬▬▬▬▬ Join this channel to get access to perks: https://www.youtube.com/channel/UCLnXq-Fr-6rAsCitq9nYiGg/join ▬▬▬▬▬▬ CHAPTERS ▬▬▬▬▬▬ 00:00 Introduction 00:18 Streaming vs batch processing 06:08 Streaming in action 11:43 General architecture 16:19 Ingest part 19:04 Rasberry PI simulator 21:34 Event hubs 48:43 Summary