Building a High Performance Real-Time Analytics Database - End to End Data Engineering Project

Google Cloud End to End Data Engineering Projects - Building a High Performance Real-Time Analytics Database - End to End Data Engineering Project

Master End-to-End Data Engineering: Real-Time Streaming, AI Integrations, and High-Performance Systems! Dive into hands-on projects, expert-guided tutorials, and cutting-edge technologies for a standout career in data engineering.

4.0 (2)

25 learners

What you'll learn

Understand and implement real-time streaming with Google Cloud for data engineering projects
Learn how to perform real-time socket streaming using Apache Spark
Master the use of Apache Airflow alongside Spark, Pyspark, Java, and Scala for data engineering
Develop skills to build and optimize high-performance, real-time analytics databases

This course includes

47.5 hours of video
Certificate of completion
Access on mobile and TV

Summary

Keywords

Data Engineering Apache Kafka Apache Druid Real-time Analytics Apache Superset Docker Orbstack High-Performance Databases Big Data Streaming Data Data Processing Open Source

Full Transcript

In this video you will be building a High Performance Real-time Analytics Database using state of the art tools in the Apache Ecosystem like Apache Kafka, Apache Druid, Apache Superset, Docker, and Orbstack. FOR MORE DATA ENGINEERING COURSES: datamasterylab.com 📚 What You'll Learn: 👉 Understand Apache Frameworks for Data Engineering 👉 Streaming data into Apache Kafka 👉 Using Zookeeper for distributed synchronization 👉 Data processing with Apache Druid 👉 Data storage and Realtime Aggregations with Apache Druid 👉 Containerising your data engineering environment with Docker ✨ Timestamps: ✨ 0:00 Introduction 1:47 List of Apache Frameworks for Data Engineering 3:20 System Architecture 10:36 Starting up a project from scratch 13:18 Setting up the containers and services on Docker 28:10 Streaming data into Apache Kafka 42:35 Apache Druid Walkthrough 48:57 Connecting Apache Druid to Apache Kafka 1:00:04 Realtime Queries and Aggregations on Apache Druid 1:07:34 Time Aggregations on Apache Druid 1:09:32 Outro 👦🏻 My Linkedin: https://www.linkedin.com/in/yusuf-ganiyu-b90140107/ 🚀 Twitter: https://twitter.com/YusufOGaniyu 📝 Medium: https://medium.com/@yusuf.ganiyu 🌟 Please LIKE ❤️ and SUBSCRIBE for more AMAZING content! 🌟 Like this video? Buy me a coffee ❤️ https://www.buymeacoffee.com/yusuf.ganiyu/ 🔗 Useful Links and Resources: 👉 Full Source Code: buymeacoffee.com/yusuf.ganiyu/full-source-code-building-high-performance-realtime-analytics-database 👉 Apache Druid Documentation: https://druid.apache.org/docs/latest/design/ 👉 Apache Superset Documentation: https://superset.apache.org/docs/ 👉 Docker Documentation: https://docs.docker.com/ 👉 Orbstack Documentation: https://orbstack.dev/docs ✨ Tags ✨ Data Engineering, Apache Kafka, Apache Druid, Real-time Analytics, Apache Superset, Docker, Orbstack, High-Performance Databases, Big Data, Streaming Data, Data Processing, Open Source, Data Aggregation ✨ Hashtags ✨ #DataEngineering #ApacheKafka #ApacheDruid #RealTimeAnalytics #ApacheSuperset #Docker #BigData #DataScience #StreamingData #OpenSource #DataAggregation #Orbstack #HighPerformanceDatabases #bigdatatechnologies

Continue this lesson in the app

Install CourseHive on Android or iOS to keep learning while you move.

Related Courses

30-Day Beginner Guitar Challenge for New Players

Master the Guitar in 30 Days: Your Ultimate Beginner Challenge! Unleash your inner guitarist with step-by-step lessons designed to transform you from novice to confident player. Join Your Guitar Academy and kickstart your musical journey today!

⭐ 4.3

36 ratings

7 hours