Spark Streaming with PySpark | Structured Streaming Tutorial 2025 - 03 Spark Streaming Local Environment Setup - Docker, Jupyter, PySpark and Kafka

Video covers - How to setup Spark Streaming environment in Local Machine using Docker? Chapters: 00:00 - Introduction 00:40 - Docker Desktop Installation 01:14 - Github clone 01:43 - Docker Setup 03:02 - PySpark Jupyter Lab Setup URLs: Docker Desktop: https://www.docker.com/products/docker-desktop/ Github URL: https://github.com/subhamkharwal/docker-images Jupyter Lab URL: http://localhost:8888/ Spark UI URL: http://localhost:4040/ Kafka bootstrap server URL: ed-kafka:9092 Docker command: docker compose up Kafka command: To view all topics: kafka-topics --list --bootstrap-server ed-kafka:9092 To create a new topic: kafka-topics --create --topic test-topic --bootstrap-server ed-kafka:9092 Keywords: Apache Spark, PySpark, Spark Streaming, Real-time Data Processing, Data Streaming, Big Data Analytics, PySpark Tutorial, Apache Spark Tutorial, Streaming Analytics, Spark Structured Streaming, PySpark Streaming, Big Data Processing. New video in every 3 days ❤️ Make sure to like and Subscribe.