Summary
Keywords
Full Transcript
Video covers - How to handle Late Arrival of Data ? What is the difference between Event and Processing Time? How Spark handles Stateful Processing ? What are Watermarks in Spark Streaming ? Chapters: 00:00 - Introduction 00:29 - Event Time vs Processing Time 02:34 - Stateful Processing 03:38 - How Spark handles Late Data ? URLs: Github Code - https://github.com/subhamkharwal/spark-streaming-with-pyspark/ Device data samples - https://github.com/subhamkharwal/spark-streaming-with-pyspark/blob/master/datasets/devices/device_data_samples.txt To setup Kafka with Spark in Local environment - https://youtu.be/BeZRe0f23hI JSON data Flattening and reading from files - https://youtu.be/wp_2CXyXV0w Keywords: Apache Spark, PySpark, Spark Streaming, Real-time Data Processing, Data Streaming, Big Data Analytics, PySpark Tutorial, Apache Spark Tutorial, Streaming Analytics, Spark Structured Streaming, PySpark Streaming, Big Data Processing. New video in every 3 days ❤️ Make sure to like and Subscribe.
