34 Write PySpark Unit Test Cases using PyTest module | Setup PyTest with PySpark

PySpark - Zero to Hero | PySpark Tutorial 2025 | Spark Tutorial 2025 | Learn from Basics to Advanced Performance Optimization - 34 Write PySpark Unit Test Cases using PyTest module | Setup PyTest with PySpark

4.0 (1)

18 learners

What you'll learn

This course includes

9 hours of video
Certificate of completion
Access on mobile and TV

Summary

Keywords

PySpark Unit Test Case using PyTest Spark Unit Test cases with PyTest

Full Transcript

Unit Test Cases in PySpark using PyTest | PySpark Tutorial 2026 | Spark Training Video explains - How to write Unit Test Cases in PySpark using PyTest Module? How to use PyTest Module? What are fixtures in PyTest? How to integrate PyTest with PySpark? How to integrate Unit Test Cases of PySpark using PyTest on CICD? Chapters 000:00 - Introduction 00:50 - Setup PyTest Module 02:25 - How to write Unit Tests in PySpark using PyTest? 15:19 - Running Test Cases using PyTest - PySpark Playlist - https://www.youtube.com/playlist?list=PL2IsFZBGM_IHCl9zhRVC1EXTomkEp_1zm - Databricks Playlist - https://www.youtube.com/playlist?list=PL2IsFZBGM_IGiAvVZWAEKX8gg1ItnxEEb - Medium Blog - https://blog.devgenius.io/pyspark-unit-test-cases-using-pytest-3199f78c3471 - GitHub URL for docker images for cluster setup - https://github.com/subhamkharwal/docker-images Connect with me - https://www.linkedin.com/in/subhamkharwal The series provides a step-by-step guide to learning PySpark, a popular open-source distributed computing framework that is used for big data processing.