Summary
Keywords
Full Transcript
#spark #pyspark #pysparkinterview #dataengineer Scenario based pyspark interview question 1. group multiple rows in single using pyspark 2. combine list and convert into dataframe using pyspark Want more similar videos- hit like, comment, share and subscribe ❤️Do Like, Share and Comment ❤️ ❤️ Like Aim 5000 likes! ❤️ ➖➖➖➖➖➖➖➖➖➖➖➖➖ Please like & share the video. ➖➖➖➖➖➖➖➖➖➖➖➖➖ data 1. data=[(1,'Manish','Mobile'),(1,'Manish','Washing Mavhine'),(2,'Rahul','Car'),(2,'Rahul','mobile'),(2,'Rahul','scooty'),(3,'Monu','Scooty')] schema=["Customer_ID", "Customer_Name",'Purchase'] df = spark.createDataFrame(data,schema) df.show() 2. list1 = ["a", "b", "c", "d"] list2 = [1, 2, 3, 4] rdd = spark.sparkContext.parallelize(list(zip(list1,list2))) df=rdd.toDF(['column1','column2']) df.show() ➖➖➖➖➖➖➖➖➖➖➖➖➖ AWS DATA ENGINEER : https://www.youtube.com/playlist?list=PLOlK8ytA0MghpdMjb0m9zu1v9s_qbRP0q Azure data factory : https://youtube.com/playlist?list=PLOlK8ytA0MgguN5XidtQXbILxwCdJCUJE&si=iEICXHP80zGchUYh Azure data engineer playlist : https://youtube.com/playlist?list=PLOlK8ytA0MghBrzu0i6WlTBdoO1WdwV_e SQL PLAYLIST : https://www.youtube.com/playlist?list=PLOlK8ytA0MggGXIKmbfZ-_Xqcos3JKdV- PYSPARK PLAYLIST - https://www.youtube.com/playlist?list=PLOlK8ytA0MgjvOpd-088NRU_vTYw_aryH ➖➖➖➖➖➖➖➖➖➖➖➖➖ 📣Want to connect with me? Check out these links:📣 Join telegram to discuss https://t.me/+Cb98j1_fnZs3OTA1 ➖➖➖➖➖➖➖➖➖➖➖➖➖ what we have covered in this video: ➖➖➖➖➖➖➖➖➖➖➖➖➖ Hope you liked this video and learned something new :) See you in next video, until then Bye-Bye! ➖➖➖➖➖➖➖➖➖➖➖➖➖
Continue this lesson in the app
Install CourseHive on Android or iOS to keep learning while you move.
