Data Engineering on Microsoft Azure
Master Azure Data Engineering: From Ingestion to Analytics with Real-World Projects
4.0
(5)
45 learners
What you'll learn
- Design and implement a data ingestion pipeline using Azure Data Factory
- Apply security and access controls to an Azure Data Lake Storage account
- Transform raw data into a modeled structure using Azure Databricks or Synapse Spark
- Deploy and manage Azure Data Factory pipelines using CI/CD practices
This course includes
- 49.5 hours of video
- Certificate of completion
- Access on mobile and TV
Course content
1 modules • 58 lessons • 49.5 hours of video
Azure Data Engineering: End-to-End Cloud ETL & Analytics Course
58 lessons
• 49.5 hours
Azure Data Engineering: End-to-End Cloud ETL & Analytics Course
58 lessons
• 49.5 hours
- Azure for DE: 01 - Introduction 26:45
- Azure for DE: 02 - Brief history of data architectures 48:45
- Azure for DE: 03 - Storage account overview 01:09:39
- Azure for DE: 04 - Storage account redundancy 40:33
- Azure for DE: 05 - Storage account access tiers 40:55
- Azure for DE: 06 - Common file types (CSV, XML, JSON) 52:40
- Azure for DE: 07 - Common file types (PARQUET) 44:12
- Azure for DE: 08 - Common file types (Delta lake) 59:18
- Azure for DE: 09 - Data lake structure - Raw layer 47:53
- Azure for DE: 10 - Introduction to Azure Data Factory 50:03
- Azure for DE: 11 - Dynamic Azure Data Factory 57:29
- Azure for DE: 12 - Azure Data Factory integration runtimes (part 1) 45:55
- Azure for DE: 13 - Azure Data Factory integration runtimes (part 2) 34:58
- Azure for DE: 14 - Error handling in Azure Data Factory 01:18:42
- Azure for DE: 15 - Questions to ask about new data source 46:41
- Azure for DE: 16 - Executing ADF pipelines 57:21
- Azure for DE: 17 - Azure data lake security - anonymous access, access keys 45:17
- Azure for DE: 18 - Azure data lake security - Azure Key Vault and managed identity 47:48
- Azure for DE: 19 - Azure data lake security - Shared Access Signature (SAS) 47:40
- Azure for DE: 20 - Azure data lake security - Role-Based Access Control (RBAC) 51:45
- Azure for DE: 21 - Azure data lake security - Access Control Lists (ACL) 42:43
- Azure for DE: 22 - ARM-based CI/CD for Azure Data Factory (part 1) 59:21
- Azure for DE: 23 - ARM-based CI/CD for Azure Data Factory (part 2) 55:23
- Azure for DE: 24 - Selective CI/CD for Azure Data Factory with ADF Tools (part 1) 49:38
- Azure for DE: 25 - Selective CI/CD for Azure Data Factory with ADF Tools (part 2) 49:45
- Azure for DE: 26 - Introduction to Azure Logic Apps 46:54
- Azure for DE: 27 - Azure Synapse Analytics: Overview, Pipelines 01:07:55
- Azure for DE: 28 - 1st milestone (ingest phase recapped + challenge) 39:05
- Channel membership announcement 11:18
- Azure for DE: 29 - Introduction to dimensional modeling 55:35
- Azure for DE: 30 - Slowly changing dimensions, data lake structure 49:00
- Azure for DE: 31 - Introduction to Azure Databricks 01:06:25
- Azure for DE: 32 - A closer look at Databricks notebooks 01:15:00
- Azure for DE: 33 - Connecting to ADLSg2 from Azure Databricks 59:34
- Azure for DE: 34 - Common data transformations 01:05:15
- Azure for DE: 35 - Writing data to ADLSg2 from Azure Databricks 47:15
- Azure for DE: 36 - Automating the process with Azure Databricks Autoloader 45:05
- Azure for DE: 37 - Orchestrating Databricks notebooks 01:02:49
- Azure for DE: 38 - Transforming data with dbt 47:13
- Azure for DE: 39 - Azure Synapse Analytics - Spark Pools 53:18
- Azure for DE: 40 - Extra features in Azure Synapse Analytics Spark Pools 58:52
- Azure for DE: 41 - Transforming data with Data Flows 58:43
- Azure for DE: 42 - 2nd milestone - transform phase recapped 31:47
- Azure for DE: 43 - Azure Synapse Dedicated SQL Pool - Architecture and overview 01:13:36
- Azure for DE: 44 - Loading data to Dedicated SQL Pool 01:20:37
- Azure for DE: 45 - Azure Synapse Dedicated SQL Pool - extra features 01:02:40
- Azure for DE: 46 - Dedicated SQL Pool security 49:57
- Azure for DE: 47 - Azure Synapse Analytics - Serverless SQL Pool 59:40
- Azure for DE: 48 - 3rd milestone - Serve phase recapped 32:15
- Azure for DE: 49 - Introduction to streaming, Event Hubs 49:45
- Azure for DE: 50 - Azure Stream Analytics overview 01:36:47
- Azure for DE: 51 - Handling time & performance in Azure Stream Analytics 50:26
- Azure for DE: 52 - Data governance with Microsoft Purview 55:10
- Azure for DE: 53 - 4th milestone - Streaming and data governance recapped 30:30
- Azure for DE: 54 - Exam overview and course summary 38:07
- Azure for DE: The universal pattern for incremental data processing 39:34
- Azure for DE: Processing new and changed data with MERGE 38:44
- Data lineage in practice: Unity Catalog and Microsoft Purview 39:23
Related learning topics
Take this course with you
Download the app to continue lessons from your phone anytime.
![[DP-100] Azure Data Scientist](https://i.ytimg.com/vi/giPNL3WKC_I/hqdefault.jpg)
![Exam DP-900: Microsoft Azure Data Fundamentals Course Playlist [20 Videos]](https://i.ytimg.com/vi/iyloGFjdlm0/hqdefault.jpg)
![Ultimate Data Analyst Bootcamp [24 Hours!] for FREE | SQL, Excel, Tableau, Power BI, Python, Azure](https://i.ytimg.com/vi/wQQR60KtnFY/hqdefault.jpg)