In this project, we setup and end to end data engineering using Apache Spark, Azure Databricks, Data Build Tool (DBT) using Azure as our cloud provider.
-
Updated
Dec 18, 2023
In this project, we setup and end to end data engineering using Apache Spark, Azure Databricks, Data Build Tool (DBT) using Azure as our cloud provider.
Extract Load Transform (ELT) framework is a metadata based batch orchestration framework for modern data platforms. Implemented using Azure PaaS data services. Common ingestion and transformation patterns available out of box. Reusable code can be easily extended to cater to custom patterns.
Add a description, image, and links to the modern-data-engineering topic page so that developers can more easily learn about it.
To associate your repository with the modern-data-engineering topic, visit your repo's landing page and select "manage topics."