This project pulls historical and forecast weather data for multiple cities, cleans and transforms it, performs quality checks, and stores the results in tidy daily and monthly summary datasets.
-
Updated
Nov 23, 2025 - Python
#
incremental-loading
Here are 2 public repositories matching this topic...
Designed a production-grade Azure Data Engineering project centered on Azure Data Factory. Built dynamic, metadata-driven pipelines to ingest data from on-prem systems, REST APIs, and Azure SQL into ADLS Gen2 using Medallion Architecture, incremental loading, and enterprise-scale orchestration patterns.
data-engineering data-platform watermarking azure-data-lake azure-data-factory etl-pipeline data-orchestration analytics-engineering self-hosted-integration-runtime azure-data-engineering medallion-architecture metadata-driven-pipelines adls-gen2 incremental-loading
-
Updated
Jan 13, 2026
Improve this page
Add a description, image, and links to the incremental-loading topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the incremental-loading topic, visit your repo's landing page and select "manage topics."