End-to-end Metadata-Driven Data Engineering framework built on Azure. Features dynamic SQL/REST API ingestion with range pagination, automated schema mapping, and event-driven orchestration. Implements robust CI/CD via GitHub Actions/YAML and automated failure alerting with Logic Apps. Optimized for scalability and DE best practices.
rest-api data-engineering cicd azure-sql-database azure-data-factory etl-pipeline data-engineering-pipeline azure-logic-apps data-orchestration yaml-templating data-lifecycle-management etl-developer metadata-driven-pipelines cloud-data-engineering adls-gen2 azure-data-engineer incremental-load
-
Updated
Jan 16, 2026