The Fabric Metadata‑Driven Framework (FMD) is a powerful and extensible Microsoft Fabric framework designed to automate, orchestrate, and standardize metadata‑driven data pipelines. Built for lakehouse‑first architectures, FMD provides a configurable and reusable approach to ingestion, transformation, governance, and automation across Microsoft Fabric.
Modern data platforms demand agility, scalability, and consistency. FMD simplifies these challenges by enabling:
- Dynamic, metadata‑driven pipelines
- Consistent orchestration across ingestion, processing, and publishing
- Centralized configuration for all data entities
- Alignment with Microsoft Fabric Lakehouse & Medallion Architecture
- Reduced engineering effort through reusable patterns
- Faster delivery with standardized, tested components
The FMD Framework enables organizations to streamline data operations by leveraging metadata to drive dynamic data pipelines and parameterized notebooks. Built on Fabric SQL Database, the framework supports secure, flexible, and modern data management at scale.
Tip
The FMD Framework is designed for rapid deployment and extensibility. You can use it out-of-the-box or customize it to meet your organization's evolving data needs.
Watch this conversation with the Azure Data Factory team for a walkthrough of the concepts described in 🚀 Why the FMD Framework?.
Automatically adjusts pipeline execution based on metadata—ideal for large‑scale, multi‑source environments.
Modular design allows custom logic, new transformations, and custom patterns without breaking existing workloads.
Track rows processed, load statuses, timestamps, and operational metrics through centralized metadata.
The FMD Framework is built on top of core Microsoft Fabric capabilities:
The FMD Framework uses a modular architecture that separates data, code, and orchestration for enhanced security and manageability.
- Workspace architecture
- Medallion architecture
- Supported data sources
- Data Cleansing
- Logging and auditing
- Variable Library
- Data Pipelines and Notebooks
- Taskflow orchestration
- Business Domains
To get started:
- Review the FMD Framework Deployment Guide.
- Set up the required connections in your Fabric environment.
- Configure the deployment parameters as per your environment.
- Deploy the FMD Framework using the provided deployment scripts.
- Import the taskflow and configure your workspaces as recommended.
- Refer to wiki for data model, pipelines, and logging.
Review the Business Domain Deployment Framework Deployment Guide.
| Resource | Description |
|---|---|
| FMD Integration Framework reference | Overview on how to add sources and demo data to the FMD Framework |
| FMD Data Model reference | Overview of the data model used in the FMD Framework |
| Configure and load demo data | Instructions for loading demo data into the FMD Framework |
Full overview & tutorials: https://erwindekreuk.com/fmd-framework/ | FMD Integration Framework reference | Overview on how to add sources and demo data to the FMD Framework |
- Enterprise‑grade ingestion frameworks
- Reusable ingestion + transformation patterns
- Standardized data architecture for Fabric Lakehouse
- Multi‑source data ingestion using dynamic pipelines
- Rapid onboarding of new datasets
- Migration from Synapse or ADF metadata‑driven solutions
Contributions, enhancements, and feature requests are welcome. To contribute:
- Fork the repository.
- Create a feature branch.
- Commit your changes.
- Push to the feature branch.
- Create a pull request and add documentation on what you have changed.
📝 License Contributions, enhancements, and feature requests are welcome. To contribute:
Contributors:
Erwin de Kreuk
This project is released under the MIT License. See the LICENSE file for details.