AI Workflow Engine (MVP) Goal: Orchestrate ML workflow → ingest → preprocess → train → evaluate → save artifacts, with reproducible runs.

Tech Stack: Prefect (orchestration)

MLflow (experiment tracking)

Docker (containerize train job)

Python 3.11

Project Structure: ai-workflow/ flows/ train_flow.py # Prefect Flow - ingest → preprocess → train → evaluate artifacts/ # model + metrics output saved here mlflow/ # local MLflow tracking store Dockerfile README.md .gitignore

1. Setup (Local): cd "D:\Day 10\ai-workflow"

create virtual env if not created

py -3.11 -m venv .venv

activate

..venv\Scripts\Activate.ps1

install dependencies

pip install prefect==2.19.9 mlflow==2.16.0 scikit-learn pandas numpy joblib

2. Run Pipeline: ..venv\Scripts\Activate.ps1 python .\flows\train_flow.py

Outputs:

./artifacts/model_.joblib

./artifacts/metrics_.json

MLflow run created inside ./mlflow

3. MLflow UI: Always run UI via python module (so correct venv is used) Get-Process -Name python,uvicorn -ErrorAction SilentlyContinue | Stop-Process -Force

cd "D:\Day 10\ai-workflow" ..venv\Scripts\Activate.ps1

python -m mlflow server --backend-store-uri file:./mlflow --host 127.0.0.1 --port 5005 open → http://127.0.0.1:5005

4. Docker (optional - later) docker build -t ai-workflow:latest . docker run --rm -it -v "${PWD}\artifacts:/app/artifacts" ai-workflow:latest

Next Steps (Future):

Add deploy step → package model as wheel or FastAPI app

Add monitoring step (Prometheus + Grafana)

Turn pipeline into scheduled Cron based Prefect deployments

Status: MVP training pipeline working end-to-end.