ZenML on Juju with Microk8s

DISCLAIMER: This project was inspired by the Charmed MLFlow Project and also implements solutions found there.

• Get Started

  • Prerequisites
  • Install and prepare Juju
  • Deploy Sandalone ZenML Server Bundle
  • Build and deploy the charm manually

• Integrate ZenML Server with Charmed Kubeflow

• Ingress

• Examples

Get started

Prerequisites

We are assuming that you are running this tutorial on a local machine or a EC2 instance with the following specs:

• Runs Ubuntu 22.04 or later

• Has at least 50GB free disk space

Install and prepare MicroK8s

Install MicroK8s from a snap package

sudo snap install microk8s --classic --channel=1.28/stable

Add the current user to the microk8s group and generate configuration directory for kubectl

sudo usermod -a -G microk8s $USER
newgrp microk8s
sudo chown -f -R $USER ~/.kube
cd $HOME
mkdir .kube
cd .kube
microk8s config > config

Enable the following MicroK8s addons to configure your Kubernetes cluster with extra services needed to run Charmed Kubeflow.

microk8s enable dns hostpath-storage ingress metallb:10.64.140.43-10.64.140.49

Wait until the command

microk8s status --wait-ready

Returns

microk8s is running

NOTE: To use the MicroK8s built-in registry in the ZenML stack, please refer to the guide

Install and prepare Juju

To install Juju from snap, run this command:

sudo snap install juju --classic --channel=3.1/stable

On some machines there might be a missing folder which is required for juju to run correctly. Because of this please make sure to create this folder with:

mkdir -p ~/.local/share

Deploy a Juju controller to the Kubernetes we set up with MicroK8s:

microk8s config | juju add-k8s my-k8s --client
juju bootstrap my-k8s uk8sx

Create a model

juju add-model <model name>

Deploy Sandalone ZenML Server Bundle

To deploy the ZenML bundle run:

juju deploy zenml --trust

Run juju status --watch 2s to observe the charm deployment and after all the apps reach the Active status, the ZenML Server dashboard should be accessible as a NodePort under

http://localhost:31375/

With the default settings for username = default and no password.

To connect the zenml SDK to it run

zenml connect --uri http://localhost:31375/ --username default --password ''

Build and deploy the charm manually

Install dependencies

sudo snap install charmcraft --classic

Create ZenML Charm

charmcraft pack

This step will generate a charm file zenml-server_ubuntu-20.04-amd64.charm

Deploy the ZenML server charm

juju deploy ./zenml-server_ubuntu-20.04-amd64.charm zenml-server \
    --resource oci-image=$(yq '.resources."oci-image"."upstream-source"' metadata.yaml)

Deploy mysql-k8s to be used as ZenML Server backend and create relation

juju deploy mysql-k8s zenml-mysql --channel 8.0/stable

juju relate zenml-server zenml-mysql

Run juju status --watch 2s to observe the charm deployment and after all the apps reach the Active status, the ZenML Server dashboard should be accessible as a NodePort under

http://localhost:31375/

With the same credentials

Integrate ZenML Server with Charmed Kubeflow

To use Charmed Kubeflow as a orchestrator for a ZenML stack some configurations have to be done:

To install Charmed Kubeflow follow this guide

If running Charmed Kubeflow on a EC2 instance, configure the istio-ingress-gateway service type to NodePort:

kubectl -n kubeflow patch svc istio-ingressgateway-workload \
 -p '{"spec":{"type":"NodePort"}}'

And reconfigure dex and oidc-gatekeeper to point on the instance public IP:

export NODE_IP=<the instance public IP>

export NODE_PORT=$(kubectl -n kubeflow get svc istio-ingressgateway-workload -o=json | \
 jq '(.spec.ports) | .[] | select(.name=="http2") | (.nodePort)')

export PUBLIC_URL="http://${NODE_IP}:${NODE_PORT}"

juju config dex-auth public-url=${PUBLIC_URL}
juju config oidc-gatekeeper public-url=${PUBLIC_URL}

Set authentication methods:

juju config dex-auth static-username="<your username>"
juju config dex-auth static-password="<your password>"

And make sure the security group allows ingress to the instance on CIDR of echo "${NODE_IP}/32" on the echo $NODE_PORT port

After this the Kubeflow Dashboard will be accessible under the value of echo $PUBLIC_URL

The pipeline API, required for configuring the Kubeflow Orchestrator for ZenML will have the value of:

echo "${PUBLIC_URL}/pipeline/"

Configuring Kubeflow Pipelines as a orchestrator for your ZenML workload also requires setting up an Artifact Store.

For this tutorial we will use already deployed minio service and expose it as NodePort

kubectl -n kubeflow patch svc minio \
 -p '{"spec":{"type":"NodePort"}}'

After running kubectl get svc -n kubeflow, we should find the minio service configuration:

NAME                                  TYPE           CLUSTER-IP       EXTERNAL-IP                        PORT(S)
.
.
.
minio                                 NodePort       10.152.183.227   <none>                             9000:<API port>/TCP,9001:<Dashboard port>/TCP

Now the MinIO dashboard should be accessible under:

http://localhost:<Dashboard port>

Or, when running on a EC2 with ingress configured to allow both <Dashboard port> and <API port> from your IP and the public IP of the machine:

echo "${PUBLIC_URL}:<Dashboard port>"

The MINIO_ACCESS_KEY and MINIO_SECRET_KEY in base64 can be received form Kubernetes Secrets:

kubectl get secrets minio-secret -n kubeflow -o yaml

Before using them, remember to decode the.

ZenML will require a specified bucket so use the dashboard to create one and the artifact store can be configured with:

# Store the AWS access key in a ZenML secret
zenml secret create s3_secret \
    --aws_access_key_id='<DECODED_MINIO_ACCESS_KEY>' \
    --aws_secret_access_key='<DECODED_MINIO_SECRET_KEY>'

# Register the MinIO artifact-store and reference the ZenML secret
zenml artifact-store register minio_store -f s3 \
    --path='s3://<minio_bucket>' \
    --authentication_secret=s3_secret \
    --client_kwargs='{"endpoint_url": http://localhost:<API port>}' # or for EC2 <publicIP>:<API port>

When running a ZenML pipeline with Kubeflow Orchestrator, the client will either use the current machine docker client or preconfigured image builder to build the pipeline image and publish it to a registry. By default it is the local registry that comes with installing ZenML, but to allow an remote Kubeflow to use it, a remote registry has to be available.

For this the mentioned MicroK8s built-in registry comes in handy and can be configured with:

zenml container-registry register <NAME> \
    --flavor=default \
    --uri=localhost:32000 # or for EC2 <publicIP>:32000

Ingress

Currently the charmed zenml-server-operator supports ingress integration with Charmed Istio

NOTE: According to ZenML Docs:

This method has one current limitation: the ZenML UI does not support URL rewriting and will not work properly if you use a dedicated Ingress URL path. You can still connect your client to the ZenML server and use it to run pipelines as usual, but you will not be able to use the ZenML UI.

Additionally, the ZenML client does not support DEX auth - this might require configuring the server to use DEX as an external auth provider - more info here

Deploy Istio Gateway and Istio Pilot charms and configure the relation

juju deploy istio-gateway istio-ingressgateway --channel 1.17/stable --config kind=ingress --trust

juju deploy istio-pilot --channel 1.17/stable --config default-gateway=test-gateway -trust

juju relate istio-pilot istio-ingressgateway

To integrate zenml-server with currently deployed istio-operator run the command:

juju relate zenml-server istio-pilot

After this done the ZenML Client can connect to the server over the ingress IP found by running:

microk8s kubectl -n <your namespace> get svc istio-ingressgateway-workload -o jsonpath='{.status.loadBalancer.ingress[0].ip}'

Over the URL http:/<ingress ip>/zenml/

Examples

Check out the examples directory