---

Note

Project Status Update 🚀

After several years of inactivity, we’re excited to announce that Kale development has restarted! 🎉 Kale was widely appreciated by the community back in the day, and our current goal is to re-establish a solid baseline by updating all components to the latest versions and ensuring full compatibility with the most recent Kubeflow releases.

See all details in the Road to 2.0 issue

KALE (Kubeflow Automated pipeLines Engine) is a project that aims at simplifying the Data Science experience of deploying Kubeflow Pipelines workflows.

Kubeflow is a great platform for orchestrating complex workflows on top of Kubernetes, and Kubeflow Pipelines provide the means to create reusable components that can be executed as part of workflows. The self-service nature of Kubeflow makes it extremely appealing for Data Science use, at it provides an easy access to advanced distributed jobs orchestration, re-usability of components, Jupyter Notebooks, rich UIs and more. Still, developing and maintaining Kubeflow workflows can be hard for data scientists, who may not be experts in working orchestration platforms and related SDKs. Additionally, data science often involve processes of data exploration, iterative modelling and interactive environments (mostly Jupyter notebook).

Kale bridges this gap by providing a simple UI to define Kubeflow Pipelines workflows directly from your JupyterLab interface, without the need to change a single line of code.

See the Kale v2.0 Demo video at the bottom of the README for more details.

Read more about Kale and how it works in this Medium post: Automating Jupyter Notebook Deployments to Kubeflow Pipelines with Kale

Getting started

Requirements

• Python 3.10+
• Kubeflow Pipelines v2.4.0+ - Install as recommended in the official Kubeflow Pipelines Installation documentation
• A Kubernetes cluster (minikube, kind, or any K8s cluster)

Installation

Important

Kale v2.0 is not yet released on PyPI. Until then, install from source:

git clone https://github.com/kubeflow-kale/kale.git
cd kale
make dev      # Set up development environment
make jupyter  # Start JupyterLab

See CONTRIBUTING.md for detailed setup instructions.

Once v2.0 is released, you'll be able to install from PyPI:

pip install "jupyterlab>=4.0.0" kubeflow-kale kubeflow-kale-labextension
jupyter lab

Verify installation

1. Start your Kubernetes cluster and KFP:

   minikube start
   kubectl port-forward -n kubeflow svc/ml-pipeline-ui 8080:80

2. Test the CLI:

   kale --nb examples/base/candies_sharing.ipynb --kfp_host http://127.0.0.1:8080 --run_pipeline

   This generates a pipeline in .kale/ and submits it to KFP.

3. Test the JupyterLab extension:

   • Open JupyterLab (make jupyter or jupyter lab)
   • Open a notebook from examples/base/
   • Click the Kale icon in the left panel
   • Enable the Kale panel with the toggle

FAQ

To build images to be used as a NotebookServer in Kubeflow, refer to the Dockerfile in the docker folder.

Head over to FAQ to read about some known issues and some of the limitations imposed by the Kale data marshalling model.

Resources

• Kale introduction blog post
• Codelabs showcasing Kale working in MiniKF with Arrikto's Rok:
  • From Notebook to Kubeflow Pipelines
  • From Notebook to Kubeflow Pipelines with HP Tuning
• KubeCon NA Tutorial 2019: From Notebook to Kubeflow Pipelines: An End-to-End Data Science Workflow / video
• CNCF Webinar 2020: From Notebook to Kubeflow Pipelines with MiniKF & Kale / video
• KubeCon EU Tutorial 2020: From Notebook to Kubeflow Pipelines with HP Tuning: A Data Science Journey / video

Contribute

make dev      # Set up development environment
make test     # Run all tests
make jupyter  # Start JupyterLab

See CONTRIBUTE.md for detailed development instructions, including:

• Available make commands
• Testing with KFP clusters
• Building release artifacts
• Live reload setup

Kale v2.0 Demo

Watch the KubeFlow Kale Demo - Introduction video below.