Skip to content

docs: Add architecture overview and update documentation for 2.0#584

Draft
StefanoFioravanzo wants to merge 3 commits intokubeflow:mainfrom
StefanoFioravanzo:docs/architecture-overview
Draft

docs: Add architecture overview and update documentation for 2.0#584
StefanoFioravanzo wants to merge 3 commits intokubeflow:mainfrom
StefanoFioravanzo:docs/architecture-overview

Conversation

@StefanoFioravanzo
Copy link
Member

Summary

  • Add comprehensive architecture documentation for newcomers
  • Fix Python version requirement
  • Update FAQ for KFPv2 compatibility

Changes

  • New: docs/ARCHITECTURE.md - Complete architecture overview including:
    • ASCII diagram of data flow (JupyterLab → Backend → KFP)
    • Component overview and directory structure
    • Key classes documentation (NotebookProcessor, Pipeline, Step, Compiler)
    • Cell tag language reference
    • KFP v2 integration details
    • Marshal system and configuration documentation
  • Updated: README.md
    • Fixed Python version: 3.10+ → 3.12+
    • Added link to architecture documentation
  • Updated: FAQ.md
    • Removed MiniKF/Rok references
    • Removed outdated PersistentVolumeClaim section (KFPv2 uses artifacts)
    • Updated compiler errors section for KFPv2

Context

The architecture documentation addresses the need for a high-level overview for newcomers to the repository. The FAQ updates remove references to deprecated integrations and KFPv1 patterns.

Test plan

  • Verify documentation renders correctly on GitHub
  • Review architecture diagram accuracy

@google-oss-prow
Copy link

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from stefanofioravanzo. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@google-oss-prow google-oss-prow bot requested a review from ederign February 3, 2026 07:29
@StefanoFioravanzo StefanoFioravanzo force-pushed the docs/architecture-overview branch from 04396ad to 38db2fb Compare February 3, 2026 07:37
Add ARCHITECTURE.md with:

- High-level system diagram showing data flow from notebook to KFP
- Component overview (Backend + JupyterLab Extension)
- Detailed data flow explanation (annotation → processing → compilation → execution)
- Backend directory structure and key classes
- Cell tag language reference
- KFP v2 integration details
- Configuration and marshal system overview
- ASCII diagrams for visual learners

This document helps newcomers understand how Kale converts notebooks
into Kubeflow Pipelines and how the components interact.

Signed-off-by: Stefano Fioravanzo <stefano.fioravanzo@gmail.com>
- Add link to docs/ARCHITECTURE.md for newcomers
- Fix Python version requirement: 3.10+ → 3.12+

Signed-off-by: Stefano Fioravanzo <stefano.fioravanzo@gmail.com>
- Remove MiniKF/Rok integration mention (no longer maintained)
- Remove PersistentVolumeClaim section (KFPv2 uses artifacts, not volumes)
- Update compiler errors section for KFPv2 compatibility
- Simplify ModuleNotFoundError solution with current approaches

Signed-off-by: Stefano Fioravanzo <stefano.fioravanzo@gmail.com>
@StefanoFioravanzo StefanoFioravanzo force-pushed the docs/architecture-overview branch from 4fcd0a0 to f92232e Compare February 3, 2026 08:22
@ederign
Copy link
Member

ederign commented Feb 3, 2026

/assign

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants