Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions content/deployment/helm-chart-reference/dataplane.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: Dataplane chart
variants: -flyte -byoc +selfmanaged
chart_version: 2026.3.10
chart_version: 2026.3.11
weight: 1
---

Expand All @@ -11,8 +11,8 @@ Deploys the Union dataplane components to onboard a kubernetes cluster to the Un

| | |
|---|---|
| **Chart version** | 2026.3.10 |
| **App version** | 2026.3.7 |
| **Chart version** | 2026.3.11 |
| **App version** | 2026.3.8 |
| **Kubernetes version** | `>= 1.28.0-0` |

## Dependencies
Expand All @@ -38,6 +38,7 @@ Deploys the Union dataplane components to onboard a kubernetes cluster to the Un
| additionalPodSpec | object | Define additional PodSpec values for all of the Union pods. | `{}` |
| clusterName | string | Cluster name should be shared with Union for proper functionality. | `"{{ .Values.global.CLUSTER_NAME }}"` |
| clusterresourcesync | object | clusterresourcesync contains the configuration information for the syncresources service. | `(see values.yaml)` |
| clusterresourcesync.additionalTemplates | list | Additional cluster resource templates to create per project namespace. Use this instead of overriding `templates` to avoid accidentally removing the default namespace, service account, and resource quota templates. Each entry has a `key` (filename stem) and `value` (Kubernetes manifest). | `[]` |
| clusterresourcesync.additionalVolumeMounts | list | Appends additional volume mounts to the main container's spec. May include template values. | `[]` |
| clusterresourcesync.additionalVolumes | list | Appends additional volumes to the deployment spec. May include template values. | `[]` |
| clusterresourcesync.affinity | object | affinity configurations for the syncresources pods | `{}` |
Expand Down
22 changes: 13 additions & 9 deletions content/deployment/selfhosted-deployment/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -35,18 +35,18 @@ In a self-hosted intra-cluster deployment, the control plane and data plane comm
```mermaid
graph TB
subgraph cluster["Kubernetes Cluster"]
subgraph cp["Namespace: union-cp (Control Plane)"]
subgraph cp["Controlplane Namespace"]
cpingress["NGINX Ingress\n(TLS/HTTP2)\nClusterIP"]
flyteadmin["Flyteadmin"]
admin["Admin"]
identity["Identity"]
executions["Executions"]
services["Services"]

cpingress --> flyteadmin
cpingress --> admin
cpingress --> identity
cpingress --> executions
cpingress --> services
end

subgraph dp["Namespace: union (Data Plane)"]
subgraph dp["Dataplane Namespace"]
dpingress["NGINX Ingress\nClusterIP"]
operator["Operator"]
propeller["Propeller"]
Expand All @@ -65,10 +65,10 @@ graph TB
dpingress -.->|"Internal DNS"| cpingress
cpingress -.->|"Internal DNS"| dpingress

flyteadmin --> db
admin --> db
identity --> db
executions --> db
flyteadmin --> storage
services --> db
admin --> storage
operator --> storage
end
```
Expand Down Expand Up @@ -126,6 +126,10 @@ Deploy the data plane with GCS and Workload Identity
Configure OIDC/OAuth2 authentication for your deployment
{{< /link-card >}}

{{< link-card target="./authorization" icon="shield" title="Authorization" >}}
Configure authorization mode (Noop, External, or Union built-in RBAC)
{{< /link-card >}}

{{< link-card target="./image-builder" icon="package" title="Image builder" >}}
Register the image builder for automatic container image builds
{{< /link-card >}}
Expand Down
56 changes: 29 additions & 27 deletions content/deployment/selfhosted-deployment/authentication.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,7 +127,7 @@ global:
AUTH_TOKEN_URL: "https://your-idp.example.com/oauth2/default/v1/token"
```

Enable authentication in FlyteAdmin:
Enable authentication in the admin service:

```yaml
flyte:
Expand All @@ -146,21 +146,23 @@ flyte:
The control plane needs secrets for the browser login app (App 1) and the service-to-service app (App 3):

```shell
# Secret for flyteadmin (mounted at /etc/secrets/)
# Secret for admin service (mounted at /etc/secrets/)
# Note: "flyte-admin-secrets" is the default name expected by the Helm chart
kubectl create secret generic flyte-admin-secrets \
--from-literal=client_secret='<BROWSER_LOGIN_CLIENT_SECRET>' \
-n union-cp
-n <controlplane-namespace>

# Secret for flyte-scheduler (mounted at /etc/secrets/)
# Secret for scheduler (mounted at /etc/secrets/)
# Note: "flyte-secret-auth" is the default name expected by the Helm chart
kubectl create secret generic flyte-secret-auth \
--from-literal=client_secret='<BROWSER_LOGIN_CLIENT_SECRET>' \
-n union-cp
-n <controlplane-namespace>

# Add service-to-service client secret to the main secret
kubectl create secret generic union-controlplane-secrets \
# Add service-to-service client secret to the controlplane secrets
kubectl create secret generic <controlplane-secrets> \
--from-literal=pass.txt='<DB_PASSWORD>' \
--from-literal=client_secret='<SERVICE_TO_SERVICE_CLIENT_SECRET>' \
-n union-cp --dry-run=client -o yaml | kubectl apply -f -
-n <controlplane-namespace> --dry-run=client -o yaml | kubectl apply -f -
```

> [!NOTE]
Expand All @@ -180,7 +182,7 @@ Create the data plane auth secret:
```shell
kubectl create secret generic union-secret-auth \
--from-literal=client_secret='<OPERATOR_CLIENT_SECRET>' \
-n union
-n <dataplane-namespace>
```

## Step 5: Configure EAGER_API_KEY
Expand All @@ -199,7 +201,7 @@ Create the Kubernetes secret in the data plane namespace:
```shell
kubectl create secret generic <eager-secret-name> \
--from-literal=<eager-secret-key>='<BASE64_ENCODED_EAGER_API_KEY>' \
-n union
-n <dataplane-namespace>
```

> [!NOTE]
Expand All @@ -212,15 +214,15 @@ Deploy or upgrade both the control plane and data plane with the updated configu
```shell
# Upgrade control plane
helm upgrade unionai-controlplane unionai/controlplane \
--namespace union-cp \
--namespace <controlplane-namespace> \
-f values.<cloud>.selfhosted-intracluster.yaml \
-f values.registry.yaml \
-f values.<cloud>.selfhosted-overrides.yaml \
--timeout 15m --wait

# Upgrade data plane
helm upgrade unionai-dataplane unionai/dataplane \
--namespace union \
--namespace <dataplane-namespace> \
-f values.<cloud>.selfhosted-intracluster.yaml \
-f values.<cloud>.selfhosted-overrides.yaml \
--timeout 10m --wait
Expand All @@ -229,31 +231,31 @@ helm upgrade unionai-dataplane unionai/dataplane \
## Verification

```shell
# Check flyteadmin logs for auth initialization
kubectl logs -n union-cp deploy/flyteadmin | grep -i auth
# Check admin service logs for auth initialization
kubectl logs -n <controlplane-namespace> deploy/<admin-service> | grep -i auth

# Test the /me endpoint (should return 401 without a token)
kubectl exec -n union-cp deploy/flyteadmin -- \
kubectl exec -n <controlplane-namespace> deploy/<admin-service> -- \
curl -s -o /dev/null -w "%{http_code}" \
https://controlplane-nginx-controller.union-cp.svc.cluster.local/me -k
https://<controlplane-ingress>.<controlplane-namespace>.svc.cluster.local/me -k

# Test CLI login
uctl config init --host https://<your-domain>
uctl get project

# Check data plane operator auth
kubectl logs -n union -l app.kubernetes.io/name=operator --tail=50 | grep -i "token\|auth"
kubectl logs -n <dataplane-namespace> -l app.kubernetes.io/name=operator --tail=50 | grep -i "token\|auth"
```

## Summary of secrets

| Secret name | Namespace | Keys | Source |
|-------------|-----------|------|--------|
| `flyte-admin-secrets` | `union-cp` | `client_secret` | Browser login app (App 1) secret |
| `flyte-secret-auth` | `union-cp` | `client_secret` | Browser login app (App 1) secret |
| `union-controlplane-secrets` | `union-cp` | `pass.txt`, `client_secret` | DB password, Service-to-service app (App 3) secret |
| `union-secret-auth` | `union` | `client_secret` | Operator app (App 4) secret |
| EAGER secret | `union` | varies | EAGER app (App 5) encoded key |
| `flyte-admin-secrets` (Helm chart default) | `<controlplane-namespace>` | `client_secret` | Browser login app (App 1) secret |
| `flyte-secret-auth` (Helm chart default) | `<controlplane-namespace>` | `client_secret` | Browser login app (App 1) secret |
| `<controlplane-secrets>` | `<controlplane-namespace>` | `pass.txt`, `client_secret` | DB password, Service-to-service app (App 3) secret |
| `union-secret-auth` | `<dataplane-namespace>` | `client_secret` | Operator app (App 4) secret |
| EAGER secret | `<dataplane-namespace>` | varies | EAGER app (App 5) encoded key |

## Self-hosted vs. self-managed authentication

Expand All @@ -266,26 +268,26 @@ kubectl logs -n union -l app.kubernetes.io/name=operator --tail=50 | grep -i "to

## Troubleshooting

### FlyteAdmin auth endpoints return 404
### Admin service auth endpoints return 404

Ensure `useAuth: true` is set under `flyte.configmap.adminServer.server.security`. Without this, the `/login`, `/callback`, and `/me` endpoints are not registered.

### Token validation fails with "audience mismatch"

The `allowedAudience` in the FlyteAdmin configuration must include `https://<your-domain>`. This should match the audience configured on your authorization server.
The `allowedAudience` in the admin service configuration must include `https://<your-domain>`. This should match the audience configured on your authorization server.

### Data plane cannot authenticate to control plane

```shell
# Verify AUTH_CLIENT_ID is set
kubectl get configmap -n union -o yaml | grep -i auth_client
kubectl get configmap -n <dataplane-namespace> -o yaml | grep -i auth_client

# Check that union-secret-auth exists
kubectl get secret union-secret-auth -n union \
kubectl get secret union-secret-auth -n <dataplane-namespace> \
-o jsonpath='{.data.client_secret}' | base64 -d

# Check operator logs
kubectl logs -n union -l app.kubernetes.io/name=operator --tail=50 \
kubectl logs -n <dataplane-namespace> -l app.kubernetes.io/name=operator --tail=50 \
| grep -i "auth\|token\|401"
```

Expand Down
Loading
Loading