Skip to content

openshift-hyperfleet/adapter-landing-zone

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

14 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

HyperFleet Landing Zone Adapter

Event-driven adapter for HyperFleet cluster provisioning. Handles environment preparation and prerequisite setup for GCP-based cluster provisioning operations. Consumes CloudEvents from message brokers (GCP Pub/Sub, RabbitMQ), processes AdapterConfig, manages Kubernetes resources, and reports status via API.

Table of Contents

Prerequisites

  • Kubernetes 1.19+
  • Helm 3.0+
  • GCP Workload Identity (for Pub/Sub access)
  • gcloud CLI configured with appropriate permissions

Local Development

Run the adapter locally for development and testing.

Prerequisites

  • hyperfleet-adapter binary installed and in PATH
  • GCP service account key for Pub/Sub access (see GCP Authentication)
  • Access to a GKE cluster (for applying Kubernetes resources)
  • podman or docker for RabbitMQ (if BROKER_TYPE=rabbitmq)

Setup

  1. Copy environment template:
cp env.example .env
  1. Edit .env with your configuration:
# Required for Google Pub/Sub (default)
GCP_PROJECT_ID="your-gcp-project-id"
BROKER_TOPIC="hyperfleet-adapter-topic"
BROKER_SUBSCRIPTION_ID="hyperfleet-adapter-landing-zone-subscription"

# Required for all broker types
HYPERFLEET_API_BASE_URL="https://localhost:8000"

# Optional (defaults provided)
SUBSCRIBER_PARALLELISM="1"
HYPERFLEET_API_VERSION="v1"

# Required for RabbitMQ (if BROKER_TYPE=rabbitmq)
# RABBITMQ_URL="amqp://guest:guest@localhost:5672/"
  1. Set up GCP authentication (see GCP Authentication for detailed steps):
# Create service account key and set in .env
export GOOGLE_APPLICATION_CREDENTIALS="./sa-key.json"
  1. Connect to your GKE cluster (required for the adapter to apply Kubernetes resources):
# Get credentials for your GKE cluster (using variables from .env)
gcloud container clusters get-credentials "$GKE_CLUSTER_NAME" \
  --region "$GKE_CLUSTER_REGION" \
  --project "$GCP_PROJECT_ID"

# Verify connection
kubectl cluster-info

Run

# For Google Pub/Sub (default)
make run-local

# For RabbitMQ
BROKER_TYPE=rabbitmq make run-local

# For RabbitMQ with Docker (override default podman)
BROKER_TYPE=rabbitmq CONTAINER_RUNTIME=docker make run-local

The script will:

  • Auto-source .env if it exists
  • Verify hyperfleet-adapter is installed
  • Validate required environment variables
  • Auto-create Pub/Sub topic and subscription if missing (for googlepubsub type)
  • Manage RabbitMQ container (start/create for rabbitmq type)
  • Generate broker config from configs/broker-local-pubsub.yaml or configs/broker-local-rabbitmq.yaml
  • Start the adapter with verbose logging

Local Environment Variables

Variable Required Description Default
GCP_PROJECT_ID Yes* GCP project ID -
GKE_CLUSTER_NAME Yes GKE cluster name for kubeconfig -
GKE_CLUSTER_REGION Yes GKE cluster region (or use GKE_CLUSTER_ZONE) -
BROKER_TOPIC Yes* Pub/Sub topic name -
BROKER_SUBSCRIPTION_ID Yes* Pub/Sub subscription ID -
HYPERFLEET_API_BASE_URL Yes HyperFleet API base URL -
SUBSCRIBER_PARALLELISM No Number of parallel workers 1
HYPERFLEET_API_VERSION No API version v1
GOOGLE_APPLICATION_CREDENTIALS Yes* Path to service account key file (recommended) -
RABBITMQ_URL No** RabbitMQ connection URL (when using RabbitMQ broker) amqp://guest:guest@localhost:5672/
BROKER_TYPE No Broker type: googlepubsub or rabbitmq googlepubsub
CONTAINER_RUNTIME No Container runtime for RabbitMQ podman

* Required when using GCP Pub/Sub broker (default) ** Required when using RabbitMQ broker

GCP Authentication

The adapter uses GCP Application Default Credentials (ADC). Recommended: Use a service account key file to avoid conflicts with other applications.

# 1. Create service account
gcloud iam service-accounts create hyperfleet-adapter-local \
  --project="$GCP_PROJECT_ID" \
  --display-name="HyperFleet Adapter Local Dev"

# 2. Grant Pub/Sub permissions
gcloud projects add-iam-policy-binding "$GCP_PROJECT_ID" \
  --member="serviceAccount:hyperfleet-adapter-local@${GCP_PROJECT_ID}.iam.gserviceaccount.com" \
  --role="roles/pubsub.subscriber"

# 3. Create key file
gcloud iam service-accounts keys create ./sa-key.json \
  --iam-account="hyperfleet-adapter-local@${GCP_PROJECT_ID}.iam.gserviceaccount.com"

# 4. Add to .env
export GOOGLE_APPLICATION_CREDENTIALS="./sa-key.json"

⚠️ Warning: Do NOT use gcloud auth application-default login - it will override your default credentials and may block other applications using ADC from a different project.

Helm Chart Installation

Installing the Chart

helm install landing-zone ./charts/

Install to a Specific Namespace

helm install landing-zone ./charts/ \
  --namespace hyperfleet-system \
  --create-namespace

Uninstalling the Chart

helm delete landing-zone

# Or with namespace
helm delete landing-zone --namespace hyperfleet-system

Configuration

All configurable parameters are in values.yaml. For advanced customization, modify the templates directly.

Image & Replica

Parameter Description Default
replicaCount Number of replicas 1
image.registry Image registry quay.io/openshift-hyperfleet
image.repository Image repository hyperfleet-adapter
image.tag Image tag latest
image.pullPolicy Image pull policy Always
imagePullSecrets Image pull secrets []

Naming

Parameter Description Default
nameOverride Override chart name ""
fullnameOverride Override full release name ""

ServiceAccount & RBAC

Parameter Description Default
serviceAccount.create Create ServiceAccount true
serviceAccount.name ServiceAccount name (auto-generated if empty) ""
serviceAccount.annotations ServiceAccount annotations (for Workload Identity) {}
rbac.create Create ClusterRole and ClusterRoleBinding false
rbac.namespaceAdmin Grant namespace admin permissions false

When rbac.namespaceAdmin=true, the adapter gets full access to:

  • Namespaces (create, update, delete)
  • Core resources (configmaps, secrets, serviceaccounts, services, pods, PVCs)
  • Apps (deployments, statefulsets, daemonsets, replicasets)
  • Batch (jobs, cronjobs)
  • Networking (ingresses, networkpolicies)
  • RBAC (roles, rolebindings)

Logging

Parameter Description Default
logging.level Log level (debug, info, warn, error) info
logging.format Log format (text, json) text
logging.output Log output (stdout, stderr) stderr

Scheduling

Parameter Description Default
nodeSelector Node selector {}
tolerations Tolerations []
affinity Affinity rules {}

Adapter Configuration

The adapter config is always created from charts/configs/adapter-landing-zone.yaml:

  • Mounted at /etc/adapter/adapter.yaml
  • Exposed via ADAPTER_CONFIG_PATH environment variable

To customize, edit charts/configs/adapter-landing-zone.yaml directly.

Broker Configuration

The broker configuration generates a broker.yaml file that is mounted as a ConfigMap.

General Settings

Parameter Description Default
broker.type Broker type: googlepubsub or rabbitmq (required) ""
broker.subscriber.parallelism Number of parallel workers 1
broker.yaml Raw YAML override (advanced use) ""

Google Pub/Sub (when broker.type=googlepubsub)

Parameter Description Default
broker.googlepubsub.projectId GCP project ID (required) ""
broker.googlepubsub.topic Pub/Sub topic name (required) ""
broker.googlepubsub.subscription Pub/Sub subscription ID (required) ""
broker.googlepubsub.deadLetterTopic Dead letter topic name (optional) ""

Other Pub/Sub settings (ack deadline, retention, goroutines, etc.) are configured with sensible defaults in the broker config template.

RabbitMQ (when broker.type=rabbitmq)

Parameter Description Default
broker.rabbitmq.url RabbitMQ connection URL (required) ""

Note: The broker.rabbitmq.url must be provided via --set or values file. Do not commit credentials to version control. Format: amqp://username:password@hostname:port/vhost

Other RabbitMQ settings (exchange type, prefetch count, etc.) are configured with sensible defaults in the broker config template.

When broker.type is set:

  • Generates broker.yaml from structured values
  • Creates ConfigMap with broker.yaml key
  • Mounts at /etc/broker/broker.yaml
  • Sets BROKER_CONFIG_FILE=/etc/broker/broker.yaml

HyperFleet API

Parameter Description Default
hyperfleetApi.baseUrl HyperFleet API base URL ""
hyperfleetApi.version API version v1

Environment Variables

Parameter Description Default
env Additional environment variables []

Example:

env:
  - name: MY_VAR
    value: "my-value"
  - name: MY_SECRET
    valueFrom:
      secretKeyRef:
        name: my-secret
        key: key

Examples

Basic Installation with Google Pub/Sub

helm install landing-zone ./charts/ \
  --set broker.type=googlepubsub \
  --set broker.googlepubsub.projectId=my-gcp-project \
  --set broker.googlepubsub.topic=my-topic \
  --set broker.googlepubsub.subscription=my-subscription

With HyperFleet API Configuration

helm install landing-zone ./charts/ \
  --set hyperfleetApi.baseUrl=https://api.hyperfleet.example.com \
  --set broker.type=googlepubsub \
  --set broker.googlepubsub.projectId=my-gcp-project \
  --set broker.googlepubsub.topic=my-topic \
  --set broker.googlepubsub.subscription=my-subscription

With RabbitMQ

helm install landing-zone ./charts/ \
  --set broker.type=rabbitmq \
  --set broker.rabbitmq.url="amqp://user:password@rabbitmq.svc:5672/"

Security: For production, store credentials in a Kubernetes Secret and reference via env.

With GCP Workload Identity and RBAC

First, grant Pub/Sub permissions to the KSA (before deploying):

# Get project number
gcloud projects describe my-gcp-project --format="value(projectNumber)"

# Grant permissions using direct principal binding (no GSA needed)
gcloud projects add-iam-policy-binding my-gcp-project \
  --role="roles/pubsub.subscriber" \
  --member="principal://iam.googleapis.com/projects/PROJECT_NUMBER/locations/global/workloadIdentityPools/my-gcp-project.svc.id.goog/subject/ns/hyperfleet-system/sa/landing-zone"

Then deploy:

helm install landing-zone ./charts/ \
  --namespace hyperfleet-system \
  --create-namespace \
  --set image.registry=us-central1-docker.pkg.dev/my-project/my-repo \
  --set image.repository=hyperfleet-adapter \
  --set image.tag=v0.1.0 \
  --set broker.type=googlepubsub \
  --set broker.googlepubsub.projectId=my-gcp-project \
  --set broker.googlepubsub.topic=my-topic \
  --set broker.googlepubsub.subscription=my-subscription \
  --set hyperfleetApi.baseUrl=https://api.hyperfleet.example.com \
  --set rbac.create=true \
  --set rbac.namespaceAdmin=true

With Custom Logging

helm install landing-zone ./charts/ \
  --set logging.level=debug \
  --set logging.format=json \
  --set logging.output=stdout \
  --set broker.type=googlepubsub \
  --set broker.googlepubsub.projectId=my-gcp-project \
  --set broker.googlepubsub.topic=my-topic \
  --set broker.googlepubsub.subscription=my-subscription

Using Existing ServiceAccount

helm install landing-zone ./charts/ \
  --set serviceAccount.create=false \
  --set serviceAccount.name=my-existing-sa

With Values File

Example my-values.yaml
replicaCount: 2

image:
  registry: us-central1-docker.pkg.dev/my-project/my-repo
  repository: hyperfleet-adapter
  tag: v0.1.0

serviceAccount:
  create: true
  annotations:
    iam.gke.io/gcp-service-account: adapter@my-project.iam.gserviceaccount.com

rbac:
  create: true
  namespaceAdmin: true

logging:
  level: debug
  format: json
  output: stderr

hyperfleetApi:
  baseUrl: https://api.hyperfleet.example.com
  version: v1

broker:
  type: googlepubsub
  googlepubsub:
    projectId: my-gcp-project
    topic: hyperfleet-events
    subscription: hyperfleet-adapter-subscription
  subscriber:
    parallelism: 10

Install with values file:

helm install landing-zone ./charts/ -f my-values.yaml

Deployment Environment Variables

The deployment sets these environment variables automatically:

Variable Value Condition
HYPERFLEET_API_BASE_URL From hyperfleetApi.baseUrl When set
HYPERFLEET_API_VERSION From hyperfleetApi.version Always (default: v1)
ADAPTER_CONFIG_PATH /etc/adapter/adapter.yaml Always
BROKER_CONFIG_FILE /etc/broker/broker.yaml When broker.type is set
BROKER_SUBSCRIPTION_ID From broker.googlepubsub.subscription When broker.type=googlepubsub
BROKER_TOPIC From broker.googlepubsub.topic When broker.type=googlepubsub
GCP_PROJECT_ID From broker.googlepubsub.projectId When broker.type=googlepubsub

GCP Workload Identity Setup

Grant GCP Pub/Sub permissions to the Kubernetes Service Account using Workload Identity Federation.

Step 1: Get Project Number

gcloud projects describe MY_PROJECT --format="value(projectNumber)"

Step 2: Grant Pub/Sub Permissions to KSA (Direct Principal Binding)

Run this before deploying so the pod works immediately:

# Grant subscriber permission
gcloud projects add-iam-policy-binding MY_PROJECT \
  --role="roles/pubsub.subscriber" \
  --member="principal://iam.googleapis.com/projects/MY_PROJECT_NUMBER/locations/global/workloadIdentityPools/MY_PROJECT.svc.id.goog/subject/ns/MY_NAMESPACE/sa/landing-zone" \
  --condition=None

# Grant viewer permission (required to read subscription metadata)
gcloud projects add-iam-policy-binding MY_PROJECT \
  --role="roles/pubsub.viewer" \
  --member="principal://iam.googleapis.com/projects/MY_PROJECT_NUMBER/locations/global/workloadIdentityPools/MY_PROJECT.svc.id.goog/subject/ns/MY_NAMESPACE/sa/landing-zone" \
  --condition=None

Note: This uses direct principal binding - no Google Service Account (GSA) required. The binding works even before the KSA exists.

Step 3: Wait for IAM Propagation

IAM changes can take 1-2 minutes to propagate. Wait until permissions are active:

# Wait until permissions are propagated
echo "Waiting for IAM propagation..."
while ! gcloud pubsub subscriptions describe MY_SUBSCRIPTION \
  --project=MY_PROJECT &>/dev/null; do
  echo "  Waiting for permissions to propagate..."
  sleep 10
done
echo "Permissions propagated!"

Step 4: Deploy

helm install landing-zone ./charts/ \
  --namespace MY_NAMESPACE \
  --create-namespace \
  --set broker.type=googlepubsub \
  --set broker.googlepubsub.projectId=MY_PROJECT \
  --set broker.googlepubsub.topic=MY_TOPIC \
  --set broker.googlepubsub.subscription=MY_SUBSCRIPTION \
  --set rbac.create=true \
  --set rbac.namespaceAdmin=true

Note: Replace the following placeholders:

  • MY_PROJECT - Your GCP project ID
  • MY_PROJECT_NUMBER - Your GCP project number (from Step 1)
  • MY_NAMESPACE - Kubernetes namespace (e.g., hyperfleet-system)
  • MY_TOPIC - Pub/Sub topic name
  • MY_SUBSCRIPTION - Pub/Sub subscription name
  • landing-zone - The Helm release name (KSA name)

Step 5: Verify Workload Identity

# Test authentication from pod
kubectl run -it --rm debug \
  --image=google/cloud-sdk:slim \
  --serviceaccount=landing-zone \
  --namespace=MY_NAMESPACE \
  -- gcloud auth list

Notes

  • The adapter runs as non-root user (UID 65532) with read-only filesystem
  • Health probes are disabled by default (adapter is a message consumer, not HTTP server)
  • Uses distroless base image for minimal attack surface
  • Config checksum annotation triggers pod restart on ConfigMap changes
  • Default resource limits: 500m CPU, 512Mi memory
  • Default resource requests: 100m CPU, 128Mi memory

License

See LICENSE for details.

About

Handles environment preparation and prerequisite setup for GCP-based cluster provisioning operations.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Contributors 4

  •  
  •  
  •  
  •