ROSA automation #49

RishabhKodes · 2025-07-03T14:05:35Z

Added the scripts for creating and managing ROSA clusters.

mjiao · 2025-07-11T22:47:42Z

Please remove the two pipeline runs in the .tekton folder.

Signed-off-by: RishabhKodes <rishabhbhandari6@gmail.com>

mjiao · 2025-07-16T11:54:44Z

Hi @RishabhKodes

Thanks for your progress on this.

After reviewing, I think we need to pivot our approach on the IaC tool. Given our broader strategy involving a hybrid cloud, Terraform is much better suited for our needs than CloudFormation. Could you please switch the implementation from CloudFormation to Terraform? Once the foundation is in Terraform, we can proceed with adding the following services:

Amazon RDS for PostgreSQL (with password authentication)
Amazon ElastiCache for Redis (with AUTH for the default user)

Happy to sync up to discuss the implementation. Let me know your thoughts.

Signed-off-by: RishabhKodes <rishabhbhandari6@gmail.com>

RishabhKodes · 2025-08-04T09:19:09Z

Hi @RishabhKodes

Thanks for your progress on this.

After reviewing, I think we need to pivot our approach on the IaC tool. Given our broader strategy involving a hybrid cloud, Terraform is much better suited for our needs than CloudFormation. Could you please switch the implementation from CloudFormation to Terraform? Once the foundation is in Terraform, we can proceed with adding the following services:

Amazon RDS for PostgreSQL (with password authentication)

Amazon ElastiCache for Redis (with AUTH for the default user)

Happy to sync up to discuss the implementation. Let me know your thoughts.

@mjiao I've added Terraform as the IaC tool, would appreciate your review.

Signed-off-by: RishabhKodes <rishabhbhandari6@gmail.com>

rosa.makefile

rosa/domain-records.yaml

rosa/network.yaml

rosa/rosa-domain-records.sh

mjiao · 2025-08-06T22:10:37Z

setup-rosa-efs.sh

@@ -0,0 +1,572 @@
+#!/bin/bash


EFS can also be managed via terraform https://registry.terraform.io/providers/hashicorp/aws/latest/docs/resources/efs_file_system

You might need to add further configuration to make it usable in the openshift.

Signed-off-by: RishabhKodes <rishabhbhandari6@gmail.com>

RishabhKodes · 2025-08-15T09:21:53Z

@mjiao please review the latest changes, which have end-to-end Terraform integration. FYI, the EFS has still not been integrated completely. I'm in the process of testing that.

Signed-off-by: RishabhKodes <rishabhbhandari6@gmail.com>

kksat · 2025-08-21T11:06:45Z

as we now have terraform code I suggest to run terraform linter in CI/CD, here is what tflint output at the moment

tflint       
5 issue(s) found:

Warning: [Fixable] Comparing a collection with an empty list is invalid. To detect an empty collection, check its length. (terraform_empty_list_equali
ty)

  on rosa-hcp.tf line 24:
  24:   aws_availability_zones = var.availability_zones != [] ? var.availability_zones : []

Reference: https://github.com/terraform-linters/tflint-ruleset-terraform/blob/v0.12.0/docs/rules/terraform_empty_list_equality.md

Warning: [Fixable] variable "channel_group" is declared but not used (terraform_unused_declarations)

  on variables.tf line 102:
 102: variable "channel_group" {

Reference: https://github.com/terraform-linters/tflint-ruleset-terraform/blob/v0.12.0/docs/rules/terraform_unused_declarations.md

Warning: [Fixable] variable "enable_autoscaling" is declared but not used (terraform_unused_declarations)

  on variables.tf line 200:
 200: variable "enable_autoscaling" {

Reference: https://github.com/terraform-linters/tflint-ruleset-terraform/blob/v0.12.0/docs/rules/terraform_unused_declarations.md

Warning: [Fixable] variable "min_replicas" is declared but not used (terraform_unused_declarations)

  on variables.tf line 206:
 206: variable "min_replicas" {

Reference: https://github.com/terraform-linters/tflint-ruleset-terraform/blob/v0.12.0/docs/rules/terraform_unused_declarations.md

Warning: [Fixable] variable "max_replicas" is declared but not used (terraform_unused_declarations)

  on variables.tf line 212:
 212: variable "max_replicas" {

Reference: https://github.com/terraform-linters/tflint-ruleset-terraform/blob/v0.12.0/docs/rules/terraform_unused_declarations.md

kksat · 2025-07-16T19:49:35Z

rosa.makefile

 rosa-login:  ## Login using ROSA token
 	$(call required-environment-variables,ROSA_TOKEN)
 	@rosa login --token="${ROSA_TOKEN}"
+	@rosa create account-roles --mode auto


maybe separate makefile target?

kksat · 2025-07-16T19:51:52Z

rosa.makefile

+.PHONY: rosa-domain-zone-exists
+rosa-domain-zone-exists:  ## Fail if Route53 hosted zone does not exist
+	$(call required-environment-variables,ROSA_DOMAIN)
+	ROSA_DOMAIN=${ROSA_DOMAIN} rosa/rosa-domain-zone-exists.sh


ROSA_DOMAIN is already environment variable, no need to reassign it.

kksat · 2025-07-16T19:53:11Z

rosa.makefile

+	ROSA_DOMAIN=${ROSA_DOMAIN} rosa/rosa-domain-zone-exists.sh
+
+.PHONY: rosa-deploy
+rosa-deploy: rosa-domain-zone-exists rosa-network-deploy rosa-account-roles rosa-cluster rosa-operator-roles rosa-oidc-provider  ## Deploy ROSA cluster with all dependencies


rosa account roles will not be created here, because rosa-login is not in dependencies list, one more reason to have it as separate makefile target

kksat · 2025-07-17T08:41:22Z

rosa/rosa-domain-records.sh

+# deploys the appropriate CloudFormation stack (A-records or CNAMEs) under your domain.
+
+# shellcheck disable=SC2059
+set -euo pipefail


I think there is no need for if statements below to check status if 'euo' is set. Script will fail as soon as any command will fail.

kksat · 2025-07-17T08:42:29Z

rosa/rosa-domain-records.sh

+# shellcheck disable=SC2059
+set -euo pipefail
+
+DOMAIN=""


I see domain is required below, no need to set it to empty string here, I think

kksat · 2025-08-21T11:46:55Z

rosa.makefile

+	@echo "Destroying ROSA HCP cluster using Terraform..."
+	@cd rosa/terraform && \
+	terraform destroy \
+		-var="rosa_token=${ROSA_TOKEN}" \


I would propose to set secret parameters via env variables

export TF_VAR_rosa_token="$ROSA_TOKEN"

This guarantees that we will not forget some @ at the begining of the command and expose secret in the log

kksat · 2025-08-21T12:00:20Z

rosa.makefile

+	echo 'aws_region = "${ROSA_REGION}"' > network.tfvars && \
+	echo 'cluster_name = "${CLUSTER_NAME}"' >> network.tfvars && \
+	echo 'vpc_name = "rosa-${CLUSTER_NAME}-vpc"' >> network.tfvars && \
+	echo 'environment_tag = "rosa"' >> network.tfvars && \
+	echo 'vpc_cidr = "10.0.0.0/16"' >> network.tfvars && \
+	echo 'public_subnet_1_cidr = "10.0.0.0/24"' >> network.tfvars && \
+	echo 'public_subnet_2_cidr = "10.0.1.0/24"' >> network.tfvars && \
+	echo 'public_subnet_3_cidr = "10.0.4.0/24"' >> network.tfvars && \
+	echo 'private_subnet_1_cidr = "10.0.2.0/24"' >> network.tfvars && \
+	echo 'private_subnet_2_cidr = "10.0.3.0/24"' >> network.tfvars && \
+	echo 'private_subnet_3_cidr = "10.0.5.0/24"' >> network.tfvars && \


Here you are just creating a file. Makefile targets are perfect for this. So we should have makefile target rosa/terraform/network.tfvars

Second point - there is command envsubst that uses template file and substites environment variables with their values to create new file. For instance:

envsubst < rosa/terraform/network.tfvars.envsubst > rosa/terraform/network.tfvars

kksat · 2025-08-21T12:01:24Z

rosa.makefile

+	echo 'private_subnet_1_cidr = "10.0.2.0/24"' >> network.tfvars && \
+	echo 'private_subnet_2_cidr = "10.0.3.0/24"' >> network.tfvars && \
+	echo 'private_subnet_3_cidr = "10.0.5.0/24"' >> network.tfvars && \
+	terraform plan -var-file=network.tfvars


this is a separate target, its prerequsite is that network.tfvars file is present.

rosa.makefile

kksat · 2025-08-21T12:15:56Z

rosa.makefile

+	echo 'private_subnet_3_cidr = "10.0.5.0/24"' >> network.tfvars && \
+	terraform plan -var-file=network.tfvars
+
+.PHONY: rosa-terraform-output


There is a way to run any command with make targets. I will show you example separately, so you can adopt it to this use case.

Signed-off-by: RishabhKodes <rishabhbhandari6@gmail.com>

mjiao · 2025-08-26T09:04:07Z

rosa/terraform/terraform.tfvars.example

+# availability_zones = ["eu-central-1a", "eu-central-1b", "eu-central-1c"]
+
+# VPC Configuration (when create_vpc = true)
+vpc_name               = "rosa-sapeic-vpc"


Pls make sure vpc name is derived from cluster_name, e.g. vpc_name = "rosa-${var.cluster_name}-vpc"

@mjiao the vpc_name is already being set as rosa-${var.cluster_name}-vpc here https://github.com/redhat-sap/sap-edge/blob/rosa/rosa.makefile#L113, I also checked in the AWS console it is being set correctly.

Nevertheless, I have updated the code in terraform.tfvars.example as well, ensuring that it does not create any confusion.

rosa/terraform/env.example

rosa/terraform/provider.tf

mjiao · 2025-08-26T22:37:47Z

rosa/terraform/network.tf

@@ -0,0 +1,244 @@
+# SPDX-FileCopyrightText: 2024 SAP edge team


When running terraform apply multiple times on this ROSA HCP deployment, Terraform detects
"drift" because ROSA/OpenShift automatically adds tags to subnets (like
kubernetes.io/cluster/, api.openshift.com/, etc.) that aren't managed in the Terraform
configuration.

Could you pls fix this?

rosa/terraform/outputs.tf

rosa/terraform/network.tf

RishabhKodes force-pushed the rosa branch from 3e53f39 to c0ef000 Compare July 8, 2025 10:16

RishabhKodes changed the title ~~[WIP]: added rosa automation~~ added rosa automation Jul 8, 2025

RishabhKodes force-pushed the rosa branch from ac1947c to 22668a3 Compare July 14, 2025 07:20

added rosa automation

b0e4ca7

Signed-off-by: RishabhKodes <rishabhbhandari6@gmail.com>

RishabhKodes force-pushed the rosa branch from 22668a3 to 603cab3 Compare July 14, 2025 10:51

fixed linting errors

c522ca8

Signed-off-by: RishabhKodes <rishabhbhandari6@gmail.com>

RishabhKodes force-pushed the rosa branch from 603cab3 to c522ca8 Compare July 14, 2025 14:08

removed tekton pipelines

c2c5877

Signed-off-by: RishabhKodes <rishabhbhandari6@gmail.com>

RishabhKodes requested review from kksat and mjiao July 14, 2025 15:25

RishabhKodes changed the title ~~added rosa automation~~ ROSA automation Jul 22, 2025

RishabhKodes added 3 commits August 3, 2025 22:24

added efs setup script and initialized terraform

db28b99

Signed-off-by: RishabhKodes <rishabhbhandari6@gmail.com>

minor terraform config changes

afec034

Signed-off-by: RishabhKodes <rishabhbhandari6@gmail.com>

added terraform gitignore

0ecfff3

Signed-off-by: RishabhKodes <rishabhbhandari6@gmail.com>

RishabhKodes force-pushed the rosa branch from 6a542b8 to 0ecfff3 Compare August 3, 2025 20:25

RishabhKodes added 2 commits August 5, 2025 13:46

added third availability zone support with public/private subnet pairs

299994f

Signed-off-by: RishabhKodes <rishabhbhandari6@gmail.com>

removed further mentions of CloudFormation in the repo

95efad4

Signed-off-by: RishabhKodes <rishabhbhandari6@gmail.com>

RishabhKodes force-pushed the rosa branch from 2bbaeca to 95efad4 Compare August 11, 2025 10:17

RishabhKodes added 2 commits August 11, 2025 12:27

added efs script to gitignore (temporary)

c0205c5

Signed-off-by: RishabhKodes <rishabhbhandari6@gmail.com>

efs script lint fixes

825f820

Signed-off-by: RishabhKodes <rishabhbhandari6@gmail.com>

RishabhKodes force-pushed the rosa branch from adabf93 to 825f820 Compare August 11, 2025 12:18

cspell lint fixes

aa7589b

Signed-off-by: RishabhKodes <rishabhbhandari6@gmail.com>

mjiao reviewed Aug 13, 2025

View reviewed changes

e2e Terraform integration

f31b24c

Signed-off-by: RishabhKodes <rishabhbhandari6@gmail.com>

RishabhKodes added 2 commits August 18, 2025 11:34

efs terraform changes

6295e12

Signed-off-by: RishabhKodes <rishabhbhandari6@gmail.com>

pre-cluster setup checks

1ae672f

Signed-off-by: RishabhKodes <rishabhbhandari6@gmail.com>

RishabhKodes force-pushed the rosa branch from 18151b4 to 0900c44 Compare August 21, 2025 09:55

lint fixes

09ed523

Signed-off-by: RishabhKodes <rishabhbhandari6@gmail.com>

RishabhKodes force-pushed the rosa branch from 0900c44 to 09ed523 Compare August 21, 2025 10:11

kksat requested changes Aug 21, 2025

View reviewed changes

minor variable fixes

0f57f94

Signed-off-by: RishabhKodes <rishabhbhandari6@gmail.com>

mjiao reviewed Aug 26, 2025

View reviewed changes

mjiao force-pushed the rosa branch from 00a63fa to 0f57f94 Compare August 26, 2025 22:27

mjiao reviewed Aug 26, 2025

View reviewed changes

rosa/terraform/provider.tf Show resolved Hide resolved

mjiao requested changes Aug 26, 2025

View reviewed changes

rosa/terraform/outputs.tf Show resolved Hide resolved

mjiao reviewed Aug 27, 2025

View reviewed changes

rosa/terraform/network.tf Show resolved Hide resolved

		@@ -0,0 +1,244 @@
		# SPDX-FileCopyrightText: 2024 SAP edge team

ROSA automation #49

Are you sure you want to change the base?

ROSA automation #49

Uh oh!

Conversation

RishabhKodes commented Jul 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mjiao commented Jul 11, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

mjiao commented Jul 16, 2025

Uh oh!

RishabhKodes commented Aug 4, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

RishabhKodes commented Aug 15, 2025

Uh oh!

kksat commented Aug 21, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

RishabhKodes commented Jul 3, 2025 •

edited

Loading

mjiao commented Jul 11, 2025 •

edited

Loading