fix: improve TSA probes with startup probe for initialization verific…

[osmman]

…ation

Add StartupProbe that checks /api/v1/timestamp/certchain to verify TSA is fully initialized before liveness/readiness probes begin. This ensures certificate chain is loaded and validated at startup while keeping ongoing probes lightweight with /ping endpoint.

[qodo-code-review]

CI Feedback 🧐

A test triggered by this PR failed. Here is an AI-generated analysis of the failure:

Action: Test with High Availability

Failed stage: Post Install Cluster [❌]

Failed test name: ""

Failure summary: The action failed during the Kubernetes-based e2e/install workflow due to cluster resource conflicts and subsequent cleanup failure: - The operator reconciliation failed when creating Ingress resources because the NGINX validating webhook rejected duplicate host/path combinations: - base_action.go:93: could not create ingress object: admission webhook "validate.nginx.ingress.kubernetes.io" denied the request: host "fulcio-server.local" and path "/" is already defined in ingress common-ha-install-test-kddv7/fulcio-server - Similar conflict for host "tsa-server.local" already defined in common-ha-install-test-kddv7/tsa-server. - This indicates multiple test namespaces attempted to create identical Ingress hosts/paths in the same cluster without unique hostnames or proper cleanup between tests. - As a downstream effect, services/deployments (e.g., rekor-server) were not ready, causing requests like public key retrieval to fail with connect: connection refused (resolve_pub_key.go:152 / base_action.go:93). - After the workflow, the runner failed to delete the Kind cluster because Docker could not stop/remove the kind-control-plane container: - failed to delete nodes: command "docker rm -f -v kind-control-plane" ... cannot remove container ... could not kill: tried to kill container, but did not receive an exit event

Relevant error logs: 1: ##[group]Runner Image Provisioner 2: Hosted Compute Agent ... 373: configmap/ingress-nginx-controller created 374: service/ingress-nginx-controller created 375: service/ingress-nginx-controller-admission created 376: deployment.apps/ingress-nginx-controller created 377: job.batch/ingress-nginx-admission-create created 378: job.batch/ingress-nginx-admission-patch created 379: ingressclass.networking.k8s.io/nginx created 380: validatingwebhookconfiguration.admissionregistration.k8s.io/ingress-nginx-admission created 381: pod/ingress-nginx-controller-bcdf75cfc-shd92 condition met 382: ##[group]Run # Download the bundle.yaml 383: �[36;1m# Download the bundle.yaml�[0m 384: �[36;1mcurl -sL https://github.com/prometheus-operator/prometheus-operator/releases/download/v0.84.0/bundle.yaml -o bundle.yaml �[0m 385: �[36;1m�[0m 386: �[36;1m# Check if the download was successful and the file is not empty�[0m 387: �[36;1mif [ ! -s "bundle.yaml" ]; then�[0m 388: �[36;1m echo "Error: Downloaded bundle.yaml is empty or failed to download."�[0m 389: �[36;1m exit 1�[0m ... 672: BUNDLE_IMG: ghcr.io/securesign/secure-sign-operator-bundle:dev-2f4be9e666af73bd894a30cbebb411367264ff61 673: CATALOG_IMG: ghcr.io/securesign/secure-sign-operator-fbc:dev-2f4be9e666af73bd894a30cbebb411367264ff61 674: NEW_OLM_CHANNEL: rhtas-operator.v1.4.0 675: OCP_VERSION: v4.19 676: REGISTRY_AUTH_FILE: /tmp/config.json 677: ##[endgroup] 678: mkdir -p /home/runner/work/secure-sign-operator/secure-sign-operator/bin 679: Downloading sigs.k8s.io/controller-tools/cmd/controller-gen@v0.17.0 680: /home/runner/work/secure-sign-operator/secure-sign-operator/bin/controller-gen-v0.17.0 rbac:roleName=manager-role crd webhook paths="./..." output:crd:artifacts:config=config/crd/bases 681: Downloading sigs.k8s.io/kustomize/kustomize/v5@v5.6.0 682: go: downloading sigs.k8s.io/kustomize/kustomize/v5 v5.6.0 683: go: downloading sigs.k8s.io/kustomize/api v0.19.0 684: go: downloading github.com/spf13/cobra v1.8.0 685: go: downloading sigs.k8s.io/kustomize/cmd/config v0.19.0 686: go: downloading sigs.k8s.io/kustomize/kyaml v0.19.0 687: go: downloading github.com/go-errors/errors v1.4.2 688: go: downloading github.com/sergi/go-diff v1.2.0 689: go: downloading github.com/davecgh/go-spew v1.1.1 690: go: downloading k8s.io/kube-openapi v0.0.0-20241212222426-2c72e554b1e7 691: go: downloading github.com/monochromegane/go-gitignore v0.0.0-20200626010858-205db1a8cc00 692: go: downloading github.com/xlab/treeprint v1.2.0 693: go: downloading gopkg.in/evanphx/json-patch.v4 v4.12.0 694: go: downloading github.com/google/gnostic-models v0.6.9 695: go: downloading google.golang.org/protobuf v1.35.1 696: go: downloading github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510 697: go: downloading github.com/pkg/errors v0.9.1 698: go: downloading github.com/go-openapi/jsonreference v0.20.2 ... 832: go: downloading github.com/aws/aws-sdk-go-v2/internal/ini v1.8.0 833: go: downloading github.com/aws/aws-sdk-go-v2/service/ssooidc v1.26.4 834: go: downloading github.com/aws/aws-sdk-go-v2/service/sts v1.30.3 835: go: downloading github.com/aws/aws-sdk-go-v2/internal/configsources v1.3.15 836: go: downloading cloud.google.com/go/auth v0.7.3 837: go: downloading golang.org/x/oauth2 v0.22.0 838: go: downloading github.com/Azure/azure-sdk-for-go/sdk/security/keyvault/internal v1.0.0 839: go: downloading github.com/Azure/azure-sdk-for-go/sdk/internal v1.10.0 840: go: downloading github.com/AzureAD/microsoft-authentication-library-for-go v1.2.2 841: go: downloading github.com/containerd/stargz-snapshotter/estargz v0.14.3 842: go: downloading github.com/google/go-cmp v0.6.0 843: go: downloading github.com/cenkalti/backoff/v3 v3.2.2 844: go: downloading github.com/go-jose/go-jose/v4 v4.0.2 845: go: downloading github.com/hashicorp/errwrap v1.1.0 846: go: downloading github.com/hashicorp/go-cleanhttp v0.5.2 847: go: downloading github.com/hashicorp/go-multierror v1.1.1 848: go: downloading github.com/hashicorp/go-retryablehttp v0.7.7 ... 860: go: downloading k8s.io/utils v0.0.0-20240502163921-fe8a2dddb1d0 861: go: downloading github.com/theupdateframework/go-tuf v0.7.0 862: go: downloading github.com/moby/term v0.5.0 863: go: downloading github.com/Azure/go-autorest/autorest/azure/auth v0.5.12 864: go: downloading github.com/docker/docker-credential-helpers v0.8.0 865: go: downloading github.com/shibumi/go-pathspec v1.3.0 866: go: downloading golang.org/x/sys v0.22.0 867: go: downloading github.com/sirupsen/logrus v1.9.3 868: go: downloading github.com/sigstore/fulcio v1.5.1 869: go: downloading go.step.sm/crypto v0.51.1 870: go: downloading github.com/dustin/go-humanize v1.0.1 871: go: downloading cuelang.org/go v0.9.2 872: go: downloading github.com/Azure/go-autorest/autorest v0.11.29 873: go: downloading github.com/Azure/go-autorest v14.2.0+incompatible 874: go: downloading github.com/open-policy-agent/opa v0.67.0 875: go: downloading github.com/go-openapi/errors v0.22.0 876: go: downloading github.com/go-openapi/validate v0.24.0 ... 1103: I0127 13:39:40.454946 1 controller.go:303] "Starting Controller" controller="securesign" controllerGroup="rhtas.redhat.com" controllerKind="Securesign" 1104: I0127 13:39:40.454975 1 controller.go:306] "Starting workers" controller="securesign" controllerGroup="rhtas.redhat.com" controllerKind="Securesign" worker count=1 1105: I0127 13:39:40.457291 1 controller.go:303] "Starting Controller" controller="fulcio" controllerGroup="rhtas.redhat.com" controllerKind="Fulcio" 1106: I0127 13:39:40.457313 1 controller.go:306] "Starting workers" controller="fulcio" controllerGroup="rhtas.redhat.com" controllerKind="Fulcio" worker count=1 1107: I0127 13:39:40.457336 1 controller.go:303] "Starting Controller" controller="timestampauthority" controllerGroup="rhtas.redhat.com" controllerKind="TimestampAuthority" 1108: I0127 13:39:40.457346 1 controller.go:306] "Starting workers" controller="timestampauthority" controllerGroup="rhtas.redhat.com" controllerKind="TimestampAuthority" worker count=1 1109: I0127 13:39:40.457362 1 controller.go:303] "Starting Controller" controller="rekor" controllerGroup="rhtas.redhat.com" controllerKind="Rekor" 1110: I0127 13:39:40.457371 1 controller.go:306] "Starting workers" controller="rekor" controllerGroup="rhtas.redhat.com" controllerKind="Rekor" worker count=1 1111: I0127 13:39:40.457397 1 controller.go:303] "Starting Controller" controller="tuf" controllerGroup="rhtas.redhat.com" controllerKind="Tuf" 1112: I0127 13:39:40.457410 1 controller.go:306] "Starting workers" controller="tuf" controllerGroup="rhtas.redhat.com" controllerKind="Tuf" worker count=1 1113: I0127 13:39:40.457397 1 controller.go:303] "Starting Controller" controller="ctlog" controllerGroup="rhtas.redhat.com" controllerKind="CTlog" 1114: I0127 13:39:40.457424 1 controller.go:306] "Starting workers" controller="ctlog" controllerGroup="rhtas.redhat.com" controllerKind="CTlog" worker count=1 1115: I0127 13:39:40.457426 1 controller.go:303] "Starting Controller" controller="trillian" controllerGroup="rhtas.redhat.com" controllerKind="Trillian" 1116: I0127 13:39:40.457435 1 controller.go:306] "Starting workers" controller="trillian" controllerGroup="rhtas.redhat.com" controllerKind="Trillian" worker count=1 1117: I0127 13:41:45.368647 1 warning_handler.go:64] "metadata.finalizers: \"tas.rhtas.redhat.com\": prefer a domain-qualified finalizer name including a path (/) to avoid accidental conflicts with other finalizer writers" controller="securesign" controllerGroup="rhtas.redhat.com" controllerKind="Securesign" Securesign="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="ed04f921-809b-4586-b9fa-1a34090c6195" 1118: I0127 13:41:45.944537 1 initialize.go:42] "deployment is not ready" logger="initialize" controller="fulcio" controllerGroup="rhtas.redhat.com" controllerKind="Fulcio" Fulcio="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="29b79570-dca0-4cd4-b001-b7d66b8443a2" error="deployment not ready(fulcio-server): not available" 1119: I0127 13:41:45.944562 1 initialize.go:47] "Waiting for deployment" logger="initialize" controller="fulcio" controllerGroup="rhtas.redhat.com" controllerKind="Fulcio" Fulcio="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="29b79570-dca0-4cd4-b001-b7d66b8443a2" 1120: I0127 13:41:45.961478 1 initialize.go:42] "deployment is not ready" logger="initialize" controller="fulcio" controllerGroup="rhtas.redhat.com" controllerKind="Fulcio" Fulcio="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="b4d5d337-915c-4aef-8f71-3f37c22f9f47" error="deployment not ready(fulcio-server): not available" 1121: I0127 13:41:45.961858 1 initialize.go:47] "Waiting for deployment" logger="initialize" controller="fulcio" controllerGroup="rhtas.redhat.com" controllerKind="Fulcio" Fulcio="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="b4d5d337-915c-4aef-8f71-3f37c22f9f47" 1122: I0127 13:41:46.010385 1 initialize.go:42] "deployment is not ready" logger="initialize" controller="timestampauthority" controllerGroup="rhtas.redhat.com" controllerKind="TimestampAuthority" TimestampAuthority="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="af6661fa-f93d-4e30-a170-cbb588741328" error="deployment not ready(tsa-server): not available" 1123: I0127 13:41:46.010419 1 initialize.go:47] "Waiting for deployment" logger="initialize" controller="timestampauthority" controllerGroup="rhtas.redhat.com" controllerKind="TimestampAuthority" TimestampAuthority="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="af6661fa-f93d-4e30-a170-cbb588741328" 1124: I0127 13:41:46.023936 1 initialize.go:42] "deployment is not ready" logger="initialize" controller="timestampauthority" controllerGroup="rhtas.redhat.com" controllerKind="TimestampAuthority" TimestampAuthority="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="6e2384db-108d-4fe5-8b2a-c346fa1f95a6" error="deployment not ready(tsa-server): not available" 1125: I0127 13:41:46.023967 1 initialize.go:47] "Waiting for deployment" logger="initialize" controller="timestampauthority" controllerGroup="rhtas.redhat.com" controllerKind="TimestampAuthority" TimestampAuthority="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="6e2384db-108d-4fe5-8b2a-c346fa1f95a6" 1126: I0127 13:41:46.784420 1 initialize.go:40] "deployment is not ready" logger="db initialize" controller="trillian" controllerGroup="rhtas.redhat.com" controllerKind="Trillian" Trillian="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="49922aa6-400f-454e-ae1f-bad455510d27" error="deployment not ready(trillian-db): not available" 1127: I0127 13:41:46.784492 1 initialize.go:45] "Waiting for deployment" logger="db initialize" controller="trillian" controllerGroup="rhtas.redhat.com" controllerKind="Trillian" Trillian="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="49922aa6-400f-454e-ae1f-bad455510d27" 1128: I0127 13:41:46.837081 1 initialize.go:40] "deployment is not ready" logger="db initialize" controller="trillian" controllerGroup="rhtas.redhat.com" controllerKind="Trillian" Trillian="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="b08eb27d-f477-4b25-a41f-f24106810e47" error="deployment not ready(trillian-db): not available" 1129: I0127 13:41:46.837108 1 initialize.go:45] "Waiting for deployment" logger="db initialize" controller="trillian" controllerGroup="rhtas.redhat.com" controllerKind="Trillian" Trillian="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="b08eb27d-f477-4b25-a41f-f24106810e47" 1130: I0127 13:41:49.833000 1 initialize.go:42] "deployment is not ready" logger="initialize" controller="fulcio" controllerGroup="rhtas.redhat.com" controllerKind="Fulcio" Fulcio="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="682658cf-a789-489f-88ff-5152098a32db" error="deployment not ready(fulcio-server): not available" 1131: I0127 13:41:49.833055 1 initialize.go:47] "Waiting for deployment" logger="initialize" controller="fulcio" controllerGroup="rhtas.redhat.com" controllerKind="Fulcio" Fulcio="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="682658cf-a789-489f-88ff-5152098a32db" 1132: I0127 13:42:02.855781 1 initialize.go:42] "deployment is not ready" logger="initialize" controller="timestampauthority" controllerGroup="rhtas.redhat.com" controllerKind="TimestampAuthority" TimestampAuthority="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="92fe614d-231d-4842-8ae2-35dcdfca7303" error="deployment not ready(tsa-server): not available" 1133: I0127 13:42:02.855815 1 initialize.go:47] "Waiting for deployment" logger="initialize" controller="timestampauthority" controllerGroup="rhtas.redhat.com" controllerKind="TimestampAuthority" TimestampAuthority="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="92fe614d-231d-4842-8ae2-35dcdfca7303" 1134: I0127 13:42:20.958777 1 initialize.go:38] "deployment is not ready" logger="server initialize" controller="trillian" controllerGroup="rhtas.redhat.com" controllerKind="Trillian" Trillian="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="d385d03e-11ce-4071-9178-361c433d52f4" error="deployment not ready(trillian-logserver): not available" 1135: I0127 13:42:20.958801 1 initialize.go:43] "Waiting for deployment" logger="server initialize" controller="trillian" controllerGroup="rhtas.redhat.com" controllerKind="Trillian" Trillian="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="d385d03e-11ce-4071-9178-361c433d52f4" 1136: I0127 13:42:20.975279 1 initialize.go:38] "deployment is not ready" logger="server initialize" controller="trillian" controllerGroup="rhtas.redhat.com" controllerKind="Trillian" Trillian="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="7eb06fa4-755a-423a-afaf-d33f7f157518" error="deployment not ready(trillian-logserver): not available" 1137: I0127 13:42:20.975303 1 initialize.go:43] "Waiting for deployment" logger="server initialize" controller="trillian" controllerGroup="rhtas.redhat.com" controllerKind="Trillian" Trillian="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="7eb06fa4-755a-423a-afaf-d33f7f157518" 1138: I0127 13:42:36.962562 1 initialize.go:38] "deployment is not ready" logger="server initialize" controller="trillian" controllerGroup="rhtas.redhat.com" controllerKind="Trillian" Trillian="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="75506fda-a3b3-45b5-90cd-58428a87372b" error="deployment not ready(trillian-logserver): not available" 1139: I0127 13:42:36.962604 1 initialize.go:43] "Waiting for deployment" logger="server initialize" controller="trillian" controllerGroup="rhtas.redhat.com" controllerKind="Trillian" Trillian="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="75506fda-a3b3-45b5-90cd-58428a87372b" 1140: I0127 13:42:38.992457 1 initialize.go:38] "deployment is not ready" logger="server initialize" controller="trillian" controllerGroup="rhtas.redhat.com" controllerKind="Trillian" Trillian="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="6d5083db-3f73-41f0-98f7-f650383a096d" error="deployment not ready(trillian-logserver): not available" 1141: I0127 13:42:38.992491 1 initialize.go:43] "Waiting for deployment" logger="server initialize" controller="trillian" controllerGroup="rhtas.redhat.com" controllerKind="Trillian" Trillian="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="6d5083db-3f73-41f0-98f7-f650383a096d" 1142: I0127 13:43:11.134309 1 server_config.go:209] "Server config secret created" logger="server config" controller="ctlog" controllerGroup="rhtas.redhat.com" controllerKind="CTlog" CTlog="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="1fd9cde9-fbc7-4b77-9f8d-14584147fcdd" secret="ctlog-config-testc59ww" 1143: I0127 13:43:11.230135 1 initialize.go:42] "deployment is not ready" logger="initialize" controller="ctlog" controllerGroup="rhtas.redhat.com" controllerKind="CTlog" CTlog="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="5cb49fe0-d378-46b3-846a-2ba27b2f51ee" error="deployment not ready(ctlog): not available" 1144: I0127 13:43:11.230168 1 initialize.go:47] "Waiting for deployment" logger="initialize" controller="ctlog" controllerGroup="rhtas.redhat.com" controllerKind="CTlog" CTlog="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="5cb49fe0-d378-46b3-846a-2ba27b2f51ee" 1145: I0127 13:43:11.240234 1 initialize.go:42] "deployment is not ready" logger="initialize" controller="ctlog" controllerGroup="rhtas.redhat.com" controllerKind="CTlog" CTlog="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="dba3c019-706b-4fce-919b-e1c439321615" error="deployment not ready(ctlog): not available" 1146: I0127 13:43:11.240284 1 initialize.go:47] "Waiting for deployment" logger="initialize" controller="ctlog" controllerGroup="rhtas.redhat.com" controllerKind="CTlog" CTlog="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="dba3c019-706b-4fce-919b-e1c439321615" 1147: I0127 13:43:16.952948 1 initialize.go:44] "deployment is not ready" logger="initialize" controller="rekor" controllerGroup="rhtas.redhat.com" controllerKind="Rekor" Rekor="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="7acea921-40bb-4c46-9764-6b40b11c5958" error="deployment not ready(rekor-server): not available" 1148: I0127 13:43:16.952993 1 initialize.go:49] "Waiting for deployment" logger="initialize" controller="rekor" controllerGroup="rhtas.redhat.com" controllerKind="Rekor" Rekor="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="7acea921-40bb-4c46-9764-6b40b11c5958" 1149: I0127 13:43:16.993318 1 initialize.go:44] "deployment is not ready" logger="initialize" controller="rekor" controllerGroup="rhtas.redhat.com" controllerKind="Rekor" Rekor="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="46780dba-d15e-49c5-ba87-e735106e8056" error="deployment not ready(rekor-server): not available" 1150: I0127 13:43:16.993339 1 initialize.go:49] "Waiting for deployment" logger="initialize" controller="rekor" controllerGroup="rhtas.redhat.com" controllerKind="Rekor" Rekor="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="46780dba-d15e-49c5-ba87-e735106e8056" 1151: I0127 13:43:24.115068 1 initialize.go:42] "deployment is not ready" logger="initialize" controller="ctlog" controllerGroup="rhtas.redhat.com" controllerKind="CTlog" CTlog="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="2168f745-3ea4-490c-bab8-023ab2871c10" error="deployment not ready(ctlog): not available" 1152: I0127 13:43:24.115091 1 initialize.go:47] "Waiting for deployment" logger="initialize" controller="ctlog" controllerGroup="rhtas.redhat.com" controllerKind="CTlog" CTlog="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="2168f745-3ea4-490c-bab8-023ab2871c10" 1153: I0127 13:43:27.867755 1 initialize.go:44] "deployment is not ready" logger="initialize" controller="rekor" controllerGroup="rhtas.redhat.com" controllerKind="Rekor" Rekor="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="7a88264c-645d-4738-bd11-c397943ec293" error="deployment not ready(rekor-server): not available" 1154: I0127 13:43:27.867780 1 initialize.go:49] "Waiting for deployment" logger="initialize" controller="rekor" controllerGroup="rhtas.redhat.com" controllerKind="Rekor" Rekor="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="7a88264c-645d-4738-bd11-c397943ec293" 1155: I0127 13:43:27.886640 1 initialize.go:44] "deployment is not ready" logger="initialize" controller="rekor" controllerGroup="rhtas.redhat.com" controllerKind="Rekor" Rekor="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="bc0fd92f-2bfb-42c7-9370-7caaaf23eb18" error="deployment not ready(rekor-server): not available" 1156: I0127 13:43:27.886667 1 initialize.go:49] "Waiting for deployment" logger="initialize" controller="rekor" controllerGroup="rhtas.redhat.com" controllerKind="Rekor" Rekor="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="bc0fd92f-2bfb-42c7-9370-7caaaf23eb18" 1157: I0127 13:43:30.174924 1 initialize.go:44] "deployment is not ready" logger="initialize" controller="rekor" controllerGroup="rhtas.redhat.com" controllerKind="Rekor" Rekor="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="e472658c-a369-4985-951e-b4dc861438e8" error="deployment not ready(rekor-server): not available" 1158: I0127 13:43:30.174944 1 initialize.go:49] "Waiting for deployment" logger="initialize" controller="rekor" controllerGroup="rhtas.redhat.com" controllerKind="Rekor" Rekor="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="e472658c-a369-4985-951e-b4dc861438e8" 1159: I0127 13:43:36.391084 1 initialize.go:44] "deployment is not ready" logger="initialize" controller="rekor" controllerGroup="rhtas.redhat.com" controllerKind="Rekor" Rekor="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="bdb30d5c-117d-4d70-b589-6d0199ee121e" error="deployment not ready(rekor-server): not available" 1160: I0127 13:43:36.391103 1 initialize.go:49] "Waiting for deployment" logger="initialize" controller="rekor" controllerGroup="rhtas.redhat.com" controllerKind="Rekor" Rekor="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="bdb30d5c-117d-4d70-b589-6d0199ee121e" 1161: I0127 13:43:36.407451 1 initialize.go:44] "deployment is not ready" logger="initialize" controller="rekor" controllerGroup="rhtas.redhat.com" controllerKind="Rekor" Rekor="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="0df8bc1d-a59a-48ea-b722-2763f0e725c5" error="deployment not ready(rekor-server): not available" 1162: I0127 13:43:36.407469 1 initialize.go:49] "Waiting for deployment" logger="initialize" controller="rekor" controllerGroup="rhtas.redhat.com" controllerKind="Rekor" Rekor="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="0df8bc1d-a59a-48ea-b722-2763f0e725c5" 1163: I0127 13:43:37.191091 1 initialize.go:44] "deployment is not ready" logger="initialize" controller="rekor" controllerGroup="rhtas.redhat.com" controllerKind="Rekor" Rekor="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="b73e1224-fc4a-45e6-9a60-d2c8e77a5338" error="deployment not ready(rekor-server): not available" 1164: I0127 13:43:37.191111 1 initialize.go:49] "Waiting for deployment" logger="initialize" controller="rekor" controllerGroup="rhtas.redhat.com" controllerKind="Rekor" Rekor="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="b73e1224-fc4a-45e6-9a60-d2c8e77a5338" 1165: I0127 13:43:46.187979 1 tuf_init_job.go:63] "Tuf tuf-repository-init is present." logger="controller.tuf.tuf-init job" controller="tuf" controllerGroup="rhtas.redhat.com" controllerKind="Tuf" Tuf="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="e91e230c-7d1f-44d9-8122-fd0629b04df0" Succeeded=0 Failures=0 1166: I0127 13:43:51.194334 1 tuf_init_job.go:63] "Tuf tuf-repository-init is present." logger="controller.tuf.tuf-init job" controller="tuf" controllerGroup="rhtas.redhat.com" controllerKind="Tuf" Tuf="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="2a25793c-ab97-4297-add0-ce99a4589a48" Succeeded=1 Failures=0 1167: I0127 13:43:51.332717 1 initialize.go:43] "deployment is not ready" logger="controller.tuf.initialize" controller="tuf" controllerGroup="rhtas.redhat.com" controllerKind="Tuf" Tuf="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="0222dfa7-21c8-4584-bd81-49bb0ea82e69" error="deployment not ready(tuf): not available" 1168: I0127 13:43:51.332807 1 initialize.go:48] "Waiting for deployment" logger="controller.tuf.initialize" controller="tuf" controllerGroup="rhtas.redhat.com" controllerKind="Tuf" Tuf="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="0222dfa7-21c8-4584-bd81-49bb0ea82e69" 1169: I0127 13:43:51.347727 1 initialize.go:43] "deployment is not ready" logger="controller.tuf.initialize" controller="tuf" controllerGroup="rhtas.redhat.com" controllerKind="Tuf" Tuf="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="18f380cb-013c-4805-bb14-dd852aa17592" error="deployment not ready(tuf): not available" 1170: I0127 13:43:51.347754 1 initialize.go:48] "Waiting for deployment" logger="controller.tuf.initialize" controller="tuf" controllerGroup="rhtas.redhat.com" controllerKind="Tuf" Tuf="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="18f380cb-013c-4805-bb14-dd852aa17592" 1171: I0127 13:44:06.305277 1 initialize.go:43] "deployment is not ready" logger="controller.tuf.initialize" controller="tuf" controllerGroup="rhtas.redhat.com" controllerKind="Tuf" Tuf="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="23c46042-6421-4927-bee1-85910c8c8eee" error="deployment not ready(tuf): not available" 1172: I0127 13:44:06.305301 1 initialize.go:48] "Waiting for deployment" logger="controller.tuf.initialize" controller="tuf" controllerGroup="rhtas.redhat.com" controllerKind="Tuf" Tuf="common-ha-install-test-kddv7/test" namespace="common-ha-install-test-kddv7" name="test" reconcileID="23c46042-6421-4927-bee1-85910c8c8eee" ... 1176: I0127 13:45:54.891553 1 fulcio_controller.go:89] "namespace is marked for deletion, stopping reconciliation" controller="fulcio" controllerGroup="rhtas.redhat.com" controllerKind="Fulcio" Fulcio="common-ha-install-test-kddv7/test" name="test" reconcileID="fd10e344-61a8-4563-8623-6725c9779b43" namespace="common-ha-install-test-kddv7" 1177: I0127 13:45:54.939709 1 rekor_controller.go:95] "namespace is marked for deletion, stopping reconciliation" controller="rekor" controllerGroup="rhtas.redhat.com" controllerKind="Rekor" Rekor="common-ha-install-test-kddv7/test" name="test" reconcileID="e1094a7d-0a10-426f-a12d-4244288bcbbb" namespace="common-ha-install-test-kddv7" 1178: I0127 13:45:54.949662 1 rekor_controller.go:95] "namespace is marked for deletion, stopping reconciliation" controller="rekor" controllerGroup="rhtas.redhat.com" controllerKind="Rekor" Rekor="common-ha-install-test-kddv7/test" name="test" reconcileID="491a84d1-6ba1-4ef2-9139-a3ed90734ac4" namespace="common-ha-install-test-kddv7" 1179: I0127 13:45:55.041445 1 rekor_controller.go:95] "namespace is marked for deletion, stopping reconciliation" controller="rekor" controllerGroup="rhtas.redhat.com" controllerKind="Rekor" Rekor="common-ha-install-test-kddv7/test" name="test" reconcileID="9db18d84-b93b-4404-8d42-e3eb2f7a74eb" namespace="common-ha-install-test-kddv7" 1180: I0127 13:45:55.074984 1 rekor_controller.go:95] "namespace is marked for deletion, stopping reconciliation" controller="rekor" controllerGroup="rhtas.redhat.com" controllerKind="Rekor" Rekor="common-ha-install-test-kddv7/test" name="test" reconcileID="5a961855-0706-4a80-b6f3-b6f0e52587b4" namespace="common-ha-install-test-kddv7" 1181: I0127 13:45:55.102302 1 trillian_controller.go:92] "namespace is marked for deletion, stopping reconciliation" controller="trillian" controllerGroup="rhtas.redhat.com" controllerKind="Trillian" Trillian="common-ha-install-test-kddv7/test" name="test" reconcileID="da8677bd-4d74-4a94-8834-6a1a32f49fdb" namespace="common-ha-install-test-kddv7" 1182: I0127 13:45:55.142509 1 trillian_controller.go:92] "namespace is marked for deletion, stopping reconciliation" controller="trillian" controllerGroup="rhtas.redhat.com" controllerKind="Trillian" Trillian="common-ha-install-test-kddv7/test" name="test" reconcileID="45a826ee-a44b-4321-b035-53d38c823ec4" namespace="common-ha-install-test-kddv7" 1183: I0127 13:45:55.155636 1 rekor_controller.go:95] "namespace is marked for deletion, stopping reconciliation" controller="rekor" controllerGroup="rhtas.redhat.com" controllerKind="Rekor" Rekor="common-ha-install-test-kddv7/test" name="test" reconcileID="394fdf8c-b523-4d95-bc53-267fa7ea4440" namespace="common-ha-install-test-kddv7" 1184: I0127 13:45:55.212432 1 trillian_controller.go:92] "namespace is marked for deletion, stopping reconciliation" controller="trillian" controllerGroup="rhtas.redhat.com" controllerKind="Trillian" Trillian="common-ha-install-test-kddv7/test" name="test" reconcileID="7723a7ca-298b-40c3-abf7-1fa59d201545" namespace="common-ha-install-test-kddv7" 1185: I0127 13:45:55.294232 1 timestampauthority_controller.go:88] "namespace is marked for deletion, stopping reconciliation" controller="timestampauthority" controllerGroup="rhtas.redhat.com" controllerKind="TimestampAuthority" TimestampAuthority="common-ha-install-test-kddv7/test" name="test" reconcileID="323504da-cf12-4651-a959-3a1b7ae27719" namespace="common-ha-install-test-kddv7" 1186: I0127 13:45:55.305501 1 trillian_controller.go:92] "namespace is marked for deletion, stopping reconciliation" controller="trillian" controllerGroup="rhtas.redhat.com" controllerKind="Trillian" Trillian="common-ha-install-test-kddv7/test" name="test" reconcileID="89ce1f9d-f8af-4a49-983b-22e83c026e05" namespace="common-ha-install-test-kddv7" 1187: I0127 13:45:55.325071 1 trillian_controller.go:92] "namespace is marked for deletion, stopping reconciliation" controller="trillian" controllerGroup="rhtas.redhat.com" controllerKind="Trillian" Trillian="common-ha-install-test-kddv7/test" name="test" reconcileID="59a66db1-44ad-4705-aee6-35e10f1078a9" namespace="common-ha-install-test-kddv7" 1188: I0127 13:45:55.334062 1 timestampauthority_controller.go:88] "namespace is marked for deletion, stopping reconciliation" controller="timestampauthority" controllerGroup="rhtas.redhat.com" controllerKind="TimestampAuthority" TimestampAuthority="common-ha-install-test-kddv7/test" name="test" reconcileID="a221ace0-cc3c-4061-9651-0ca9857bd937" namespace="common-ha-install-test-kddv7" 1189: I0127 13:45:55.355502 1 tuf_controller.go:90] "namespace is marked for deletion, stopping reconciliation" logger="controller.tuf" controller="tuf" controllerGroup="rhtas.redhat.com" controllerKind="Tuf" Tuf="common-ha-install-test-kddv7/test" name="test" reconcileID="84d508f6-2495-4565-80ec-2661d27a75a6" namespace="common-ha-install-test-kddv7" 1190: I0127 13:46:02.740986 1 warning_handler.go:64] "metadata.finalizers: \"tas.rhtas.redhat.com\": prefer a domain-qualified finalizer name including a path (/) to avoid accidental conflicts with other finalizer writers" controller="securesign" controllerGroup="rhtas.redhat.com" controllerKind="Securesign" Securesign="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="b43d1556-4a85-464c-bc3b-7fa92a8fff9f" 1191: E0127 13:46:02.926635 1 base_action.go:93] "error during action execution" err="could not create ingress object: admission webhook \"validate.nginx.ingress.kubernetes.io\" denied the request: host \"fulcio-server.local\" and path \"/\" is already defined in ingress common-ha-install-test-kddv7/fulcio-server" logger="ingress" controller="fulcio" controllerGroup="rhtas.redhat.com" controllerKind="Fulcio" Fulcio="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="905b633a-02a1-4516-8943-fa867121758b" 1192: E0127 13:46:02.926672 1 controller.go:495] "Reconciler error" err="could not create ingress object: admission webhook \"validate.nginx.ingress.kubernetes.io\" denied the request: host \"fulcio-server.local\" and path \"/\" is already defined in ingress common-ha-install-test-kddv7/fulcio-server" controller="fulcio" controllerGroup="rhtas.redhat.com" controllerKind="Fulcio" Fulcio="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="905b633a-02a1-4516-8943-fa867121758b" 1193: E0127 13:46:02.948570 1 base_action.go:93] "error during action execution" err="could not create ingress object: admission webhook \"validate.nginx.ingress.kubernetes.io\" denied the request: host \"fulcio-server.local\" and path \"/\" is already defined in ingress common-ha-install-test-kddv7/fulcio-server" logger="ingress" controller="fulcio" controllerGroup="rhtas.redhat.com" controllerKind="Fulcio" Fulcio="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="5fadb4d1-e0c4-4bb5-bbeb-095b94a3cb14" 1194: E0127 13:46:02.948614 1 controller.go:495] "Reconciler error" err="could not create ingress object: admission webhook \"validate.nginx.ingress.kubernetes.io\" denied the request: host \"fulcio-server.local\" and path \"/\" is already defined in ingress common-ha-install-test-kddv7/fulcio-server" controller="fulcio" controllerGroup="rhtas.redhat.com" controllerKind="Fulcio" Fulcio="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="5fadb4d1-e0c4-4bb5-bbeb-095b94a3cb14" 1195: E0127 13:46:02.985078 1 base_action.go:93] "error during action execution" err="could not create ingress object: admission webhook \"validate.nginx.ingress.kubernetes.io\" denied the request: host \"fulcio-server.local\" and path \"/\" is already defined in ingress common-ha-install-test-kddv7/fulcio-server" logger="ingress" controller="fulcio" controllerGroup="rhtas.redhat.com" controllerKind="Fulcio" Fulcio="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="c8067619-f856-4f02-a47f-c98b842a8a6d" 1196: E0127 13:46:02.985117 1 controller.go:495] "Reconciler error" err="could not create ingress object: admission webhook \"validate.nginx.ingress.kubernetes.io\" denied the request: host \"fulcio-server.local\" and path \"/\" is already defined in ingress common-ha-install-test-kddv7/fulcio-server" controller="fulcio" controllerGroup="rhtas.redhat.com" controllerKind="Fulcio" Fulcio="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="c8067619-f856-4f02-a47f-c98b842a8a6d" 1197: E0127 13:46:03.016469 1 base_action.go:93] "error during action execution" err="could not create ingress object: admission webhook \"validate.nginx.ingress.kubernetes.io\" denied the request: host \"fulcio-server.local\" and path \"/\" is already defined in ingress common-ha-install-test-kddv7/fulcio-server" logger="ingress" controller="fulcio" controllerGroup="rhtas.redhat.com" controllerKind="Fulcio" Fulcio="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="434d68d3-a82a-4274-91b9-5e82c6eeb4ca" 1198: E0127 13:46:03.016506 1 controller.go:495] "Reconciler error" err="could not create ingress object: admission webhook \"validate.nginx.ingress.kubernetes.io\" denied the request: host \"fulcio-server.local\" and path \"/\" is already defined in ingress common-ha-install-test-kddv7/fulcio-server" controller="fulcio" controllerGroup="rhtas.redhat.com" controllerKind="Fulcio" Fulcio="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="434d68d3-a82a-4274-91b9-5e82c6eeb4ca" 1199: E0127 13:46:03.075841 1 base_action.go:93] "error during action execution" err="could not create ingress object: admission webhook \"validate.nginx.ingress.kubernetes.io\" denied the request: host \"fulcio-server.local\" and path \"/\" is already defined in ingress common-ha-install-test-kddv7/fulcio-server" logger="ingress" controller="fulcio" controllerGroup="rhtas.redhat.com" controllerKind="Fulcio" Fulcio="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="8d5b6b60-aee2-4914-ac88-6ec241cf258d" 1200: E0127 13:46:03.077075 1 controller.go:495] "Reconciler error" err="could not create ingress object: admission webhook \"validate.nginx.ingress.kubernetes.io\" denied the request: host \"fulcio-server.local\" and path \"/\" is already defined in ingress common-ha-install-test-kddv7/fulcio-server" controller="fulcio" controllerGroup="rhtas.redhat.com" controllerKind="Fulcio" Fulcio="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="8d5b6b60-aee2-4914-ac88-6ec241cf258d" 1201: E0127 13:46:03.203135 1 base_action.go:93] "error during action execution" err="could not create ingress object: admission webhook \"validate.nginx.ingress.kubernetes.io\" denied the request: host \"fulcio-server.local\" and path \"/\" is already defined in ingress common-ha-install-test-kddv7/fulcio-server" logger="ingress" controller="fulcio" controllerGroup="rhtas.redhat.com" controllerKind="Fulcio" Fulcio="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="f7b7357e-43a3-4aa7-9ea7-dc6c87bc7aaa" 1202: E0127 13:46:03.203177 1 controller.go:495] "Reconciler error" err="could not create ingress object: admission webhook \"validate.nginx.ingress.kubernetes.io\" denied the request: host \"fulcio-server.local\" and path \"/\" is already defined in ingress common-ha-install-test-kddv7/fulcio-server" controller="fulcio" controllerGroup="rhtas.redhat.com" controllerKind="Fulcio" Fulcio="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="f7b7357e-43a3-4aa7-9ea7-dc6c87bc7aaa" 1203: E0127 13:46:03.525273 1 base_action.go:93] "error during action execution" err="could not create ingress object: admission webhook \"validate.nginx.ingress.kubernetes.io\" denied the request: host \"tsa-server.local\" and path \"/\" is already defined in ingress common-ha-install-test-kddv7/tsa-server" logger="ingress" controller="timestampauthority" controllerGroup="rhtas.redhat.com" controllerKind="TimestampAuthority" TimestampAuthority="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="c1b33974-73a4-478e-b76d-c2bbd7ffc6d8" 1204: E0127 13:46:03.525313 1 controller.go:495] "Reconciler error" err="could not create ingress object: admission webhook \"validate.nginx.ingress.kubernetes.io\" denied the request: host \"tsa-server.local\" and path \"/\" is already defined in ingress common-ha-install-test-kddv7/tsa-server" controller="timestampauthority" controllerGroup="rhtas.redhat.com" controllerKind="TimestampAuthority" TimestampAuthority="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="c1b33974-73a4-478e-b76d-c2bbd7ffc6d8" 1205: E0127 13:46:03.525843 1 base_action.go:93] "error during action execution" err="could not create ingress object: admission webhook \"validate.nginx.ingress.kubernetes.io\" denied the request: host \"fulcio-server.local\" and path \"/\" is already defined in ingress common-ha-install-test-kddv7/fulcio-server" logger="ingress" controller="fulcio" controllerGroup="rhtas.redhat.com" controllerKind="Fulcio" Fulcio="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="666d0c3d-28c3-4b96-ba48-bf04e3600ef7" 1206: E0127 13:46:03.525868 1 controller.go:495] "Reconciler error" err="could not create ingress object: admission webhook \"validate.nginx.ingress.kubernetes.io\" denied the request: host \"fulcio-server.local\" and path \"/\" is already defined in ingress common-ha-install-test-kddv7/fulcio-server" controller="fulcio" controllerGroup="rhtas.redhat.com" controllerKind="Fulcio" Fulcio="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="666d0c3d-28c3-4b96-ba48-bf04e3600ef7" 1207: E0127 13:46:03.609968 1 base_action.go:93] "error during action execution" err="could not create ingress object: admission webhook \"validate.nginx.ingress.kubernetes.io\" denied the request: host \"tsa-server.local\" and path \"/\" is already defined in ingress common-ha-install-test-kddv7/tsa-server" logger="ingress" controller="timestampauthority" controllerGroup="rhtas.redhat.com" controllerKind="TimestampAuthority" TimestampAuthority="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="21a10b01-5a2a-4e90-82a2-6f7061326477" 1208: E0127 13:46:03.612795 1 controller.go:495] "Reconciler error" err="could not create ingress object: admission webhook \"validate.nginx.ingress.kubernetes.io\" denied the request: host \"tsa-server.local\" and path \"/\" is already defined in ingress common-ha-install-test-kddv7/tsa-server" controller="timestampauthority" controllerGroup="rhtas.redhat.com" controllerKind="TimestampAuthority" TimestampAuthority="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="21a10b01-5a2a-4e90-82a2-6f7061326477" 1209: E0127 13:46:03.682479 1 base_action.go:93] "error during action execution" err="could not create ingress object: admission webhook \"validate.nginx.ingress.kubernetes.io\" denied the request: host \"tsa-server.local\" and path \"/\" is already defined in ingress common-ha-install-test-kddv7/tsa-server" logger="ingress" controller="timestampauthority" controllerGroup="rhtas.redhat.com" controllerKind="TimestampAuthority" TimestampAuthority="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="4d2e6755-c012-4d37-ba1e-961503e3f909" 1210: E0127 13:46:03.682777 1 controller.go:495] "Reconciler error" err="could not create ingress object: admission webhook \"validate.nginx.ingress.kubernetes.io\" denied the request: host \"tsa-server.local\" and path \"/\" is already defined in ingress common-ha-install-test-kddv7/tsa-server" controller="timestampauthority" controllerGroup="rhtas.redhat.com" controllerKind="TimestampAuthority" TimestampAuthority="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="4d2e6755-c012-4d37-ba1e-961503e3f909" 1211: E0127 13:46:03.754782 1 base_action.go:93] "error during action execution" err="could not create ingress object: admission webhook \"validate.nginx.ingress.kubernetes.io\" denied the request: host \"tsa-server.local\" and path \"/\" is already defined in ingress common-ha-install-test-kddv7/tsa-server" logger="ingress" controller="timestampauthority" controllerGroup="rhtas.redhat.com" controllerKind="TimestampAuthority" TimestampAuthority="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="a2bf9ab8-fd89-4fd1-bb3a-6e01a5206ba5" 1212: E0127 13:46:03.754824 1 controller.go:495] "Reconciler error" err="could not create ingress object: admission webhook \"validate.nginx.ingress.kubernetes.io\" denied the request: host \"tsa-server.local\" and path \"/\" is already defined in ingress common-ha-install-test-kddv7/tsa-server" controller="timestampauthority" controllerGroup="rhtas.redhat.com" controllerKind="TimestampAuthority" TimestampAuthority="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="a2bf9ab8-fd89-4fd1-bb3a-6e01a5206ba5" 1213: E0127 13:46:03.821686 1 base_action.go:93] "error during action execution" err="could not create ingress object: admission webhook \"validate.nginx.ingress.kubernetes.io\" denied the request: host \"tsa-server.local\" and path \"/\" is already defined in ingress common-ha-install-test-kddv7/tsa-server" logger="ingress" controller="timestampauthority" controllerGroup="rhtas.redhat.com" controllerKind="TimestampAuthority" TimestampAuthority="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="600befc9-b734-4162-abc7-f11dab02def5" 1214: E0127 13:46:03.821829 1 controller.go:495] "Reconciler error" err="could not create ingress object: admission webhook \"validate.nginx.ingress.kubernetes.io\" denied the request: host \"tsa-server.local\" and path \"/\" is already defined in ingress common-ha-install-test-kddv7/tsa-server" controller="timestampauthority" controllerGroup="rhtas.redhat.com" controllerKind="TimestampAuthority" TimestampAuthority="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="600befc9-b734-4162-abc7-f11dab02def5" 1215: I0127 13:46:03.844309 1 initialize.go:40] "deployment is not ready" logger="db initialize" controller="trillian" controllerGroup="rhtas.redhat.com" controllerKind="Trillian" Trillian="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="375382eb-af0c-4c30-a0d3-65fa86b2087b" error="deployment not ready(trillian-db): not available" 1216: I0127 13:46:03.844449 1 initialize.go:45] "Waiting for deployment" logger="db initialize" controller="trillian" controllerGroup="rhtas.redhat.com" controllerKind="Trillian" Trillian="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="375382eb-af0c-4c30-a0d3-65fa86b2087b" 1217: E0127 13:46:03.890874 1 base_action.go:93] "error during action execution" err="could not create ingress object: admission webhook \"validate.nginx.ingress.kubernetes.io\" denied the request: host \"fulcio-server.local\" and path \"/\" is already defined in ingress common-ha-install-test-kddv7/fulcio-server" logger="ingress" controller="fulcio" controllerGroup="rhtas.redhat.com" controllerKind="Fulcio" Fulcio="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="368e065f-df61-416c-886f-d5a161832be5" 1218: E0127 13:46:03.891487 1 controller.go:495] "Reconciler error" err="could not create ingress object: admission webhook \"validate.nginx.ingress.kubernetes.io\" denied the request: host \"fulcio-server.local\" and path \"/\" is already defined in ingress common-ha-install-test-kddv7/fulcio-server" controller="fulcio" controllerGroup="rhtas.redhat.com" controllerKind="Fulcio" Fulcio="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="368e065f-df61-416c-886f-d5a161832be5" 1219: I0127 13:46:03.924867 1 initialize.go:40] "deployment is not ready" logger="db initialize" controller="trillian" controllerGroup="rhtas.redhat.com" controllerKind="Trillian" Trillian="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="0ba1ca5e-8ebe-43e4-a1d5-37c654ece90e" error="deployment not ready(trillian-db): not available" 1220: I0127 13:46:03.925133 1 initialize.go:45] "Waiting for deployment" logger="db initialize" controller="trillian" controllerGroup="rhtas.redhat.com" controllerKind="Trillian" Trillian="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="0ba1ca5e-8ebe-43e4-a1d5-37c654ece90e" 1221: E0127 13:46:03.941119 1 base_action.go:93] "error during action execution" err="could not create ingress object: admission webhook \"validate.nginx.ingress.kubernetes.io\" denied the request: host \"tsa-server.local\" and path \"/\" is already defined in ingress common-ha-install-test-kddv7/tsa-server" logger="ingress" controller="timestampauthority" controllerGroup="rhtas.redhat.com" controllerKind="TimestampAuthority" TimestampAuthority="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="76205cd7-fc02-465e-893b-3d620de25086" 1222: E0127 13:46:03.941205 1 controller.go:495] "Reconciler error" err="could not create ingress object: admission webhook \"validate.nginx.ingress.kubernetes.io\" denied the request: host \"tsa-server.local\" and path \"/\" is already defined in ingress common-ha-install-test-kddv7/tsa-server" controller="timestampauthority" controllerGroup="rhtas.redhat.com" controllerKind="TimestampAuthority" TimestampAuthority="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="76205cd7-fc02-465e-893b-3d620de25086" 1223: E0127 13:46:04.119843 1 base_action.go:93] "error during action execution" err="could not create ingress object: admission webhook \"validate.nginx.ingress.kubernetes.io\" denied the request: host \"tsa-server.local\" and path \"/\" is already defined in ingress common-ha-install-test-kddv7/tsa-server" logger="ingress" controller="timestampauthority" controllerGroup="rhtas.redhat.com" controllerKind="TimestampAuthority" TimestampAuthority="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="2b127c2e-8f6d-4b65-864d-4fa1fe44584f" 1224: E0127 13:46:04.119881 1 controller.go:495] "Reconciler error" err="could not create ingress object: admission webhook \"validate.nginx.ingress.kubernetes.io\" denied the request: host \"tsa-server.local\" and path \"/\" is already defined in ingress common-ha-install-test-kddv7/tsa-server" controller="timestampauthority" controllerGroup="rhtas.redhat.com" controllerKind="TimestampAuthority" TimestampAuthority="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="2b127c2e-8f6d-4b65-864d-4fa1fe44584f" 1225: E0127 13:46:04.449227 1 base_action.go:93] "error during action execution" err="could not create ingress object: admission webhook \"validate.nginx.ingress.kubernetes.io\" denied the request: host \"tsa-server.local\" and path \"/\" is already defined in ingress common-ha-install-test-kddv7/tsa-server" logger="ingress" controller="timestampauthority" controllerGroup="rhtas.redhat.com" controllerKind="TimestampAuthority" TimestampAuthority="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="61bcadd7-a142-4673-942a-deb75f798a4a" 1226: E0127 13:46:04.449263 1 controller.go:495] "Reconciler error" err="could not create ingress object: admission webhook \"validate.nginx.ingress.kubernetes.io\" denied the request: host \"tsa-server.local\" and path \"/\" is already defined in ingress common-ha-install-test-kddv7/tsa-server" controller="timestampauthority" controllerGroup="rhtas.redhat.com" controllerKind="TimestampAuthority" TimestampAuthority="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="61bcadd7-a142-4673-942a-deb75f798a4a" 1227: E0127 13:46:04.540148 1 base_action.go:93] "error during action execution" err="could not create ingress object: admission webhook \"validate.nginx.ingress.kubernetes.io\" denied the request: host \"fulcio-server.local\" and path \"/\" is already defined in ingress common-ha-install-test-kddv7/fulcio-server" logger="ingress" controller="fulcio" controllerGroup="rhtas.redhat.com" controllerKind="Fulcio" Fulcio="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="ed06d61b-e49e-44df-9e86-66d75a7f0e67" 1228: E0127 13:46:04.540195 1 controller.go:495] "Reconciler error" err="could not create ingress object: admission webhook \"validate.nginx.ingress.kubernetes.io\" denied the request: host \"fulcio-server.local\" and path \"/\" is already defined in ingress common-ha-install-test-kddv7/fulcio-server" controller="fulcio" controllerGroup="rhtas.redhat.com" controllerKind="Fulcio" Fulcio="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="ed06d61b-e49e-44df-9e86-66d75a7f0e67" 1229: E0127 13:46:05.082752 1 base_action.go:93] "error during action execution" err="could not create ingress object: admission webhook \"validate.nginx.ingress.kubernetes.io\" denied the request: host \"fulcio-server.local\" and path \"/\" is already defined in ingress common-ha-install-test-kddv7/fulcio-server" logger="ingress" controller="fulcio" controllerGroup="rhtas.redhat.com" controllerKind="Fulcio" Fulcio="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="31fef289-c998-4d9c-8002-47a924d9ef9c" 1230: E0127 13:46:05.082798 1 controller.go:495] "Reconciler error" err="could not create ingress object: admission webhook \"validate.nginx.ingress.kubernetes.io\" denied the request: host \"fulcio-server.local\" and path \"/\" is already defined in ingress common-ha-install-test-kddv7/fulcio-server" controller="fulcio" controllerGroup="rhtas.redhat.com" controllerKind="Fulcio" Fulcio="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="31fef289-c998-4d9c-8002-47a924d9ef9c" 1231: E0127 13:46:05.098232 1 base_action.go:93] "error during action execution" err="could not create ingress object: admission webhook \"validate.nginx.ingress.kubernetes.io\" denied the request: host \"tsa-server.local\" and path \"/\" is already defined in ingress common-ha-install-test-kddv7/tsa-server" logger="ingress" controller="timestampauthority" controllerGroup="rhtas.redhat.com" controllerKind="TimestampAuthority" TimestampAuthority="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="5c0439d9-0f25-4ec4-a414-3401e918f209" 1232: E0127 13:46:05.098320 1 controller.go:495] "Reconciler error" err="could not create ingress object: admission webhook \"validate.nginx.ingress.kubernetes.io\" denied the request: host \"tsa-server.local\" and path \"/\" is already defined in ingress common-ha-install-test-kddv7/tsa-server" controller="timestampauthority" controllerGroup="rhtas.redhat.com" controllerKind="TimestampAuthority" TimestampAuthority="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="5c0439d9-0f25-4ec4-a414-3401e918f209" 1233: E0127 13:46:06.385397 1 base_action.go:93] "error during action execution" err="could not create ingress object: admission webhook \"validate.nginx.ingress.kubernetes.io\" denied the request: host \"tsa-server.local\" and path \"/\" is already defined in ingress common-ha-install-test-kddv7/tsa-server" logger="ingress" controller="timestampauthority" controllerGroup="rhtas.redhat.com" controllerKind="TimestampAuthority" TimestampAuthority="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="d90f6135-5768-4418-8581-4f3554128a46" 1234: E0127 13:46:06.385427 1 controller.go:495] "Reconciler error" err="could not create ingress object: admission webhook \"validate.nginx.ingress.kubernetes.io\" denied the request: host \"tsa-server.local\" and path \"/\" is already defined in ingress common-ha-install-test-kddv7/tsa-server" controller="timestampauthority" controllerGroup="rhtas.redhat.com" controllerKind="TimestampAuthority" TimestampAuthority="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="d90f6135-5768-4418-8581-4f3554128a46" 1235: E0127 13:46:07.651395 1 base_action.go:93] "error during action execution" err="could not create ingress object: admission webhook \"validate.nginx.ingress.kubernetes.io\" denied the request: host \"fulcio-server.local\" and path \"/\" is already defined in ingress common-ha-install-test-kddv7/fulcio-server" logger="ingress" controller="fulcio" controllerGroup="rhtas.redhat.com" controllerKind="Fulcio" Fulcio="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="0274e892-21ec-4cac-99b7-416c80971bfc" 1236: E0127 13:46:07.651438 1 controller.go:495] "Reconciler error" err="could not create ingress object: admission webhook \"validate.nginx.ingress.kubernetes.io\" denied the request: host \"fulcio-server.local\" and path \"/\" is already defined in ingress common-ha-install-test-kddv7/fulcio-server" controller="fulcio" controllerGroup="rhtas.redhat.com" controllerKind="Fulcio" Fulcio="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="0274e892-21ec-4cac-99b7-416c80971bfc" 1237: E0127 13:46:08.952993 1 base_action.go:93] "error during action execution" err="could not create ingress object: admission webhook \"validate.nginx.ingress.kubernetes.io\" denied the request: host \"tsa-server.local\" and path \"/\" is already defined in ingress common-ha-install-test-kddv7/tsa-server" logger="ingress" controller="timestampauthority" controllerGroup="rhtas.redhat.com" controllerKind="TimestampAuthority" TimestampAuthority="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="f51aee52-4e5e-40e8-a430-3e998d0e1429" 1238: E0127 13:46:08.953066 1 controller.go:495] "Reconciler error" err="could not create ingress object: admission webhook \"validate.nginx.ingress.kubernetes.io\" denied the request: host \"tsa-server.local\" and path \"/\" is already defined in ingress common-ha-install-test-kddv7/tsa-server" controller="timestampauthority" controllerGroup="rhtas.redhat.com" controllerKind="TimestampAuthority" TimestampAuthority="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="f51aee52-4e5e-40e8-a430-3e998d0e1429" 1239: E0127 13:46:09.072504 1 base_action.go:93] "error during action execution" err="could not create ingress object: admission webhook \"validate.nginx.ingress.kubernetes.io\" denied the request: host \"tsa-server.local\" and path \"/\" is already defined in ingress common-ha-install-test-kddv7/tsa-server" logger="ingress" controller="timestampauthority" controllerGroup="rhtas.redhat.com" controllerKind="TimestampAuthority" TimestampAuthority="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="a950173e-4f88-4428-9f47-a71ffd78f42d" 1240: E0127 13:46:09.072550 1 controller.go:495] "Reconciler error" err="could not create ingress object: admission webhook \"validate.nginx.ingress.kubernetes.io\" denied the request: host \"tsa-server.local\" and path \"/\" is already defined in ingress common-ha-install-test-kddv7/tsa-server" controller="timestampauthority" controllerGroup="rhtas.redhat.com" controllerKind="TimestampAuthority" TimestampAuthority="non-ha-to-ha-test-2k6nh/test" namespace="non-ha-to-ha-test-2k6nh" name="test" reconcileID="a950173e-4f88-4428-9f47-a71ffd78f42d" 1241: I0127 13:46:09.435063 1 ctlog_controller.go:96] "namespace is marked for deletion, stopping reconciliation" controller="ctlog" controllerGroup="rhtas.redhat.com" controllerKind="CTlog" CTlog="common-ha-install-test-kddv7/test" name="test" reconcileID="6a429815-cd5b-4e62-a9fe-bc4e2574342d" namespace="common-ha-install-test-kddv7" ... 1253: I0127 13:46:09.559996 1 rekor_controller.go:95] "namespace is marked for deletion, stopping reconciliation" controller="rekor" controllerGroup="rhtas.redhat.com" controllerKind="Rekor" Rekor="common-ha-install-test-kddv7/test" name="test" reconcileID="548e0f1b-1d51-4c66-a36b-e7c58255ed31" namespace="common-ha-install-test-kddv7" 1254: I0127 13:46:09.568215 1 rekor_controller.go:95] "namespace is marked for deletion, stopping reconciliation" controller="rekor" controllerGroup="rhtas.redhat.com" controllerKind="Rekor" Rekor="common-ha-install-test-kddv7/test" name="test" reconcileID="d46e5d24-5410-4af2-bcce-b16ad5de6ace" namespace="common-ha-install-test-kddv7" 1255: I0127 13:46:09.574967 1 rekor_controller.go:95] "namespace is marked for deletion, stopping reconciliation" controller="rekor" controllerGroup="rhtas.redhat.com" controllerKind="Rekor" Rekor="common-ha-install-test-kddv7/test" name="test" reconcileID="48647129-4cc8-4c20-b5a9-cbf0100b9695" namespace="common-ha-install-test-kddv7" 1256: I0127 13:46:09.581656 1 trillian_controller.go:92] "namespace is marked for deletion, stopping reconciliation" controller="trillian" controllerGroup="rhtas.redhat.com" controllerKind="Trillian" Trillian="common-ha-install-test-kddv7/test" name="test" reconcileID="a5d26f56-0dfb-44ee-9dbd-7b0d340c17c0" namespace="common-ha-install-test-kddv7" 1257: I0127 13:46:09.591728 1 trillian_controller.go:92] "namespace is marked for deletion, stopping reconciliation" controller="trillian" controllerGroup="rhtas.redhat.com" controllerKind="Trillian" Trillian="common-ha-install-test-kddv7/test" name="test" reconcileID="8035f3b9-ec4e-402a-96f3-0b956238706e" namespace="common-ha-install-test-kddv7" 1258: I0127 13:46:09.597151 1 trillian_controller.go:92] "namespace is marked for deletion, stopping reconciliation" controller="trillian" controllerGroup="rhtas.redhat.com" controllerKind="Trillian" Trillian="common-ha-install-test-kddv7/test" name="test" reconcileID="eaf03fb6-9478-49ec-9d1c-0f5b842a73ab" namespace="common-ha-install-test-kddv7" 1259: I0127 13:46:09.606641 1 timestampauthority_controller.go:88] "namespace is marked for deletion, stopping reconciliation" con...