CNTRLPLANE-2196: reg ote bin for cluster auth operator #30735

p0lyn0mial · 2026-01-29T13:08:06Z

are you sure that existing e2e tests for cluster-authentication-operator are executed in serial ?

xref: https://github.com/openshift/cluster-authentication-operator/blob/master/cmd/cluster-authentication-operator-tests-ext/main.go#L65

(^ not strictly related to the changes introduced in this PR :) )

Hi @p0lyn0mial
Tried checking the logs, understanding the framework. This is what i got.

Tests run serially by default in Go. You only need to explicitly opt-in to parallel execution

So currently we are executing cases from test/e2e directory by using Ginkgo framework to run the tests in serial way.

As per our OTE implementation we have assigned names like

[sig-auth] authentication operator [Operator][Certs][Serial] TestRouterCerts [sig-auth] authentication operator [Operator][Routes][Serial] TestCustomRouterCerts [sig-auth] authentication operator [OIDC][Serial] TestGitLabAsOIDCPasswordGrantCheck [sig-auth] authentication operator [OIDC][Serial] TestKeycloakAsOIDCPasswordGrantCheckAndGroupSync [sig-auth] authentication operator [Templates][Serial] TestTemplatesConfig [sig-auth] authentication operator [Tokens][Serial] TestTokenInactivityTimeout

OTE suite configuration with parallelism: 1, Tests run sequentially (serially), not concurrently

Dedicated serial CI job in OpenShift's test infrastructure in PR: https://github.com/openshift/release/pull/73999/changes

The tests in test/e2e/ modify cluster-wide global state (OAuth config, routes, templates, etc.). If they ran in parallel the test will fail.

Execution logs not from my PR

https://gcsweb-ci.apps.ci.l2s4.p1.openshiftapps.com/gcs/test-platform-results/pr-logs/pull/openshift_cluster-authentication-operator/829/pull-ci-openshift-cluster-authentic ation-operator-master-e2e-operator/2016110000199438336/artifacts/e2e-operator/test/artifacts/junit_report.xml

ANALYZING ACTUAL JUNIT XML FROM PR #829
========================================
TEST DURATIONS FROM junit_report.xml:
- TestCustomRouterCerts: 42.83s
- TestGitLabAsOIDCPasswordGrantCheck: 291.25s
- TestKeycloakAsOIDCPasswordGrantCheckAndGroupSync: 186.63s
- TestRouterCerts: 2.21s
- TestTemplatesConfig: 70.34s
- TestTokenInactivityTimeout: 2202.64s
└─ unset-inactivity-timeout-client-timeout: 610.57s
└─ with-client-timeout: 430.72s
└─ with-inactivity-timeout: 730.90s
└─ without-inactivity-timeout: 430.43s

CALCULATION: ----------- Sum of individual test times: 42.83 + 291.25 + 186.63 + 2.21 + 70.34 + 2202.64 = 2795.90s Total suite time reported in XML: 2796.978s Difference: 2796.978 - 2795.90 = 1.078s (just overhead) ANALYSIS: --------- If PARALLEL: Total would be ≈ longest test (2202.64s) If SERIAL: Total would be ≈ sum of all tests (2795.90s) Actual total: 2796.978s ≈ Sum of all tests ✓ CONCLUSION: Tests ran SERIALLY (one after another)

mathematical proof from the actual CI logs that tests ran serially, not in parallel!

Hi @p0lyn0mial Summary related to the tests under path test/e2e:

Test Name: TestRouterCerts Implementation File:Line: certs.go:31 Test Wrapper File: certs_test.go:7 Current Label: [Operator][Certs][Serial] Analysis: 🚫 Serial - Modifies global IngressController, changes cluster config ──────────────────────────────────────── Test Name: TestCustomRouterCerts Implementation File:Line: custom_route.go:33 Test Wrapper File: custom_route_test.go:7 Current Label: [Operator][Routes][Serial] Analysis: 🚫 Serial - Modifies cluster Ingresses config, updates component routes ──────────────────────────────────────── Test Name: TestGitLabAsOIDCPasswordGrantCheck Implementation File:Line: gitlab.go:15 Test Wrapper File: gitlab_test.go:7 Current Label: [OIDC][Serial] Analysis: 🚫 Serial - Adds GitLab IDP, modifies cluster OAuth config ──────────────────────────────────────── Test Name: TestKeycloakAsOIDCPasswordGrantCheckAndGroupSync Implementation File:Line: keycloak.go:27 Test Wrapper File: keycloak_test.go:7 Current Label: [OIDC][Serial] Analysis: 🚫 Serial - Adds Keycloak IDP, creates global users/groups, triggers operator rollout ──────────────────────────────────────── Test Name: TestTemplatesConfig Implementation File:Line: templates.go:28 Test Wrapper File: templates_test.go:7 Current Label: [Templates][Serial] Analysis: 🚫 Serial - Modifies OAuth templates, triggers operator rollout, waits for progressing ──────────────────────────────────────── Test Name: TestTokenInactivityTimeout Implementation File:Line: tokentimeout.go:36 Test Wrapper File: tokentimeout_test.go:7 Current Label: [Tokens][Serial] Analysis: 🚫 Serial - Modifies OAuth token config, creates global resources, triggers operator rollout

Can you PTAL and do let me know about your opinion.

Please check how tests are executed by the CI job. I think the CI job uses.

Then compare this with how tests are executed for kas-o.

I think the difference is the use of the -p=1 flag.

Hi @p0lyn0mial

Sure, Thanks. TBH i have taken reference from working service-ca-operator repo

OTA binary registration: https://github.com/openshift/origin/blob/main/pkg/test/extensions/binary.go#L219 Makefile: https://github.com/openshift/service-ca-operator/blob/main/Makefile#L53 Add serial job: https://github.com/openshift/release/pull/72396/changes suite name: https://github.com/openshift/service-ca-operator/blob/main/cmd/service-ca-operator-tests-ext/main.go#L76

✅ In your test binary (cluster-authentication-operator-tests-ext/main.go):
Parallelism: 1 // Already there!
https://github.com/openshift/cluster-authentication-operator/blob/master/cmd/cluster-authentication-operator-tests-ext/main.go#L66

✅ In Makefile (keep current):
test-e2e: GO_TEST_FLAGS += -count 1 # Prevents caching and use count 1
https://github.com/openshift/cluster-authentication-operator/blob/master/Makefile#L49

✅ In CI job config (PR #73999):
env:
TEST_SUITE: openshift/cluster-authentication-operator/operator/serial

When you run tests via the OTE binary:
./cluster-authentication-operator-tests-ext run-suite openshift/cluster-authentication-operator/operator/serial

The binary's Parallelism: 1 setting controls execution, NOT go test flags.
The Makefile's -p 1 is only relevant for:

Running go test ./test/e2e/... directly (without OTE binary)

That's why encryption tests use -p 1 (they run via go test directly)

ok, so, it looks like we will update the auth operator to execute the tests in parallel (xref: openshift/cluster-authentication-operator#833)

yeah agree, as per your suggestions here openshift/cluster-authentication-operator#833 (comment) i have updated to parallel. Will monitor the CI job once it gets merged.

-Original file line number
+Diff line change
@@ Expand Up / @@ -267,6 +267,10 @@ var extensionBinaries = []TestBinary{ @@
     		imageTag:   "cli",
     		binaryPath: "/usr/bin/oc-tests-ext.gz",
     	},
+    	{
+    		imageTag:   "cluster-authentication-operator",
+    		binaryPath: "/usr/bin/cluster-authentication-operator-tests-ext.gz",
+    	},
     }
     // Info returns information about this particular extension.
@@ Expand Down @@

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

CNTRLPLANE-2196: reg ote bin for cluster auth operator #30735

Uh oh!

Diff view

Diff view

There are no files selected for viewing

p0lyn0mial Jan 29, 2026

Uh oh!

p0lyn0mial Jan 29, 2026

Uh oh!

ropatil010 Jan 30, 2026

Uh oh!

ropatil010 Feb 2, 2026

Uh oh!

p0lyn0mial Feb 2, 2026

Uh oh!

ropatil010 Feb 2, 2026

Uh oh!

p0lyn0mial Feb 3, 2026

Uh oh!

ropatil010 Feb 3, 2026

Uh oh!

CNTRLPLANE-2196: reg ote bin for cluster auth operator #30735

Uh oh!

CNTRLPLANE-2196: reg ote bin for cluster auth operator #30735

Uh oh!

Uh oh!

Diff view

Diff view

There are no files selected for viewing

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!