Skip to content

Flaky test TestCLIAuthenticate #1329

@adombeck

Description

@adombeck

Analysis of the last 47 runs of the go-tests-coverage job on main:

════════════════════════════════════════════════════════
Summary
════════════════════════════════════════════════════════
Total runs analyzed : 47
Runs with failures  : 3
Runs clean          : 44
Percentage failed   : 6.4%

Failure counts by test (across all runs):
──────────────────────────────────────────
    3  TestCLIAuthenticate
    1  TestCLIAuthenticate/Authenticate_user_with_qr_code_after_many_regenerations
    1  TestCLIAuthenticate/Deny_authentication_if_max_attempts_reached
    1  TestCLIAuthenticate/Authenticate_user_and_reset_password_while_enforcing_policy
❌ run-21246906163  2026-01-22T11:34:32Z  conclusion: failure
     ❌ TestCLIAuthenticate
     ❌ TestCLIAuthenticate/Authenticate_user_with_qr_code_after_many_regenerations
	Error Trace:	/home/runner/work/authd/authd/pam/integration-tests/vhs-helpers_test.go:348
	            	/home/runner/work/authd/authd/pam/integration-tests/cli_test.go:291
	Error:      	Received unexpected error:
	            	exit status 1
	Messages:   	Failed to run tape "qr_code_quick_regenerate", see vhs output above


❌ run-22480044063  2026-02-27T09:11:49Z  conclusion: failure
     ❌ TestCLIAuthenticate
     ❌ TestCLIAuthenticate/Deny_authentication_if_max_attempts_reached
	Error Trace:	/home/runner/work/authd/authd/pam/integration-tests/vhs-helpers_test.go:429
	            	/home/runner/work/authd/authd/pam/integration-tests/vhs-helpers_test.go:418
	            	/opt/hostedtoolcache/go/1.25.7/x64/src/sync/once.go:78
	            	/opt/hostedtoolcache/go/1.25.7/x64/src/sync/once.go:69
	            	/home/runner/work/authd/authd/pam/integration-tests/vhs-helpers_test.go:417
	            	/home/runner/work/authd/authd/pam/integration-tests/vhs-helpers_test.go:329
	            	/home/runner/work/authd/authd/pam/integration-tests/cli_test.go:291
	Error:      	Received unexpected error:
	            	open /tmp/TestCLIAuthenticateDeny_authentication_if_max_attempts_reached1064790151/001/max_attempts.txt: no such file or directory
	Messages:   	Could not read output file of tape "max_attempts" (/tmp/TestCLIAuthenticateDeny_authentication_if_max_attempts_reached1064790151/001/max_attempts.txt)


❌ run-22951283101  2026-03-11T11:54:06Z  conclusion: failure
     ❌ TestCLIAuthenticate
     ❌ TestCLIAuthenticate/Authenticate_user_and_reset_password_while_enforcing_policy
	Error Trace:	/home/runner/work/authd/authd/pam/integration-tests/vhs-helpers_test.go:429
	            	/home/runner/work/authd/authd/pam/integration-tests/vhs-helpers_test.go:418
	            	/opt/hostedtoolcache/go/1.25.8/x64/src/sync/once.go:78
	            	/opt/hostedtoolcache/go/1.25.8/x64/src/sync/once.go:69
	            	/home/runner/work/authd/authd/pam/integration-tests/vhs-helpers_test.go:417
	            	/home/runner/work/authd/authd/pam/integration-tests/vhs-helpers_test.go:329
	            	/home/runner/work/authd/authd/pam/integration-tests/cli_test.go:291
	Error:      	Received unexpected error:
	            	open /tmp/TestCLIAuthenticateAuthenticate_user_and_reset_password_while_e2159065421/001/mandatory_password_reset.txt: no such file or directory
	Messages:   	Could not read output file of tape "mandatory_password_reset" (/tmp/TestCLIAuthenticateAuthenticate_user_and_reset_password_while_e2159065421/001/mandatory_password_reset.txt)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions