Skip to content

feat: create E2E tests for AgentCard#243

Open
r3v5 wants to merge 7 commits intokagenti:mainfrom
r3v5:create-e2e-test-suite-for-agent-card-2
Open

feat: create E2E tests for AgentCard#243
r3v5 wants to merge 7 commits intokagenti:mainfrom
r3v5:create-e2e-test-suite-for-agent-card-2

Conversation

@r3v5
Copy link
Copy Markdown
Contributor

@r3v5 r3v5 commented Mar 25, 2026

Summary

This PR is responsible for:

  • Add 6 AgentCard E2E test scenarios covering webhook validation, auto-discovery, duplicate prevention,
    audit mode, and SPIRE signature verification
  • Add test fixtures, helper functions (SPIRE lifecycle, kubectl helpers, controller patching), and CI
    workflow updates
  • Add podman support with auto-detection (no env vars needed)
  • Add E2E test documentation with architecture diagrams and scenario details

Test scenarios

Scenario Context What it tests
Reject missing targetRef Without signature Webhook rejects AgentCard with no spec.targetRef
No protocol label Without signature No auto-created card for workload missing protocol.kagenti.io/*
Auto-discovery Without signature Labeled workload gets auto-created AgentCard with correct targetRef, protocol, Synced=True
Duplicate prevention Without signature Webhook rejects second AgentCard targeting same workload
Audit mode With signature Unsigned card syncs but reports SignatureVerified=False/SignatureInvalidAudit
Signed agent With signature SPIRE-signed card: SignatureVerified=True, correct SPIFFE ID, Synced=True, Bound=True

Related issue(s)

RHAIENG-3717

(Optional) Testing Instructions

  1. kind delete cluster && kind create cluster && make test-e2e
  2. CI

r3v5 added 4 commits March 25, 2026 15:14
Signed-off-by: Ian Miller <milleryan2003@gmail.com>
Signed-off-by: Ian Miller <milleryan2003@gmail.com>
Signed-off-by: Ian Miller <milleryan2003@gmail.com>
Signed-off-by: Ian Miller <milleryan2003@gmail.com>
@r3v5 r3v5 requested review from a team as code owners March 25, 2026 16:34
Signed-off-by: Ian Miller <milleryan2003@gmail.com>
Copy link
Copy Markdown
Contributor

@rubambiza rubambiza left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent work — this is a thorough E2E suite with great documentation (the architecture section in the README is particularly good). CI is green including the actual E2E run at ~11 min. A few items to address:

Should fix:

  1. Pin azure/setup-helm@v4 to a SHA. Using a major version tag means any upstream change takes effect immediately without review. Other kagenti repos pin action SHAs for this reason. Please replace with a pinned SHA, e.g.:
    uses: azure/setup-helm@b9e51907a09c216f16ebe8536097933489208112 # v4.3.0
    You can find the current SHA with gh api repos/azure/setup-helm/git/ref/tags/v4.3.0 --jq '.object.sha' (or whichever version you want to pin).

Minor / non-blocking:

  1. Copyright year in fixtures.go says Copyright 2025 — should be 2026.

  2. Hardcoded python:3.11-slim in fixtures gets pulled from Docker Hub on every run. Docker Hub rate limits could cause flaky failures in CI. Not blocking, but worth noting for future resilience (e.g., a pre-pulled image or a registry mirror).

  3. Consistently window for "no protocol label" is 15s with 5s polling — only 3 checks. If the controller is slow to reconcile, this could produce false passes. Consider widening to 30s or reducing the polling interval.

  4. Controller deployed twice — both Manager and AgentCard E2E Describe blocks call DeployController() / UndeployController(). Works correctly but adds ~2-3 min to the suite. Could share the deployment if runtime becomes a concern.

Copy link
Copy Markdown
Contributor

@kevincogan kevincogan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Solid E2E suite. The 6 AgentCard scenarios are well-designed, semantically correct against the actual controller/webhook behavior, and the documentation is excellent. A few items need addressing before merge.

Must fix:

  1. DeployController swallows all namespace creation errors. See inline comment on utils.go.
  2. append aliasing bug in PatchControllerArgs. See inline comment on utils.go.

Should fix:

  1. KubectlApplyStdin duplicates utils.Run internals. See inline comment on utils.go.

Missing test:

  1. Controller arg restoration verification. After RestoreControllerArgs runs in each AfterAll, there is no assertion that the deployment is actually running with the original args. A read-back check would catch the append aliasing bug if the fix is not applied correctly. Worth adding in this PR since it is only a few lines.

Items 3-4 are straightforward additions that would be good to land in the same PR.

r3v5 added 2 commits March 26, 2026 14:33
…gent cards

Signed-off-by: Ian Miller <milleryan2003@gmail.com>
Signed-off-by: Ian Miller <milleryan2003@gmail.com>
@r3v5 r3v5 requested review from kevincogan and rubambiza March 26, 2026 15:57
Copy link
Copy Markdown
Contributor

@kevincogan kevincogan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, all review findings addressed. Clean implementation. Thanks! :))

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants