add e2e test for pubsub. #454
Conversation
WalkthroughAdds Google Cloud Pub/Sub emulator support across CI, local tooling, OpenShift templates, init scripts, and test/setup flows to run Maestro with MESSAGE_DRIVER_TYPE=pubsub, including emulator lifecycle Makefile targets, init Jobs to create topics/subscriptions, and a CI e2e job. Changes
Sequence Diagram(s)sequenceDiagram
autonumber
participant Dev as Developer / CI
participant K8s as Kubernetes / OpenShift
participant PubSub as Pub/Sub Emulator
participant Init as Init Job Pod
participant Agent as Maestro Agent
rect rgba(200,230,255,0.25)
Dev->>K8s: Apply `pubsub-template` (Service + Deployment + Secret)
K8s->>PubSub: Start emulator container (host:port)
PubSub-->>K8s: Emulator ready
end
rect rgba(220,255,220,0.18)
Dev->>K8s: Apply `pubsub-init-job-template` (server init Job)
K8s->>Init: Schedule init pod
Init->>PubSub: Create topics & server subscriptions
PubSub-->>Init: ACK / AlreadyExists
Init-->>K8s: Job completes
end
rect rgba(255,245,200,0.18)
Dev->>K8s: Apply `pubsub-agent-init-job-template` (consumer init with CONSUMER_NAME)
K8s->>Init: Run agent-init pod
Init->>PubSub: Create consumer subscriptions
Init-->>K8s: Job completes
end
rect rgba(240,230,255,0.18)
Dev->>K8s: Deploy agent with `PUBSUB_*` params + `maestro-agent-pubsub` Secret
K8s->>Agent: Start agent pod
Agent->>PubSub: Subscribe / pull messages
PubSub-->>Agent: Deliver messages
end
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✨ Finishing touches
Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 5
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
go.mod (1)
209-210: Establish a migration plan for the fork dependency.Using a personal fork (morvencao/ocm-sdk-go) as a replace directive creates a temporal dependency that requires clear documentation and a removal plan. While the fork is currently accessible and maintained, this pattern is not sustainable long-term—it ties the build to an external personal account and PR status.
PR #182 ("support test mode for pubsub driver") is currently open (updated Dec 25). Ensure:
- Once upstream PR #182 is merged, immediately remove this replace directive and update the require statement on line 60 to reference the merged version.
- Document this temporary workaround in the code or PR description with the expected timeline for removal (tied to PR #182's merge).
- Add a tracking issue or comment linking this dependency to PR #182 so the team knows when the replacement can be removed.
🧹 Nitpick comments (2)
test/e2e/pkg/cert_rotation_test.go (1)
80-83: Clarify the assertion failure message.The message "no CA secrets found; certificate rotation did not run" is misleading at this point in the code. If execution reaches line 83, it means
BeforeAlldid not skip, so at least one CA secret exists. A falserotatedvalue would indicate rotation failed for a different reason (e.g., invalid secret data).🔎 Suggested improvement
-Expect(rotated).To(BeTrue(), "no CA secrets found; certificate rotation did not run") +Expect(rotated).To(BeTrue(), "expected at least one certificate to be rotated")test/setup/env_setup.sh (1)
193-222: LGTM with a minor observation about the sleep.The Pub/Sub emulator setup follows the established pattern for other message brokers. The
sleep 5on line 205 is a pragmatic approach for waiting for emulator readiness, though in slower CI environments this might occasionally be insufficient.If flakiness is observed in CI, consider replacing the fixed sleep with a retry loop that checks the emulator's health, for example by attempting a simple Pub/Sub API call until it succeeds.
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Cache: Disabled due to data retention organization setting
Knowledge base: Disabled due to Reviews -> Disable Knowledge Base setting
⛔ Files ignored due to path filters (1)
go.sumis excluded by!**/*.sum
📒 Files selected for processing (17)
.github/workflows/e2e.yml.gitignoreMakefileREADME.mdgo.modhack/init-pubsub-emulator.pytemplates/README.mdtemplates/agent-template.ymltemplates/agent-tls-template.ymltemplates/pubsub-agent-init-job-template.ymltemplates/pubsub-init-job-template.ymltemplates/pubsub-template.ymltemplates/service-template.ymltemplates/service-tls-template.ymltest/e2e/pkg/cert_rotation_test.gotest/setup/deploy_agent.shtest/setup/env_setup.sh
🧰 Additional context used
🪛 Ruff (0.14.10)
hack/init-pubsub-emulator.py
35-35: Do not catch blind exception: Exception
(BLE001)
62-62: Do not catch blind exception: Exception
(BLE001)
113-113: Do not catch blind exception: Exception
(BLE001)
145-145: Do not catch blind exception: Exception
(BLE001)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (8)
- GitHub Check: Red Hat Konflux / maestro-e2e-on-pull-request
- GitHub Check: Red Hat Konflux / maestro-on-pull-request
- GitHub Check: e2e-with-istio
- GitHub Check: e2e-pubsub
- GitHub Check: e2e-broadcast-subscription
- GitHub Check: e2e-grpc-broker
- GitHub Check: upgrade
- GitHub Check: e2e
🔇 Additional comments (23)
test/e2e/pkg/cert_rotation_test.go (2)
32-32: LGTM: Skip flag declaration.The skip flag is appropriately used to track whether certificate rotation tests should be skipped when CA secrets are absent.
34-52: LGTM: Conditional test execution based on CA secret presence.The logic correctly handles three scenarios:
- Both CA secrets missing → skip (Pub/Sub emulator scenario)
- At least one CA secret present → continue (MQTT/gRPC scenario)
- Unexpected errors → fail
This aligns well with the PR objective of supporting Pub/Sub as an alternative message driver.
.gitignore (1)
50-50: LGTM!The new ignore rule for
secrets/pubsub.configfollows the existing pattern for other secret configuration files in this directory.README.md (3)
45-79: LGTM!The documentation clearly explains both MQTT and Pub/Sub setup paths, maintains consistency with existing style, and properly notes the Python package dependency for Pub/Sub initialization.
114-122: LGTM!The Pub/Sub runtime instructions are clear and follow the same pattern as the MQTT instructions.
361-366: LGTM!The KinD cluster instructions for Pub/Sub are concise and align with the MESSAGE_DRIVER_TYPE environment variable used throughout the codebase.
Makefile (2)
65-70: LGTM!The Pub/Sub configuration variables follow the same pattern as the MQTT configuration variables, with sensible defaults for local development.
446-461: Pub/Sub lifecycle targets look good overall.The targets follow a similar lifecycle pattern to the MQTT targets. One minor observation: the
pubsub/inittarget relies onpython3and thegoogle-cloud-pubsubpackage being available in the local environment, which is documented in the README. The emulator image (gcr.io/google.com/cloudsdktool/google-cloud-cli:emulators) is accessible and available.hack/init-pubsub-emulator.py (4)
1-11: LGTM!The script is well-documented with clear purpose and environment variable usage.
19-66: LGTM!The server topics and subscriptions initialization logic is well-structured with proper error handling. The broad
Exceptioncatches (flagged by static analysis) are appropriate here since this is an initialization script that needs to handle various potential API errors from the Pub/Sub client library, and the code properly logs the error and returnsFalseto signal failure.
69-117: LGTM!The agent subscription initialization follows the same pattern as the server initialization with consistent error handling.
120-147: LGTM!The main function properly handles environment variables with sensible defaults and exits with appropriate codes on failure. The
emulator_hostvariable is correctly used for logging purposes (the library readsPUBSUB_EMULATOR_HOSTenvironment variable directly).templates/service-template.yml (1)
219-222: LGTM!The dynamic secret name
maestro-${MESSAGE_DRIVER_TYPE}enables the template to work with different message drivers. Making the secretoptional: trueis appropriate since not all message driver configurations require a secret (e.g., gRPC may use different authentication mechanisms).templates/service-tls-template.yml (1)
275-278: LGTM!The change is consistent with the non-TLS service template, enabling dynamic message driver configuration.
templates/README.md (1)
33-56: Documentation references are accurate; both GCP templates exist.The referenced GCP templates (
service-template-gcp.ymlandagent-template-gcp.yml) are present in the repository. The Pub/Sub emulator documentation is comprehensive and correct..github/workflows/e2e.yml (1)
119-145: LGTM! Pub/Sub e2e job follows existing patterns.The new
e2e-pubsubjob is well-structured and consistent with the existing e2e jobs. The environment variables (MESSAGE_DRIVER_TYPE=pubsub, SERVER_REPLICAS=2, ENABLE_MAESTRO_TLS=true) are appropriate for testing Pub/Sub message driver integration.templates/agent-template.yml (2)
72-85: LGTM! Pub/Sub parameters properly defined.The new parameters (PUBSUB_HOST, PUBSUB_PORT, PUBSUB_PROJECT_ID) follow the existing pattern for message driver configuration and have sensible defaults for the emulator environment.
367-381: Clarify that this template is for development/testing with the Pub/Sub emulator, not production.The
insecure: trueflag is appropriate for this template since it's designed to work with the Pub/Sub emulator (which runs on localhost:8085 without TLS). However, the template documentation should make clear thatagent-template.ymlis for local development and e2e testing. For production GCP deployments, useagent-template-gcp.ymlinstead, which does not include theinsecureflag and is intended for actual GCP Pub/Sub endpoints.test/setup/deploy_agent.sh (2)
49-51: LGTM! Pub/Sub configuration variables properly initialized.The variables are appropriately scoped and use consistent naming with the template parameters.
55-78: [rewritten review comment]
[classification tag]templates/agent-tls-template.yml (2)
72-85: LGTM! Pub/Sub parameters consistent with agent-template.yml.The parameters are correctly defined and match the non-TLS variant, maintaining consistency across agent templates.
383-397: The template is explicitly designed for Pub/Sub emulator testing and development, not production deployments. Per the templates README, production GCP deployments use the separateagent-template-gcp.ymltemplate. Theinsecure: trueflag is appropriate for this template's intended scope. No changes needed.Likely an incorrect or invalid review comment.
templates/pubsub-template.yml (1)
102-116: The Pub/Sub topic and subscription naming is already consistent across all templates and initialization scripts. No inconsistencies were found.
morvencao
force-pushed
the
br_add_pubsub_e2e
branch
from
6a0704e to
8bbc798
Compare
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (1)
hack/init-pubsub-emulator.py (1)
35-37: Consider more specific exception handling.The static analysis tool flags catching bare
Exceptionas a code smell. For production code, catching specific exception types (e.g.,exceptions.GoogleAPICallError) would be preferable. However, for an initialization script that logs errors and exits, the current approach is pragmatic and acceptable.If you want to be more specific, consider:
🔎 More specific exception handling
except exceptions.AlreadyExists: print(f" - Topic already exists: {topic_name}") - except Exception as e: + except exceptions.GoogleAPICallError as e: print(f" ✗ Error creating topic {topic_name}: {e}", file=sys.stderr) return FalseThis pattern would apply to lines 62, 113, and 145 as well. However, the current implementation is acceptable for an init script.
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Cache: Disabled due to data retention organization setting
Knowledge base: Disabled due to Reviews -> Disable Knowledge Base setting
⛔ Files ignored due to path filters (1)
go.sumis excluded by!**/*.sum
📒 Files selected for processing (17)
.github/workflows/e2e.yml.gitignoreMakefileREADME.mdgo.modhack/init-pubsub-emulator.pytemplates/README.mdtemplates/agent-template.ymltemplates/agent-tls-template.ymltemplates/pubsub-agent-init-job-template.ymltemplates/pubsub-init-job-template.ymltemplates/pubsub-template.ymltemplates/service-template.ymltemplates/service-tls-template.ymltest/e2e/pkg/cert_rotation_test.gotest/setup/deploy_agent.shtest/setup/env_setup.sh
🚧 Files skipped from review as they are similar to previous changes (9)
- Makefile
- README.md
- test/setup/deploy_agent.sh
- go.mod
- templates/agent-tls-template.yml
- templates/README.md
- templates/agent-template.yml
- templates/pubsub-agent-init-job-template.yml
- templates/service-tls-template.yml
🧰 Additional context used
🪛 Ruff (0.14.10)
hack/init-pubsub-emulator.py
35-35: Do not catch blind exception: Exception
(BLE001)
62-62: Do not catch blind exception: Exception
(BLE001)
113-113: Do not catch blind exception: Exception
(BLE001)
145-145: Do not catch blind exception: Exception
(BLE001)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (8)
- GitHub Check: Red Hat Konflux / maestro-on-pull-request
- GitHub Check: Red Hat Konflux / maestro-e2e-on-pull-request
- GitHub Check: e2e
- GitHub Check: upgrade
- GitHub Check: e2e-grpc-broker
- GitHub Check: e2e-broadcast-subscription
- GitHub Check: e2e-with-istio
- GitHub Check: e2e-pubsub
🔇 Additional comments (9)
.gitignore (1)
50-50: ✓ Looks good!The new ignore rule for
secrets/pubsub.configfollows the existing pattern for secret files and correctly prevents Pub/Sub configuration from being accidentally committed to the repository.test/e2e/pkg/cert_rotation_test.go (2)
123-125: LGTM! Correctly addresses the past review comment.The implementation now uses an early return in
AfterAllwhen the tests are skipped, which is the idiomatic pattern for conditional cleanup in Ginkgo lifecycle hooks. TheSkip()call inBeforeAll(line 43) is appropriate for skipping test specs.
34-52: Well-designed skip mechanism for Pub/Sub compatibility.The conditional skip logic properly handles the case where certificate rotation tests are not applicable (e.g., when using Pub/Sub emulator). The implementation:
- Checks for both MQTT and gRPC CA secrets
- Skips only when both are absent
- Maintains proper error handling for unexpected failures
hack/init-pubsub-emulator.py (1)
19-147: Well-structured initialization script.The script is clearly organized with:
- Separate functions for server and agent initialization
- Proper idempotency via
AlreadyExistsexception handling- Clear success/failure reporting with exit codes
- Environment variable configuration with sensible defaults
test/setup/env_setup.sh (1)
193-222: LGTM! Pub/Sub setup follows established patterns.The Pub/Sub emulator setup is well-integrated and follows the same pattern as the existing MQTT and gRPC broker setup:
- Deploy the emulator
- Wait for availability
- Initialize topics/subscriptions via a templated job
- Wait for job completion
- Clean up the job
The implementation is consistent with the rest of the script.
.github/workflows/e2e.yml (1)
119-145: LGTM! E2E workflow job for Pub/Sub is well-configured.The new
e2e-pubsubjob is structured consistently with the existinge2e-grpc-brokerjob and properly configures:
MESSAGE_DRIVER_TYPE: pubsubto enable Pub/Sub testingSERVER_REPLICAS: 2for multi-instance testingENABLE_MAESTRO_TLS: truefor secure communicationcontainer_tool: dockerfor the CI environmenttemplates/pubsub-template.yml (1)
1-116: LGTM! Well-designed Pub/Sub emulator template.The template properly defines the Pub/Sub emulator infrastructure:
- Service exposing port 8085
- Deployment running the emulator with appropriate configuration
- Secret containing server-side configuration (topics to publish, subscriptions to consume)
The use of
emptyDirfor storage (lines 98-99) is appropriate for an emulator in testing scenarios. Based on past review discussions, the:emulatorsimage tag (line 22) is intentionally chosen for stability, and the topic configuration aligns with the broader Pub/Sub integration across the repository.templates/service-template.yml (1)
219-222: The dynamic secret reference and optional flag are correctly implemented.The
maestro-${MESSAGE_DRIVER_TYPE}secret configuration withoptional: trueis appropriate. Themaestro-pubsubsecret is properly created by pubsub-template.yml (lines 103-120), and similarly,maestro-mqttis created by mqtt-template.yml. The optional flag prevents deployment failure when a particular message driver is not deployed.Note: The MESSAGE_DRIVER_TYPE parameter description in service-template.yml should be updated from "mqtt or grpc" to include "pubsub" now that Pub/Sub is a supported driver type.
templates/pubsub-init-job-template.yml (1)
52-110: google-cloud-pubsub 2.34.0 is stable and suitable for this use case.Version 2.34.0 was released December 16, 2025, and is tested for Python 3.7–3.14. It is an actively maintained, stable release. The operations used in this script (creating topics and subscriptions with filters) are fully supported by the Pub/Sub emulator. While the emulator has known limitations with UpdateTopic/UpdateSnapshot RPC and some advanced features, they do not affect the basic topic and subscription creation logic here.
The subscription filter
attributes.ce-originalsource="maestro"and the overall topic/subscription initialization approach are correct.
morvencao
force-pushed
the
br_add_pubsub_e2e
branch
from
8bbc798 to
5b0276f
Compare
|
/assign @clyang82 |
There was a problem hiding this comment.
Actionable comments posted: 2
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Cache: Disabled due to data retention organization setting
Knowledge base: Disabled due to Reviews -> Disable Knowledge Base setting
⛔ Files ignored due to path filters (1)
go.sumis excluded by!**/*.sum
📒 Files selected for processing (17)
.github/workflows/e2e.yml.gitignoreMakefileREADME.mdgo.modhack/init-pubsub-emulator.pytemplates/README.mdtemplates/agent-template.ymltemplates/agent-tls-template.ymltemplates/pubsub-agent-init-job-template.ymltemplates/pubsub-init-job-template.ymltemplates/pubsub-template.ymltemplates/service-template.ymltemplates/service-tls-template.ymltest/e2e/pkg/cert_rotation_test.gotest/setup/deploy_agent.shtest/setup/env_setup.sh
🚧 Files skipped from review as they are similar to previous changes (8)
- templates/service-tls-template.yml
- templates/README.md
- templates/pubsub-template.yml
- .gitignore
- templates/pubsub-init-job-template.yml
- Makefile
- README.md
- templates/agent-tls-template.yml
🧰 Additional context used
🪛 Ruff (0.14.10)
hack/init-pubsub-emulator.py
35-35: Do not catch blind exception: Exception
(BLE001)
62-62: Do not catch blind exception: Exception
(BLE001)
113-113: Do not catch blind exception: Exception
(BLE001)
145-145: Do not catch blind exception: Exception
(BLE001)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (8)
- GitHub Check: Red Hat Konflux / maestro-e2e-on-pull-request
- GitHub Check: Red Hat Konflux / maestro-on-pull-request
- GitHub Check: e2e-pubsub
- GitHub Check: e2e-broadcast-subscription
- GitHub Check: e2e-grpc-broker
- GitHub Check: e2e-with-istio
- GitHub Check: upgrade
- GitHub Check: e2e
🔇 Additional comments (8)
test/setup/env_setup.sh (1)
193-222: LGTM! Pub/Sub initialization flow is well-structured.The Pub/Sub setup follows a clear pattern: deploy emulator → wait for readiness → initialize topics/subscriptions → cleanup. The hardcoded 5-second sleep at Line 205 is acceptable for test setup, though a readiness probe would be more robust in production.
test/e2e/pkg/cert_rotation_test.go (2)
32-51: LGTM! Skip logic correctly handles Pub/Sub scenario.The certificate rotation tests are appropriately skipped when neither MQTT nor gRPC CA secrets are present, which is the expected state for Pub/Sub emulator testing. The error handling distinguishes between NotFound (expected for Pub/Sub) and actual errors.
123-125: Correctly uses early return in AfterAll.The skip guard properly uses an early return instead of
Skip(), which aligns with Ginkgo best practices for cleanup hooks. This addresses the concern from the previous review..github/workflows/e2e.yml (1)
119-145: LGTM! CI job for Pub/Sub is properly configured.The new e2e-pubsub job follows the established pattern of other e2e jobs and appropriately sets MESSAGE_DRIVER_TYPE to pubsub. Testing with 2 replicas and TLS enabled provides good coverage.
templates/agent-template.yml (2)
72-85: LGTM! Pub/Sub parameters are well-defined.The new Pub/Sub configuration parameters have appropriate defaults for emulator testing and are properly marked as required.
367-381: LGTM! Agent Pub/Sub configuration is correct.The Secret properly configures Pub/Sub topics and subscriptions with consumer-specific subscription names. The
insecure: trueflag is appropriate for emulator usage in testing.test/setup/deploy_agent.sh (1)
49-78: LGTM! Agent Pub/Sub initialization is well-implemented.The agent subscription initialization flow properly creates consumer-specific subscriptions before deploying the agent. The job lifecycle management (create → wait → cleanup) follows best practices.
templates/pubsub-agent-init-job-template.yml (1)
58-120: LGTM! Agent subscription initialization logic is robust.The inline Python script properly creates consumer-specific subscriptions with correct filter syntax (ce-clustername). The use of google-cloud-pubsub 2.34.0 (as addressed in the previous review) ensures up-to-date dependencies. Error handling gracefully manages AlreadyExists while propagating unexpected errors.
morvencao
force-pushed
the
br_add_pubsub_e2e
branch
from
5b0276f to
7e3be17
Compare
There was a problem hiding this comment.
Actionable comments posted: 2
Fix all issues with AI Agents 🤖
In @test/setup/env_setup.sh:
- Around line 208-215: The test setup uses the OpenShift CLI via the oc process
invocation (seen around the pubsub init block and in deploy_agent.sh for agent
subscriptions) but env_setup.sh never installs or verifies oc like it does for
kind, step, and istioctl; update env_setup.sh to either install the OpenShift
CLI (oc) into the CI image or add a preflight check that verifies oc is on PATH
and exits with a clear error if missing, and mirror the same check/installation
where deploy_agent.sh relies on oc so Pub/Sub initialization won't fail when
enabled.
🧹 Nitpick comments (4)
hack/init-pubsub-emulator.py (1)
35-37: Consider catching more specific exceptions.The broad
Exceptioncatch works for this CLI script, but catchinggoogle.api_core.exceptions.GoogleAPIErrorwould be more precise and avoid masking unexpected programming errors.🔎 Proposed fix
- except Exception as e: + except exceptions.GoogleAPIError as e: print(f" ✗ Error creating topic {topic_name}: {e}", file=sys.stderr) return FalseApply the same pattern at lines 62-64 and 113-115.
test/setup/env_setup.sh (1)
203-206: Consider replacingsleep 5with a readiness probe or retry loop.The fixed sleep is fragile and may cause flaky tests in slower environments. Consider using a retry loop to verify the emulator is responsive before proceeding.
🔎 Proposed improvement
# Initialize topics and subscriptions in the emulator - # Wait a bit for the emulator to be fully ready - sleep 5 + # Wait for the emulator to be fully ready + for i in {1..10}; do + if curl -s "http://${pubsub_host}:${pubsub_port}" >/dev/null 2>&1; then + break + fi + sleep 1 + donetemplates/pubsub-init-job-template.yml (2)
36-48: Consider adding Job failure constraints for robustness.The Job spec doesn't define
backoffLimit(defaults to 6) oractiveDeadlineSeconds. Adding explicit constraints improves observability and prevents the Job from hanging indefinitely if the Pub/Sub emulator is unreachable or slow to respond.🔎 Suggested Job constraints
spec: + backoffLimit: 3 + activeDeadlineSeconds: 300 template: spec: restartPolicy: NeverThis limits retries to 3 attempts and enforces a 5-minute timeout for the entire initialization process.
41-48: Add resource limits for production deployments.The container doesn't specify resource requests or limits. For production environments, defining these prevents resource contention and ensures predictable scheduling.
🔎 Suggested resource constraints
containers: - name: init image: registry.access.redhat.com/ubi9/python-311 imagePullPolicy: IfNotPresent + resources: + requests: + memory: "128Mi" + cpu: "100m" + limits: + memory: "256Mi" + cpu: "500m" env: - name: PUBSUB_EMULATOR_HOST value: "${PUBSUB_HOST}:${PUBSUB_PORT}"These values are conservative for a short-lived init Job that performs pip installation and lightweight API calls.
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Cache: Disabled due to data retention organization setting
Knowledge base: Disabled due to Reviews -> Disable Knowledge Base setting
⛔ Files ignored due to path filters (1)
go.sumis excluded by!**/*.sum
📒 Files selected for processing (17)
.github/workflows/e2e.yml.gitignoreMakefileREADME.mdgo.modhack/init-pubsub-emulator.pytemplates/README.mdtemplates/agent-template.ymltemplates/agent-tls-template.ymltemplates/pubsub-agent-init-job-template.ymltemplates/pubsub-init-job-template.ymltemplates/pubsub-template.ymltemplates/service-template.ymltemplates/service-tls-template.ymltest/e2e/pkg/cert_rotation_test.gotest/setup/deploy_agent.shtest/setup/env_setup.sh
🚧 Files skipped from review as they are similar to previous changes (9)
- templates/service-template.yml
- test/e2e/pkg/cert_rotation_test.go
- .github/workflows/e2e.yml
- README.md
- Makefile
- templates/pubsub-agent-init-job-template.yml
- go.mod
- .gitignore
- templates/pubsub-template.yml
🧰 Additional context used
🪛 Ruff (0.14.10)
hack/init-pubsub-emulator.py
35-35: Do not catch blind exception: Exception
(BLE001)
62-62: Do not catch blind exception: Exception
(BLE001)
113-113: Do not catch blind exception: Exception
(BLE001)
145-145: Do not catch blind exception: Exception
(BLE001)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (8)
- GitHub Check: e2e-broadcast-subscription
- GitHub Check: e2e-pubsub
- GitHub Check: upgrade
- GitHub Check: e2e
- GitHub Check: e2e-with-istio
- GitHub Check: e2e-grpc-broker
- GitHub Check: Red Hat Konflux / maestro-on-pull-request
- GitHub Check: Red Hat Konflux / maestro-e2e-on-pull-request
🔇 Additional comments (11)
templates/README.md (1)
33-55: LGTM!The documentation clearly describes the Pub/Sub emulator templates, the topics and subscriptions created by the init jobs, and references the GCP-specific templates for production deployments.
hack/init-pubsub-emulator.py (3)
1-17: LGTM!Clear module docstring with environment variable documentation and appropriate imports.
69-117: LGTM!The agent subscription initialization logic is correct, properly creating filtered and unfiltered subscriptions for the specified consumer.
120-147: LGTM!The main function correctly orchestrates initialization with sensible defaults. The
PUBSUB_EMULATOR_HOSTenvironment variable is used directly by the Google Cloud Pub/Sub client library, so reading it for logging purposes is appropriate.templates/service-tls-template.yml (1)
275-278: LGTM!The dynamic secret name
maestro-${MESSAGE_DRIVER_TYPE}correctly enables support for multiple message driver types while maintaining the optional flag for backward compatibility.templates/agent-template.yml (2)
72-86: LGTM!The Pub/Sub parameters are well-defined with sensible defaults that align with the emulator configuration in other scripts.
367-381: LGTM!The Pub/Sub secret configuration is correctly structured with proper GCP Pub/Sub resource paths and appropriate settings for the emulator (TLS disabled).
test/setup/deploy_agent.sh (2)
49-51: LGTM!The Pub/Sub environment variables are consistent with the values defined in
env_setup.sh.
55-78: LGTM!The Pub/Sub agent initialization block correctly creates consumer-specific subscriptions before deploying the agent. The workflow mirrors the server-side initialization pattern.
templates/agent-tls-template.yml (2)
72-86: LGTM!The Pub/Sub parameters are consistent with
agent-template.yml, ensuring both TLS and non-TLS deployments use the same configuration.
383-397: LGTM!The Pub/Sub secret configuration is consistent with
agent-template.yml, maintaining parity between the TLS and non-TLS agent templates.
morvencao
force-pushed
the
br_add_pubsub_e2e
branch
from
7e3be17 to
42a9caf
Compare
There was a problem hiding this comment.
Actionable comments posted: 3
Fix all issues with AI Agents 🤖
In @test/setup/env_setup.sh:
- Around line 227-231: After waiting on job/pubsub-init, query its status with
kubectl get job pubsub-init in the ${namespace} and check the JSON fields
.status.succeeded and .status.failed (or .status.conditions) to ensure the init
job actually succeeded; if .status.succeeded is not >0 or .status.failed is >0,
log an error and exit non-zero without running the cleanup delete, otherwise
proceed to delete the job; update the script around the kubectl wait/delete
lines to perform this check and gate the kubectl delete on a successful job
status.
- Line 22: The oc_version variable in env_setup.sh is set to an outdated value;
update the oc_version assignment (oc_version="4.14.0") to the current stable
OpenShift CLI version (4.20.0 or 4.20) so scripts use the newer release; locate
the oc_version declaration in the file and change its value accordingly and run
any associated CI/setup scripts to verify compatibility.
🧹 Nitpick comments (2)
test/setup/env_setup.sh (1)
214-215: Consider using a readiness check instead of a fixed sleep.The 5-second sleep may not be sufficient in all environments. Consider polling the emulator's health endpoint or using
kubectl waitwith a readiness condition.🔎 Alternative approach using kubectl wait or health check
- # Initialize topics and subscriptions in the emulator - # Wait a bit for the emulator to be fully ready - sleep 5 + # Wait for emulator to be fully ready by checking its health + echo "Waiting for Pub/Sub emulator to be ready..." + until kubectl -n ${namespace} exec deploy/maestro-pubsub -- curl -sf http://localhost:8085/v1/projects/${pubsub_project_id}/topics >/dev/null 2>&1; do + echo "Emulator not ready yet, waiting..." + sleep 2 + done + echo "Emulator is ready"templates/pubsub-template.yml (1)
91-93: Optional: Consider removing redundant environment variable.The
PUBSUB_PROJECT_IDenvironment variable duplicates the project ID already passed via the--projectflag in the command. Unless it's consumed by the emulator or debugging scripts, it can be removed.🔎 Cleanup diff
ports: - containerPort: 8085 name: pubsub - env: - - name: PUBSUB_PROJECT_ID - value: ${PUBSUB_PROJECT_ID} volumeMounts: - name: pubsub-persistent-storage mountPath: /data
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Cache: Disabled due to data retention organization setting
Knowledge base: Disabled due to Reviews -> Disable Knowledge Base setting
⛔ Files ignored due to path filters (1)
go.sumis excluded by!**/*.sum
📒 Files selected for processing (17)
.github/workflows/e2e.yml.gitignoreMakefileREADME.mdgo.modhack/init-pubsub-emulator.pytemplates/README.mdtemplates/agent-template.ymltemplates/agent-tls-template.ymltemplates/pubsub-agent-init-job-template.ymltemplates/pubsub-init-job-template.ymltemplates/pubsub-template.ymltemplates/service-template.ymltemplates/service-tls-template.ymltest/e2e/pkg/cert_rotation_test.gotest/setup/deploy_agent.shtest/setup/env_setup.sh
🚧 Files skipped from review as they are similar to previous changes (6)
- templates/agent-template.yml
- templates/README.md
- .github/workflows/e2e.yml
- templates/pubsub-init-job-template.yml
- go.mod
- templates/pubsub-agent-init-job-template.yml
🧰 Additional context used
🪛 Ruff (0.14.10)
hack/init-pubsub-emulator.py
35-35: Do not catch blind exception: Exception
(BLE001)
62-62: Do not catch blind exception: Exception
(BLE001)
113-113: Do not catch blind exception: Exception
(BLE001)
145-145: Do not catch blind exception: Exception
(BLE001)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (8)
- GitHub Check: Red Hat Konflux / maestro-on-pull-request
- GitHub Check: Red Hat Konflux / maestro-e2e-on-pull-request
- GitHub Check: e2e-pubsub
- GitHub Check: upgrade
- GitHub Check: e2e-broadcast-subscription
- GitHub Check: e2e-grpc-broker
- GitHub Check: e2e
- GitHub Check: e2e-with-istio
🔇 Additional comments (25)
test/e2e/pkg/cert_rotation_test.go (2)
32-51: LGTM! Proper skip logic for Pub/Sub environments.The implementation correctly skips certificate rotation tests when neither MQTT nor gRPC CA secrets exist (Pub/Sub doesn't use client certificates). The logic properly:
- Checks both CA secrets independently
- Only skips when both are missing
- Preserves error handling for non-NotFound errors
- Uses Skip() in BeforeAll (the correct lifecycle hook for skipping specs)
123-125: LGTM! Correct cleanup skip pattern.The early return when
skipis true properly prevents restoration logic from running when the test was skipped. This is the idiomatic pattern for conditional cleanup in Ginkgo lifecycle hooks, and addresses the past review comment correctly.README.md (3)
45-79: LGTM! Clear Pub/Sub setup documentation.The documentation effectively presents the dual-path setup (MQTT vs Pub/Sub) with clear step labels (3a/3b). The note about the google-cloud-pubsub Python package prerequisite is helpful for users.
114-121: LGTM! Clear Pub/Sub run instructions.The documentation clearly shows how to run Maestro with Pub/Sub, including the required flags and config file reference.
360-365: LGTM! Clear test environment Pub/Sub support.The documentation effectively shows how to enable Pub/Sub in the KinD test environment using the MESSAGE_DRIVER_TYPE environment variable.
.gitignore (1)
50-50: LGTM! Appropriate ignore rule for Pub/Sub configuration.The addition of
secrets/pubsub.configaligns with the existing pattern for ignoring secret files and prevents accidental commits of Pub/Sub emulator configuration.templates/service-template.yml (1)
219-222: LGTM! Correct dynamic secret reference pattern.The change to use
maestro-${MESSAGE_DRIVER_TYPE}enables multi-broker support (MQTT, gRPC, Pub/Sub). Theoptional: trueflag is the correct approach, as clarified in past review discussions: it allows gRPC deployments (where no secret is created) while mqtt/pubsub deployments will fail fast during initialization if the required secret is missing.templates/service-tls-template.yml (1)
275-278: LGTM! Consistent dynamic secret reference pattern.This change mirrors the approach in service-template.yml, using
maestro-${MESSAGE_DRIVER_TYPE}to enable multi-broker support. The pattern is correct and consistent across both templates.test/setup/env_setup.sh (1)
55-62: LGTM! OpenShift CLI installation follows established pattern.The installation block correctly checks for
oc, downloads from the official mirror, extracts, installs to/usr/local/bin, and cleans up properly. This addresses the previous concern aboutocavailability.templates/pubsub-template.yml (2)
80-87: LGTM! Emulator command is correctly configured.The emulator start command properly binds to
0.0.0.0:8085to accept connections from other pods and passes the project ID via the--projectflag.
107-116: LGTM! Secret structure aligns with agent configuration.The config.yaml properly defines the project ID, endpoint, and topic/subscription mappings that match the agent secret structure introduced in
templates/agent-tls-template.yml. ThedisableTLS: truesetting is appropriate for the emulator.test/setup/deploy_agent.sh (1)
49-51: LGTM! Pub/Sub environment variables are consistent.The exported variables match the values used in
test/setup/env_setup.shand align with the template defaults, ensuring consistency across server and agent setup.templates/agent-tls-template.yml (2)
72-85: LGTM! Pub/Sub parameters follow established conventions.The parameter definitions are consistent with the MQTT parameters above and use appropriate default values that match the emulator configuration.
383-397: LGTM! Agent Pub/Sub secret structure is correct.The secret properly configures consumer-specific subscriptions (
sourceevents-${CONSUMER_NAME},sourcebroadcast-${CONSUMER_NAME}) and shared topics (agentevents,agentbroadcast), enabling proper message routing between server and agent.hack/init-pubsub-emulator.py (4)
1-11: LGTM! Script header and documentation are clear.The module docstring properly explains the script's purpose and documents the expected environment variables.
19-66: LGTM! Server initialization logic is robust.The function correctly creates topics and subscriptions with appropriate filters for CloudEvents attributes. The broad exception handling is appropriate here to catch any Pub/Sub API errors and provide useful feedback.
69-117: LGTM! Agent subscription initialization is correct.The function properly creates consumer-specific subscriptions with appropriate filtering. The broadcast subscription correctly has no filter to receive all broadcast messages.
120-147: LGTM! Main function orchestrates initialization correctly.The function properly sequences server initialization before agent initialization, handles environment variables with sensible defaults, and ensures proper exit codes for CI/CD integration.
Makefile (7)
65-69: LGTM!The Pub/Sub configuration variables follow the established patterns for MQTT broker configuration, with appropriate defaults for local development and testing.
88-89: LGTM!The documentation correctly reflects the addition of Pub/Sub as a message driver option.
139-145: LGTM!The help output additions clearly document the new Pub/Sub targets and follow the established format.
331-333: LGTM!The Pub/Sub parameters are correctly passed to the template generation, following the same pattern as the MQTT configuration.
446-450: LGTM!The Pub/Sub emulator setup follows the established patterns for database and MQTT broker setup. The port mapping and container configuration are correct.
452-455: LGTM!The teardown target correctly stops and removes the Pub/Sub emulator container, consistent with the cleanup patterns for other services.
457-460: Python prerequisites are already documented in the README.The
google-cloud-pubsubrequirement is documented at README line 77: "Requires google-cloud-pubsub Python package (pip3 install google-cloud-pubsub)". The README also shows the correct workflow sequence (make pubsub/setupfollowed bymake pubsub/init). No action needed; the current documentation is sufficient.
morvencao
force-pushed
the
br_add_pubsub_e2e
branch
from
42a9caf to
a7fbefa
Compare
Signed-off-by: Morven Cao <lcao@redhat.com>
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (2)
test/setup/env_setup.sh (1)
203-240: Well-structured Pub/Sub initialization with proper error handling.The initialization flow properly validates job success before cleanup (lines 231-236), which addresses the previous review concern. The deployment wait, job creation, and cleanup sequence is well-organized.
However, the arbitrary 5-second sleep (line 215) before initialization is fragile. Consider replacing it with an active readiness check.
🔎 Proposed fix to replace sleep with readiness check
- # Initialize topics and subscriptions in the emulator - # Wait a bit for the emulator to be fully ready - sleep 5 + # Wait for emulator to be ready to accept requests + echo "Waiting for Pub/Sub emulator to be ready..." + for i in {1..30}; do + if kubectl -n ${namespace} exec deploy/maestro-pubsub -- curl -s http://localhost:8085 > /dev/null 2>&1; then + echo "Pub/Sub emulator is ready" + break + fi + if [ $i -eq 30 ]; then + echo "ERROR: Pub/Sub emulator failed to become ready" >&2 + exit 1 + fi + sleep 1 + donehack/init-pubsub-emulator.py (1)
19-66: Consider more specific exception handling.The function properly propagates errors by returning
Falseand logging to stderr, which is then checked inmain(). However, catching bareException(lines 35-37, 62-64) is overly broad. Consider catching specific exceptions likegoogle.api_core.exceptions.GoogleAPIErroror letting unexpected exceptions propagate naturally.🔎 Proposed refactor for more specific exception handling
def init_server_topics_and_subscriptions(project_id: str): """Initialize topics and subscriptions for the Maestro server.""" publisher = pubsub_v1.PublisherClient() subscriber = pubsub_v1.SubscriberClient() # Topics to create topics = ['sourceevents', 'sourcebroadcast', 'agentevents', 'agentbroadcast'] print("Creating topics...") for topic_name in topics: topic_path = publisher.topic_path(project_id, topic_name) try: publisher.create_topic(request={"name": topic_path}) print(f" ✓ Created topic: {topic_name}") except exceptions.AlreadyExists: print(f" - Topic already exists: {topic_name}") - except Exception as e: + except exceptions.GoogleAPIError as e: print(f" ✗ Error creating topic {topic_name}: {e}", file=sys.stderr) return FalseApply similar changes to subscription creation and agent initialization.
📜 Review details
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro
Cache: Disabled due to data retention organization setting
Knowledge base: Disabled due to Reviews -> Disable Knowledge Base setting
⛔ Files ignored due to path filters (1)
go.sumis excluded by!**/*.sum
📒 Files selected for processing (17)
.github/workflows/e2e.yml.gitignoreMakefileREADME.mdgo.modhack/init-pubsub-emulator.pytemplates/README.mdtemplates/agent-template.ymltemplates/agent-tls-template.ymltemplates/pubsub-agent-init-job-template.ymltemplates/pubsub-init-job-template.ymltemplates/pubsub-template.ymltemplates/service-template.ymltemplates/service-tls-template.ymltest/e2e/pkg/cert_rotation_test.gotest/setup/deploy_agent.shtest/setup/env_setup.sh
🚧 Files skipped from review as they are similar to previous changes (7)
- templates/agent-template.yml
- test/e2e/pkg/cert_rotation_test.go
- templates/pubsub-agent-init-job-template.yml
- test/setup/deploy_agent.sh
- templates/README.md
- README.md
- .gitignore
🧰 Additional context used
🪛 Ruff (0.14.10)
hack/init-pubsub-emulator.py
35-35: Do not catch blind exception: Exception
(BLE001)
62-62: Do not catch blind exception: Exception
(BLE001)
113-113: Do not catch blind exception: Exception
(BLE001)
145-145: Do not catch blind exception: Exception
(BLE001)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (8)
- GitHub Check: Red Hat Konflux / maestro-e2e-on-pull-request
- GitHub Check: Red Hat Konflux / maestro-on-pull-request
- GitHub Check: e2e
- GitHub Check: e2e-pubsub
- GitHub Check: e2e-grpc-broker
- GitHub Check: upgrade
- GitHub Check: e2e-broadcast-subscription
- GitHub Check: e2e-with-istio
🔇 Additional comments (8)
go.mod (1)
60-60: LGTM! Clean dependency resolution after upstream merge.The SDK dependency has been correctly updated to the official upstream version following the merge of PR #182. This removes the temporary fork dependency and uses the proper release channel.
templates/agent-tls-template.yml (2)
72-85: LGTM! Pub/Sub parameters follow consistent patterns.The new Pub/Sub parameters are well-defined with appropriate defaults for the emulator environment. The
required: trueflag with sensible defaults ensures the template can be used out-of-the-box for development while allowing production overrides.
383-397: LGTM! Pub/Sub secret configuration aligns with emulator usage.The
disableTLS: truesetting is appropriate here since the default endpoint (maestro-pubsub.maestro:8085) points to the Pub/Sub emulator, which doesn't support TLS. The topic and subscription naming patterns are consistent with the MQTT configuration structure, and the per-consumer subscription naming (sourceevents-${CONSUMER_NAME}) provides proper isolation..github/workflows/e2e.yml (1)
119-145: LGTM! Pub/Sub e2e job follows established patterns.The new
e2e-pubsubjob is well-structured and mirrors the configuration of existing e2e jobs. The environment variables (2 replicas, TLS enabled, Pub/Sub driver) provide appropriate coverage for testing the Pub/Sub message driver in CI alongside MQTT and gRPC variants.test/setup/env_setup.sh (1)
55-62: LGTM! OpenShift CLI installation properly implemented.The installation follows the same pattern as other tools (kind, step), downloads from the official mirror, and includes proper cleanup. This addresses the previous concern about
ocavailability for Pub/Sub initialization.templates/pubsub-template.yml (1)
1-116: LGTM! Well-structured Pub/Sub emulator template.The template properly defines all necessary resources (Service, Deployment, Secret) with appropriate configuration for the Google Cloud Pub/Sub emulator. The use of
emptyDirfor storage (lines 98-99) is appropriate for an emulator in testing scenarios where persistence across pod restarts is not required.The image tag
:emulators(line 22) has been confirmed as a deliberate choice for stability based on previous discussion.templates/pubsub-init-job-template.yml (1)
52-112: LGTM! Pub/Sub initialization Job properly configured with error handling.The inline Python script correctly:
- Creates all required topics (sourceevents, sourcebroadcast, agentevents, agentbroadcast)
- Creates subscriptions with appropriate filters
- Propagates exceptions on failure (lines 82, 109), ensuring the Job fails when initialization fails
- Uses the updated google-cloud-pubsub version (2.34.0)
The subscription filter
attributes.ce-originalsource="maestro"(line 86) has been confirmed as correct based on previous discussion—it matches events published by the agent via the SDK.Makefile (1)
65-69: LGTM! Pub/Sub support well-integrated following existing patterns.The Makefile changes consistently integrate Pub/Sub support:
- Configuration variables (lines 65-69) follow the same naming convention as MQTT variables
- Help documentation (lines 143-145) clearly describes the new targets
- Template parameter passing (lines 331-333) mirrors the MQTT parameter pattern
- Lifecycle targets (lines 446-460) match the structure of existing
mqtt/setup,mqtt/teardowntargetsThe
pubsub/inittarget (lines 457-460) properly invokes the Python initialization script with the correct environment variables.Also applies to: 143-145, 331-333, 446-460
1 participant
depends on: open-cluster-management-io/sdk-go#182