Skip to content

fix(operator): align Helm template imagePullSecrets with CRD schema#1368

Merged
enoodle merged 2 commits intokai-scheduler:mainfrom
mfchanou:fix/942-images-pull-secret-crd-mismatch
Apr 9, 2026
Merged

fix(operator): align Helm template imagePullSecrets with CRD schema#1368
enoodle merged 2 commits intokai-scheduler:mainfrom
mfchanou:fix/942-images-pull-secret-crd-mismatch

Conversation

@mfchanou
Copy link
Copy Markdown
Contributor

@mfchanou mfchanou commented Mar 31, 2026

Description

The Helm template for kai-config.yaml was writing spec.global.imagesPullSecret (a single string), but the CRD schema only defines spec.global.additionalImagePullSecrets (an array of strings). Kubernetes silently prunes unknown fields, so image pull secrets set via Helm were never applied.

This PR:

  • Updates the Helm template to write additionalImagePullSecrets as an array
  • Adds a backward-compatible deprecated imagesPullSecret (string) field to the CRD schema
  • Updates the controller to merge both fields (deduplicating), ensuring existing Config CRs with the legacy field continue to work after upgrade

Related Issues

Fixes #942

Checklist

  • Self-reviewed
  • Added/updated tests (if needed)
  • Updated documentation (if needed)

Breaking Changes

None. The deprecated imagesPullSecret field is preserved for backward compatibility. Existing Config CRs using this field will continue to work — the value is merged into additionalImagePullSecrets at runtime.

Additional Notes

  • The imagesPullSecret field is marked as deprecated and should be removed in a future major version
  • Helm now correctly passes all secrets from global.imagePullSecrets[] (previously only the first element was used)

Summary by CodeRabbit

  • Bug Fixes

    • Fixed a template rendering issue where image pull secrets configuration was not being properly applied.
  • New Features

    • Added support for specifying multiple image pull secrets as an array for greater flexibility.
  • Chores

    • Deprecated legacy single-value image pull secret field; existing values automatically migrate for backward compatibility.
    • Added comprehensive tests validating image pull secrets rendering behavior.

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Mar 31, 2026

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: 0b0c391e-d6ac-4da3-a766-a601fa7a5608

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
📝 Walkthrough

Walkthrough

This pull request resolves a schema mismatch between the Helm template and CRD definition for image pull secrets. It adds deprecated support for the singular imagesPullSecret field across schema and code, updates the template to use the correct array-based additionalImagePullSecrets, and implements backward-compatible migration logic with comprehensive tests.

Changes

Cohort / File(s) Summary
CRD Schema
deployments/kai-scheduler/crds/kai.scheduler_configs.yaml
Added deprecated imagesPullSecret field (string type) to the Config CRD schema under spec.properties with deprecation notice directing users to ImagePullSecrets.
Helm Template
deployments/kai-scheduler/templates/kai-config.yaml
Replaced single imagesPullSecret field with additionalImagePullSecrets array that iterates over all values in .Values.global.imagePullSecrets instead of only selecting the first entry.
Go API Types
pkg/apis/kai/v1/global.go, pkg/apis/kai/v1/zz_generated.deepcopy.go
Added deprecated ImagePullSecret *string field to GlobalConfig struct with migration logic in SetDefaultWhereNeeded that merges deprecated field into ImagePullSecrets array and performs deduplication before zeroing the deprecated field.
Operator Logic
pkg/operator/config/image_pull_secrets.go
Updated GetGlobalImagePullSecrets function to consider both deprecated ImagePullSecret and new ImagePullSecrets fields, with deduplication by comparing LocalObjectReference.Name to prevent duplicate secrets.
Documentation & Tests
CHANGELOG.md, deployments/kai-scheduler/tests/image_pull_secrets_test.yaml, pkg/operator/config/image_pull_secrets_test.go
Documented the fix in changelog and added comprehensive test coverage verifying correct rendering of array field, absence of deprecated field in expected scenarios, and proper handling of single and multiple secret configurations.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~22 minutes

Poem

🐰 A secret was split, now the arrays align,
Deprecated strings fade to let new lists shine,
Migration paths smooth, no breaking in sight,
Backward compatible fixes make upgrades a delight! 🔐

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Title check ✅ Passed The PR title clearly and specifically describes the main change: aligning the Helm template's imagePullSecrets handling with the CRD schema.
Description check ✅ Passed The PR description is comprehensive, covering the problem, solution, related issues, completed checklist items, breaking changes statement, and additional notes. All key sections from the template are addressed.
Linked Issues check ✅ Passed The PR fully addresses the requirements in issue #942: updates Helm template to write additionalImagePullSecrets as an array, adds deprecated imagesPullSecret to CRD schema, and updates controller to merge both fields with deduplication.
Out of Scope Changes check ✅ Passed All changes are directly aligned with fixing issue #942. The CHANGELOG update, CRD schema addition, Helm template modification, controller logic updates, and corresponding tests are all within scope.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@enoodle
Copy link
Copy Markdown
Collaborator

enoodle commented Mar 31, 2026

@coderabbitai review

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Mar 31, 2026

✅ Actions performed

Review triggered.

Note: CodeRabbit is an incremental review system and does not re-review already reviewed commits. This command is applicable only when automatic reviews are paused.

Comment thread CHANGELOG.md Outdated
Copy link
Copy Markdown
Contributor

@coderabbitai coderabbitai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (2)
deployments/kai-scheduler/crds/kai.scheduler_configs.yaml (1)

3501-3505: Correct schema addition for backward compatibility.

The deprecated field is properly typed and placed. The schema aligns with the Go struct's JSON tag imagesPullSecret.

One minor note: the deprecation message says "Use ImagePullSecrets instead" (referencing the Go field name), but users interacting via kubectl explain or YAML would need to use the actual JSON field name additionalImagePullSecrets. Consider updating to:

description: |-
  Deprecated: ImagePullSecret defines a single container registry secret credential.
  Use additionalImagePullSecrets instead.

This would make the migration path clearer for users working directly with YAML manifests.

,

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@deployments/kai-scheduler/crds/kai.scheduler_configs.yaml` around lines 3501
- 3505, Update the deprecation message for the imagesPullSecret CRD field to
reference the actual JSON/YAML field name users should migrate to: change the
description text in the imagesPullSecret schema (field name imagesPullSecret) to
say "Use additionalImagePullSecrets instead" so kubectl explain/YAML users see
the correct migration target.
pkg/operator/config/image_pull_secrets.go (1)

19-30: Consider centralizing legacy/new merge logic in one place.

This merge+dedeup path now exists here and in pkg/apis/kai/v1/global.go defaulting. Keeping one normalization source would reduce drift risk.

🤖 Prompt for AI Agents
Verify each finding against the current code and only fix it if needed.

In `@pkg/operator/config/image_pull_secrets.go` around lines 19 - 30, Centralize
the merge+dedupe logic for ImagePullSecret by extracting the block that checks
globalConfig.ImagePullSecret and appends to secretDeploymentObjs into a single
helper (e.g., MergeImagePullSecret or NormalizeImagePullSecrets) that accepts
the pointer/string and the []v1.LocalObjectReference and returns a deduplicated
slice; replace the duplicate logic in pkg/operator/config/image_pull_secrets.go
(symbols: globalConfig.ImagePullSecret, secretDeploymentObjs) and the logic in
pkg/apis/kai/v1/global.go with calls to this new helper so both locations share
the same normalization code and behavior; add/update tests for the helper to
cover nil/empty, existing, and new-secret cases.
🤖 Prompt for all review comments with AI agents
Verify each finding against the current code and only fix it if needed.

Inline comments:
In `@deployments/kai-scheduler/templates/kai-config.yaml`:
- Around line 43-46: The YAML loop rendering additionalImagePullSecrets may emit
non-string scalars for numeric-like values; update the template in the
additionalImagePullSecrets block (the {{- range .Values.global.imagePullSecrets
}} loop and its item rendering) to explicitly quote each item so the rendered
list items are strings (e.g. render the item as a quoted string like "{{ . }}"
instead of - {{ . }}).

---

Nitpick comments:
In `@deployments/kai-scheduler/crds/kai.scheduler_configs.yaml`:
- Around line 3501-3505: Update the deprecation message for the imagesPullSecret
CRD field to reference the actual JSON/YAML field name users should migrate to:
change the description text in the imagesPullSecret schema (field name
imagesPullSecret) to say "Use additionalImagePullSecrets instead" so kubectl
explain/YAML users see the correct migration target.

In `@pkg/operator/config/image_pull_secrets.go`:
- Around line 19-30: Centralize the merge+dedupe logic for ImagePullSecret by
extracting the block that checks globalConfig.ImagePullSecret and appends to
secretDeploymentObjs into a single helper (e.g., MergeImagePullSecret or
NormalizeImagePullSecrets) that accepts the pointer/string and the
[]v1.LocalObjectReference and returns a deduplicated slice; replace the
duplicate logic in pkg/operator/config/image_pull_secrets.go (symbols:
globalConfig.ImagePullSecret, secretDeploymentObjs) and the logic in
pkg/apis/kai/v1/global.go with calls to this new helper so both locations share
the same normalization code and behavior; add/update tests for the helper to
cover nil/empty, existing, and new-secret cases.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

Run ID: d612cd37-8871-41b3-8fb9-c1c3d6b95555

📥 Commits

Reviewing files that changed from the base of the PR and between 6d098c5 and b4006df.

📒 Files selected for processing (8)
  • CHANGELOG.md
  • deployments/kai-scheduler/crds/kai.scheduler_configs.yaml
  • deployments/kai-scheduler/templates/kai-config.yaml
  • deployments/kai-scheduler/tests/image_pull_secrets_test.yaml
  • pkg/apis/kai/v1/global.go
  • pkg/apis/kai/v1/zz_generated.deepcopy.go
  • pkg/operator/config/image_pull_secrets.go
  • pkg/operator/config/image_pull_secrets_test.go

Comment thread deployments/kai-scheduler/templates/kai-config.yaml
@github-actions
Copy link
Copy Markdown

Merging this branch will increase overall coverage

Impacted Packages Coverage Δ 🤖
github.com/kai-scheduler/KAI-scheduler/pkg/apis/kai/v1 0.00% (ø)
github.com/kai-scheduler/KAI-scheduler/pkg/operator/config 70.00% (+20.00%) 🎉

Coverage by file

Changed files (no unit tests)

Changed File Coverage Δ Total Covered Missed 🤖
github.com/kai-scheduler/KAI-scheduler/pkg/apis/kai/v1/global.go 0.00% (ø) 36 (+9) 0 36 (+9)
github.com/kai-scheduler/KAI-scheduler/pkg/apis/kai/v1/zz_generated.deepcopy.go 0.00% (ø) 299 (+4) 0 299 (+4)
github.com/kai-scheduler/KAI-scheduler/pkg/operator/config/image_pull_secrets.go 100.00% (ø) 14 (+8) 14 (+8) 0

Please note that the "Total", "Covered", and "Missed" counts above refer to code statements instead of lines of code. The value in brackets refers to the test coverage of that file in the old version of the code.

Changed unit test files

  • github.com/kai-scheduler/KAI-scheduler/pkg/operator/config/image_pull_secrets_test.go

Comment thread pkg/operator/config/image_pull_secrets.go Outdated
@enoodle
Copy link
Copy Markdown
Collaborator

enoodle commented Mar 31, 2026

@mfchanou The DCO check requires you to "sign off" on the commit
This looks good, please fix these few comments and we can proceed with this.

@mfchanou mfchanou force-pushed the fix/942-images-pull-secret-crd-mismatch branch from b4006df to a1fffe8 Compare April 1, 2026 10:08
@mfchanou
Copy link
Copy Markdown
Contributor Author

mfchanou commented Apr 1, 2026

@enoodle done. feel free to have a look

Copy link
Copy Markdown
Collaborator

@enoodle enoodle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mfchanou this is great! I just need you to fix this CHANGELOG update please:

Comment thread CHANGELOG.md Outdated
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 1, 2026

Merging this branch will increase overall coverage

Impacted Packages Coverage Δ 🤖
github.com/kai-scheduler/KAI-scheduler/pkg/apis/kai/v1 8.21% (+8.21%) 👍

Coverage by file

Changed files (no unit tests)

Changed File Coverage Δ Total Covered Missed 🤖
github.com/kai-scheduler/KAI-scheduler/pkg/apis/kai/v1/global.go 88.89% (+88.89%) 36 (+9) 32 (+32) 4 (-23) 🌟
github.com/kai-scheduler/KAI-scheduler/pkg/apis/kai/v1/zz_generated.deepcopy.go 0.00% (ø) 299 (+4) 0 299 (+4)

Please note that the "Total", "Covered", and "Missed" counts above refer to code statements instead of lines of code. The value in brackets refers to the test coverage of that file in the old version of the code.

Changed unit test files

  • github.com/kai-scheduler/KAI-scheduler/pkg/apis/kai/v1/global_test.go

Update Helm template to write additionalImagePullSecrets (array) instead
of the non-existent imagesPullSecret (string) field. Add deprecated
imagesPullSecret field to CRD for backward compatibility with existing
Config resources.

Fixes kai-scheduler#942

Signed-off-by: mfchanou <chanoumahfuz@gmail.com>
@mfchanou mfchanou force-pushed the fix/942-images-pull-secret-crd-mismatch branch from 310945c to 912f7c6 Compare April 1, 2026 12:35
enoodle
enoodle previously approved these changes Apr 1, 2026
@mfchanou
Copy link
Copy Markdown
Contributor Author

mfchanou commented Apr 1, 2026

@enoodle any way to relaunch the failing job? seems to be an issue with the ci infra

[FAILED] failed to create test cluster context Unexpected error: <*errors.StatusError | 0x71dd8170820>: Internal error occurred: failed calling webhook "queue-validation.kai.scheduler": failed to call webhook: Post "[https://queue-controller.kai-scheduler.svc:443/validate-scheduling-run-ai-v2-queue?timeout=10s](https://queue-controller.kai-scheduler.svc/validate-scheduling-run-ai-v2-queue?timeout=10s)": dial tcp 10.96.40.11:443: connect: connection refused { ErrStatus: { TypeMeta: {Kind: "", APIVersion: ""}, ListMeta: { SelfLink: "", ResourceVersion: "", Continue: "", RemainingItemCount: nil, }, Status: "Failure", Message: "Internal error occurred: failed calling webhook \"queue-validation.kai.scheduler\": failed to call webhook: Post \"[https://queue-controller.kai-scheduler.svc:443/validate-scheduling-run-ai-v2-queue?timeout=10s\](https://queue-controller.kai-scheduler.svc/validate-scheduling-run-ai-v2-queue?timeout=10s\)": dial tcp 10.96.40.11:443: connect: connection refused",

@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 2, 2026

Merging this branch will increase overall coverage

Impacted Packages Coverage Δ 🤖
github.com/kai-scheduler/KAI-scheduler/pkg/apis/kai/v1 8.21% (+8.21%) 👍

Coverage by file

Changed files (no unit tests)

Changed File Coverage Δ Total Covered Missed 🤖
github.com/kai-scheduler/KAI-scheduler/pkg/apis/kai/v1/global.go 88.89% (+88.89%) 36 (+9) 32 (+32) 4 (-23) 🌟
github.com/kai-scheduler/KAI-scheduler/pkg/apis/kai/v1/zz_generated.deepcopy.go 0.00% (ø) 299 (+4) 0 299 (+4)

Please note that the "Total", "Covered", and "Missed" counts above refer to code statements instead of lines of code. The value in brackets refers to the test coverage of that file in the old version of the code.

Changed unit test files

  • github.com/kai-scheduler/KAI-scheduler/pkg/apis/kai/v1/global_test.go

@mfchanou
Copy link
Copy Markdown
Contributor Author

mfchanou commented Apr 2, 2026

Hi @enoodle I still need 2 approvers before merging the PR? could someone have a look ?

@enoodle enoodle requested a review from SiorMeir April 3, 2026 21:56
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Apr 7, 2026

Merging this branch will increase overall coverage

Impacted Packages Coverage Δ 🤖
github.com/kai-scheduler/KAI-scheduler/pkg/apis/kai/v1 8.21% (+8.21%) 👍

Coverage by file

Changed files (no unit tests)

Changed File Coverage Δ Total Covered Missed 🤖
github.com/kai-scheduler/KAI-scheduler/pkg/apis/kai/v1/global.go 88.89% (+88.89%) 36 (+9) 32 (+32) 4 (-23) 🌟
github.com/kai-scheduler/KAI-scheduler/pkg/apis/kai/v1/zz_generated.deepcopy.go 0.00% (ø) 299 (+4) 0 299 (+4)

Please note that the "Total", "Covered", and "Missed" counts above refer to code statements instead of lines of code. The value in brackets refers to the test coverage of that file in the old version of the code.

Changed unit test files

  • github.com/kai-scheduler/KAI-scheduler/pkg/apis/kai/v1/global_test.go

@mfchanou
Copy link
Copy Markdown
Contributor Author

mfchanou commented Apr 8, 2026

@SiorMeir could you please approve this PR ?

@enoodle enoodle added this pull request to the merge queue Apr 9, 2026
@github-merge-queue github-merge-queue Bot removed this pull request from the merge queue due to failed status checks Apr 9, 2026
@enoodle enoodle added this pull request to the merge queue Apr 9, 2026
Merged via the queue into kai-scheduler:main with commit 1499166 Apr 9, 2026
11 of 14 checks passed
github-actions Bot pushed a commit that referenced this pull request Apr 9, 2026
…1368)

Signed-off-by: mfchanou <chanoumahfuz@gmail.com>
Co-authored-by: Erez Freiberger <enoodle@gmail.com>
(cherry picked from commit 1499166)
@KaiPilotBot
Copy link
Copy Markdown

Successfully created backport PR for v0.9:

@KaiPilotBot
Copy link
Copy Markdown

Backport failed for v0.6, because it was unable to cherry-pick the commit(s).

Please cherry-pick the changes locally and resolve any conflicts.

git fetch origin v0.6
git worktree add -d .worktree/backport-1368-to-v0.6 origin/v0.6
cd .worktree/backport-1368-to-v0.6
git switch --create backport-1368-to-v0.6
git cherry-pick -x 1499166060df0027abdffd7c6e27f3557e3d8d2b

@KaiPilotBot
Copy link
Copy Markdown

Successfully created backport PR for v0.12:

github-actions Bot pushed a commit that referenced this pull request Apr 9, 2026
…1368)

Signed-off-by: mfchanou <chanoumahfuz@gmail.com>
Co-authored-by: Erez Freiberger <enoodle@gmail.com>
(cherry picked from commit 1499166)
github-actions Bot pushed a commit that referenced this pull request Apr 9, 2026
…1368)

Signed-off-by: mfchanou <chanoumahfuz@gmail.com>
Co-authored-by: Erez Freiberger <enoodle@gmail.com>
(cherry picked from commit 1499166)
@KaiPilotBot
Copy link
Copy Markdown

Successfully created backport PR for v0.13:

github-actions Bot pushed a commit that referenced this pull request Apr 9, 2026
…1368)

Signed-off-by: mfchanou <chanoumahfuz@gmail.com>
Co-authored-by: Erez Freiberger <enoodle@gmail.com>
(cherry picked from commit 1499166)
@KaiPilotBot
Copy link
Copy Markdown

Successfully created backport PR for v0.14:

enoodle added a commit that referenced this pull request Apr 22, 2026
…1368)

Backport of #1368 to v0.9.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

Signed-off-by: Erez Freiberger <enoodle@gmail.com>
enoodle added a commit that referenced this pull request Apr 22, 2026
…1368)

Backport of #1368 to v0.12.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>

Signed-off-by: Erez Freiberger <enoodle@gmail.com>
enoodle pushed a commit that referenced this pull request Apr 22, 2026
…1368)

Signed-off-by: mfchanou <chanoumahfuz@gmail.com>
Co-authored-by: Erez Freiberger <enoodle@gmail.com>
Signed-off-by: Erez Freiberger <enoodle@gmail.com>

Signed-off-by: mfchanou <38184317+mfchanou@users.noreply.github.com>
enoodle pushed a commit that referenced this pull request Apr 22, 2026
…1368)

Signed-off-by: mfchanou <chanoumahfuz@gmail.com>
Co-authored-by: Erez Freiberger <enoodle@gmail.com>
Signed-off-by: Erez Freiberger <enoodle@gmail.com>

Signed-off-by: mfchanou <38184317+mfchanou@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

imagesPullSecret used in Helm template but not defined in CRD schema

4 participants