feat: Introduce controller concurrency and Kubernetes API flags by tomergee · Pull Request #368 · kubernetes-sigs/agent-sandbox

tomergee · 2026-03-05T21:46:46Z

API flags configuration options with corresponding documentation, default values remain the same values as before:

--concurrent-workers (default: 1): The maximum number of concurrent reconciles for the controllers. Increase this value to process multiple Sandbox and SandboxClaim events in parallel.
--kube-api-qps (default: 20): The maximum Queries Per Second (QPS) sent to the Kubernetes API server from the controller.
--kube-api-burst (default: 30): The maximum burst for throttle requests to the Kubernetes API server.
--sandbox-concurrent-workers (default: 1): The maximum number of concurrent reconciles for the Sandbox controller.
--sandbox-claim-concurrent-workers (default: 1): The maximum number of concurrent reconciles for the SandboxClaim controller.
--sandbox-warm-pool-concurrent-workers (default: 1): The maximum number of concurrent reconciles for the SandboxWarmPool controller.
--kube-api-qps (default: -1 ; no rate limiting): The maximum Queries Per Second (QPS) sent to the Kubernetes API server from the controller.
--kube-api-burst (default: 10): The maximum burst for throttle requests to the Kubernetes API server.
in cmd/agent-sandbox-controller/main.go

Plumbed the new limits into restConfig before setting up the controller manager. Updated SetupWithManager signatures across all controller files.

…onfiguration options with corresponding documentation. default values remain the same values as before --kube-api-qps (default 20) --kube-api-burst (default 30) --concurrent-workers (default 1) in cmd/agent-sandbox-controller/main.go Plumbed the new limits into restConfig before setting up the controller manager. Updated SetupWithManager signatures across all controller files.

netlify · 2026-03-05T21:46:52Z

✅ Deploy Preview for agent-sandbox canceled.

Name	Link
🔨 Latest commit	`7aceee5`
🔍 Latest deploy log	https://app.netlify.com/projects/agent-sandbox/deploys/69aa4e878267160008bef37f

k8s-ci-robot · 2026-03-05T21:46:56Z

Hi @tomergee. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work.

Tip

We noticed you've done this a few times! Consider joining the org to skip this step and gain /lgtm and other bot rights. We recommend asking approvers on your previous PRs to sponsor you.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

vicentefb · 2026-03-05T21:51:47Z

/ok-to-test

tomergee · 2026-03-05T21:52:45Z

/ok-to-test

…onfiguration options with corresponding documentation.

igooch

Should we keep the QPS default the same as controller-runtime? If yes, please also update the documentation as well.

cmd/agent-sandbox-controller/main.go

docs/configuration.md

…mPool controllers, and adjust kube API rate limit defaults Added error and logical validation on flag values, updated configuration.md and refernced release manifests

… and SandboxWarmPool controllers, and adjust kube API rate limit defaults.

aditya-shantanu · 2026-03-06T19:02:21Z

cmd/agent-sandbox-controller/main.go

+	if kubeAPIQPS == 0 || kubeAPIQPS < -1 {
+		setupLog.Error(nil, "kube-api-qps must be greater than 0, or -1 for no rate limiting")
+		os.Exit(1)
+	}


Do we ever want it to be -1 ?
Wouldn't it be safer to say this has to be > 0 ?

-1 is the default value that comes with the controller (before the change)

-1 means disabling throttling (no limits).

k8s-ci-robot · 2026-03-06T19:02:51Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: aditya-shantanu, tomergee
Once this PR has been reviewed and has the lgtm label, please assign janetkuo for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

janetkuo · 2026-03-06T19:39:12Z

cmd/agent-sandbox-controller/main.go


-	mgr, err := ctrl.NewManager(ctrl.GetConfigOrDie(), ctrl.Options{
+	restConfig := ctrl.GetConfigOrDie()
+	restConfig.QPS = float32(kubeAPIQPS)


We should define kubeAPIQPS as float instead of int to allow for fractional values like 0.5

janetkuo · 2026-03-06T19:42:27Z

cmd/agent-sandbox-controller/main.go

+	}
+	// A logical maximum (too much will create unnecessary load on the API server)
+	totalWorkers := sandboxConcurrentWorkers + sandboxClaimConcurrentWorkers + sandboxWarmPoolConcurrentWorkers
+	if totalWorkers > 1000 {


How is 1000 decided? Do we need to enforce a max now? It might work in a very large cluster with a large master VM.

What do you think is a logical maximum? we should enforce something? otherwise its just wasting resources and performance...

cmd/agent-sandbox-controller/main.go

k8s-ci-robot requested review from janetkuo and justinsb March 5, 2026 21:46

k8s-ci-robot added the needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. label Mar 5, 2026

k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Mar 5, 2026

tomergee changed the title ~~feat: Introduce controller concurrency and Kubernetes API QPS/burst c…~~ feat: Introduce controller concurrency and Kubernetes API flags Mar 5, 2026

k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Mar 5, 2026

feat: Introduce controller concurrency and Kubernetes API QPS/burst c…

b5e0d31

…onfiguration options with corresponding documentation.

igooch reviewed Mar 5, 2026

View reviewed changes

cmd/agent-sandbox-controller/main.go Outdated Show resolved Hide resolved

janetkuo reviewed Mar 5, 2026

View reviewed changes

tomergee added 3 commits March 6, 2026 03:35

Seperated concurrency flags for Sandbox, SandboxClaim, and SandboxWar…

3f86875

…mPool controllers, and adjust kube API rate limit defaults Added error and logical validation on flag values, updated configuration.md and refernced release manifests

feat: Introduce granular concurrency flags for Sandbox, SandboxClaim,…

4aa6600

… and SandboxWarmPool controllers, and adjust kube API rate limit defaults.

Deleting whitespaces in messages according to linter

7aceee5

aditya-shantanu reviewed Mar 6, 2026

View reviewed changes

aditya-shantanu approved these changes Mar 6, 2026

View reviewed changes

janetkuo reviewed Mar 6, 2026

View reviewed changes

Conversation

tomergee commented Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

netlify bot commented Mar 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

✅ Deploy Preview for agent-sandbox canceled.

Uh oh!

k8s-ci-robot commented Mar 5, 2026

Uh oh!

vicentefb commented Mar 5, 2026

Uh oh!

tomergee commented Mar 5, 2026

Uh oh!

igooch left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

aditya-shantanu Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

tomergee Mar 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

janetkuo Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

k8s-ci-robot commented Mar 6, 2026

Uh oh!

janetkuo Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

janetkuo Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

tomergee Mar 6, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

7 participants

tomergee commented Mar 5, 2026 •

edited

Loading

netlify bot commented Mar 5, 2026 •

edited

Loading

tomergee Mar 6, 2026 •

edited

Loading