Skip to content

feat: add global.jsonLog Helm value for production logging mode#1040

Open
geoah wants to merge 2 commits intokai-scheduler:mainfrom
geoah:main
Open

feat: add global.jsonLog Helm value for production logging mode#1040
geoah wants to merge 2 commits intokai-scheduler:mainfrom
geoah:main

Conversation

@geoah
Copy link
Copy Markdown

@geoah geoah commented Feb 19, 2026

Description

All KAI services default to development-mode logging with colored, human-readable console output optimized for reading logs directly from pods. However, when deploying to clusters with log aggregation platforms (Datadog, Splunk, etc.), ANSI color codes and multi-line stack traces make logs difficult to parse and correlate.

This PR adds an opt-in production logging mode configurable via a single Helm value:

helm install kai-scheduler kai-scheduler --set global.jsonLog=true

Related Issues

Fixes #1039

Checklist

Note: Ensure your PR title follows the Conventional Commits format (e.g., feat(scheduler): add new feature)

  • Self-reviewed
  • Added/updated tests (if needed)
  • Updated documentation (if needed)

Breaking Changes

  • Removes dev logging by default

Additional Notes

@coderabbitai
Copy link
Copy Markdown
Contributor

coderabbitai Bot commented Feb 19, 2026

Important

Review skipped

Auto reviews are disabled on this repository. Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@enoodle
Copy link
Copy Markdown
Collaborator

enoodle commented Feb 19, 2026

The color coding is essential for humans to read those logs.

As a person that reads them a lot it is much more convenient to do it from the pod than log aggregations as most of the time issues rise in test environments that don't have them and then the colors are important.

I think that adding a different logging mode that will fit your needs is important, so maybe you can add a flag to switch between the two options with the default staying as it is today.

Comment thread docs/operator/README.md Outdated
when running a service directly:

```bash
./binder --zap-devel=true
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should be configurable from the operator / helm chart

Comment thread docs/operator/README.md Outdated
## Logging

All KAI services use production-mode logging by default, producing JSON-formatted output
without ANSI color codes. This is optimized for log aggregation platforms where single-line
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't want you to remove the colors, they are important for debugging. It is funny how LLMs have hard time to remove things so they will find ways to mention what they deleted to leave some trace of it in the world.

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @enoodle, thank you for the review and comments, I've restored the original functionality and added a logJson config that disables zap-devel. Let me know if this is closer to how you think this should work. If not, some extra guidance would be appreciated! :) Thank you very much in advance!

Add opt-in JSON logging mode for log aggregation platforms.
When global.jsonLog is enabled via Helm, the operator
passes --zap-devel=false to controller-runtime services and --log-json
to the scheduler, switching to JSON-encoded output.
@geoah geoah changed the title refactor: switch to production-mode logging defaults for all services feat: add global.jsonLog Helm value for production logging mode Feb 20, 2026
Copy link
Copy Markdown
Collaborator

@enoodle enoodle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, I have one small comment.


common.AddK8sClientConfigToArgs(config.Service.K8sClientConfig, args)

if kaiConfig.Spec.Global.JSONLog != nil && *kaiConfig.Spec.Global.JSONLog {
Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you make a utility function for all those code clauses that repeat in the resources.go files?
Can be in pkg/operator/operands/common

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Improve logging ergonomics for production

2 participants