[Kafka] Change Kafka installation - multi namespace support

[shay79il]

Migrating bitnami kafka to Strimzi Kafka operator

Add Strimzi Kafka operator configuration
Update values for Kafka deployment

JIRA

[GiladShapira94]

Looks good, can you edit the admin and the non-admin values files with the values that need to be use.
it does not need to support ingress and user that use NodePort will need to change the ports manually

[Copilot]

Pull request overview

This PR migrates the Kafka deployment from Bitnami Kafka to the Strimzi Kafka Operator, enabling multi-namespace support and modernizing the Kafka infrastructure with KRaft mode (ZooKeeper-less operation).

Key Changes:

• Replaced Bitnami Kafka chart dependency with Strimzi Kafka Operator (version 0.48.0)
• Introduced new Kubernetes custom resources for Kafka deployment including KafkaNodePool, Kafka cluster, RBAC resources, and network policies
• Configured single-node Kafka cluster with KRaft mode for simplified deployment

Reviewed changes

Copilot reviewed 8 out of 10 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
charts/mlrun-ce/values.yaml	Replaced Bitnami Kafka configuration with Strimzi operator values, including storage, resources, listeners, and RBAC settings; removed unrelated minio image config
charts/mlrun-ce/templates/kafka/kafka-cluster.yaml	Added Kafka custom resource definition for Strimzi operator with listener and config management
charts/mlrun-ce/templates/kafka/kafka-nodepool.yaml	Added KafkaNodePool resource for KRaft-mode Kafka cluster management
charts/mlrun-ce/templates/kafka/kafka-rbac.yaml	Created RBAC resources (ServiceAccount, Role, RoleBinding) for cross-namespace Kafka access
charts/mlrun-ce/templates/kafka/kafka-network-policy.yaml	Added NetworkPolicy to control egress traffic to Kafka cluster across namespaces
charts/mlrun-ce/templates/kafka/kafka-bootstrap-alias.yaml	Created service alias for simplified Kafka bootstrap server naming
charts/mlrun-ce/requirements.yaml	Updated chart dependency from bitnami/kafka to strimzi-kafka-operator
charts/mlrun-ce/requirements.lock	Updated lock file with new Strimzi operator dependency and digest
charts/mlrun-ce/Chart.yaml	Bumped chart version from 0.10.0-rc5 to 0.10.0-rc6
.gitignore	Added comprehensive .DS_Store file patterns for macOS

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

The NetworkPolicy allows unrestricted egress to all namespaces and pods (lines 56-58), which defeats the purpose of the network policy. This rule effectively allows traffic to anywhere, making the specific Kafka access rules redundant. Consider removing this overly permissive rule or adding a comment explaining why unrestricted egress is necessary for this use case.

Suggested change

	# Allow egress to the internet/other services (optional)
	# Comment out the next section if you want to restrict to Kafka only
	- to:
	- namespaceSelector: {}
	- podSelector: {}