(feat) Support StatefulSet workload for ValkeyCluster

[sandeepkunusoth]

Today the operator creates a Deployment for ValkeyCluster pods (see internal/controller/deployment.go). Add support to create ValkeyCluster pods via a StatefulSet instead.

Proposed Spec (initial version):

spec:
 workload:
   kind: StatefulSet   # default: Deployment

Acceptance Criteria:

• If spec.workload.kind: StatefulSet is set, the operator creates and reconciles a StatefulSet and other dependencies accordingly for the ValkeyCluster pods.
• Default behavior remains unchanged (operator continues creating a Deployment when the field is not set).

Contribution:

• I’m willing to implement this feature.

[jdheyburn]

Can you outline what StatefulSets would be created. Would it be a 1:1 mapping with what we currently have for Deployment, except it would be a STS?

[jdheyburn]

Discussed on weekly meeting 2025-01-16. Today the Operator supports creating workloads in singleton Deployments (Deployments that will only ever have up to 1 replica). This allows us to have the most control on where pods are scheduled, removed, etc.

Each Deployment runs a single Valkey instance (with replicas set to 1 in the Deployment spec). The formula breaks down as:

• Shards: The number of primary nodes (data partitions)
• 1 + Replicas: Each shard has 1 primary + N replicas

For example:

• 3 shards with 1 replica each → 3 × (1 + 1) = 6 Deployments
• 3 shards with 2 replicas each → 3 × (1 + 2) = 9 Deployments

The StatefulSet implementation should support the same idea. More info on the design: #14

I imagine that there would be some code reuse amongst the Deployment and StatefulSet.

Long term we discussed abstracting away these two concrete implementations via a ValkeyNode CRD, that is out of scope for this effort.

We also discussed volumes for data persistence - I suggest leaving this out for now so that we can flesh out the design for an implementation that omits the use of volumeClaimTemplate.