Config follow-up: Implement component override functionality for Config resource

[andreaskaris]

This commit adds the ability to mark components as unmanaged in the
bpfman-operator, preventing the operator from creating or updating
specific objects. The implementation includes:

• Added ComponentOverride struct to Config API with fields for
  kind, group, namespace, name, and unmanaged flag
• Modified assureResource function to check for overrides and skip
  management of unmanaged components
• Implemented isOverridden helper function to match objects against
  override specifications
• Added tests to verify override functionality works correctly
  across all component types

Documentation

This will need documentation that states how to generate the overrides, e.g. (generated with an LLM, lgtm but haven't had the time to retest the below combinations, yet):

  apiVersion: bpfman.io/v1alpha1
  kind: Config
  metadata:
    name: bpfman-config
  spec:
    # Existing configuration...
    namespace: bpfman
    image: quay.io/bpfman/bpfman:latest
    agent:
      image: quay.io/bpfman/bpfman-agent:latest
      logLevel: info
    logLevel: info

    # Component override for the main deployment
    overrides:
    - kind: DaemonSet
      group: apps
      namespace: bpfman
      name: bpfman-daemon
      unmanaged: true

Using JSON patch for more precise control:

  kubectl patch config bpfman-config --type='json' -p='[
    {
      "op": "add",
      "path": "/spec/overrides",
      "value": [
        {
          "kind": "DaemonSet",
          "group": "apps",
          "namespace": "bpfman", 
          "name": "bpfman-daemon",
          "unmanaged": true
        }
      ]
    }
  ]'

  If you want to add multiple overrides or append to existing ones:

  kubectl patch config bpfman-config --type='json' -p='[
    {
      "op": "add",
      "path": "/spec/overrides/-",
      "value": {
        "kind": "DaemonSet",
        "group": "apps",
        "namespace": "bpfman",
        "name": "bpfman-daemon", 
        "unmanaged": true
      }
    }
  ]'

The /- notation appends to the existing overrides array rather than replacing it.

[mergify]

@andreaskaris, this pull request is now in conflict and requires a rebase.

[frobware]

ConfigMap should not be overrideable given that it's now an internal implementation detail of the Config CRD (post #461). Should isOverrideable narrow the list down to the DaemonSet, CSIDriver, SCC -- everthing else is excluded.

[frobware]

typo: s/overides/overrides

[frobware]

Is preventing creation desirable? Should the behaviour be to only prevent updates, not creation? What happens if a resource is accidentally deleted? Do we just drop the unmanaged state, and let the operator recreate it?

[frobware]

Should this be: unmanaged controls if cluster version operator bpfman operator should stop managing the ...

[frobware]

Post #461, isn't the configmap now an implementation detail? Should we allow for the configmap to be unmanaged?

[frobware]

I played around with the functionality and have some suggestions for UX in #489.

[frobware]

Replying to my own question here after experimenting with this.

For a debug/escape hatch, I think blocking creation is the right behaviour. The semantics become simple and predictable: unmanaged = the operator pretends this resource doesn't exist.

If you mark something as unmanaged and then delete it, you presumably meant to delete it. Recreating something marked "unmanaged" would be semantically incoherent - you're telling the operator not to manage it, but then expecting it to manage recreation.

If deletion was accidental, the recovery is straightforward: remove the override, and the operator recreates the resource on the next reconcile. I tested this flow and it works as expected.

The alternative (only blocking updates, allowing recreation) adds complexity and creates confusing edge cases. For a feature explicitly framed as a debug escape hatch for advanced users, I'd keep it simple and trust the user knows what they're doing.