Support source CIDR when exposing services via load balancer IP

[mierea]

Hi,

Image: ghcr.io/leaseweb/cloudstack-kubernetes-provider:1.5.0
Image ID: ghcr.io/leaseweb/cloudstack-kubernetes-provider@sha256:27c365322613b5360008d76e5f1e84057166f3058002127682429dc20a571645

Is it possible to inject the source cidr list when exposing a service via the load balancer with a network offering that doesn't offer firewall capabilities?

We can do it manually but its not desirable since these configs should come from K8S.

I identified a situation in which loadBalancerSourceRanges does not work (be it defined as an annotation or a service spec) - in file https://github.com/leaseweb/cloudstack-kubernetes-provider/blob/main/cloudstack/cloudstack_loadbalancer.go

We have a network that does not have the firewall enabled, yet if I manually add a source cidr list from the cloudstack ui it will only allow traffic from those ranges; the same is not possible via the loadBalancerSourceRanges in the service spec as the cloudstack-kubecrnetes-provider will error out with:
LoadBalancerSourceRanges are ignored for Service mongodb/mongodb-mongodb-sharded because this CloudStack network does not support it