diff --git a/_posts/2025-11-05-crossplane-v2-creating-kubernetes-resources.md b/_posts/2025-11-05-crossplane-v2-creating-kubernetes-resources.md new file mode 100644 index 0000000..1189edd --- /dev/null +++ b/_posts/2025-11-05-crossplane-v2-creating-kubernetes-resources.md @@ -0,0 +1,98 @@ +--- +title: "Crossplane v2 - Creating Kubernetes Managed Resources" +layout: post +date: 2025-11-05 +categories: + - "automation" + - "containers" + - "devops" + - "integrations" +tags: + - "automation" + - "cloud" + - "crossplane" + - "integration" + - "kubernetes" + - "microservices" +--- + +I've been playing a lot with [Crossplane](https://www.crossplane.io/) recently and found myself a little stuck trying to create additional Kubernetes resources using Crossplane itself as the orchestration layer. This is pretty well documented in disparate places, but I haven't seen it covered anywhere top to bottom so in this short post I'll try and cover the topic off. + + + +## Installing and Configuring the Kubernetes Provider + +In this example I am installing the Kubernetes *Provider* and creating a *ClusterProviderConfig* to serve it out to all tenants. Since we don't have to think about authenticating with a cloud provider like AWS, we can keep everything inside the cluster and that should make our lives a lot easier. For the most part this installation behaves the same as all others, with the exception of how the *credential* source is handled: + +```yaml +apiVersion: pkg.crossplane.io/v1 +kind: Provider +metadata: + name: provider-kubernetes +spec: + package: xpkg.upbound.io/upbound/provider-kubernetes:v1.1.0 +--- +apiVersion: kubernetes.m.crossplane.io/v1alpha1 #--Namespaced endpoint +kind: ClusterProviderConfig #--Cluster wide +metadata: + name: kubernetes +spec: + credentials: + source: InjectedIdentity #--Use the provider's existing SA +``` + +The *credential source* above is configured to use an *InjectedIdentity*. What this means is that the *Service Account* attached to the *Kubernetes Provider Pod* is used as the source of auth when attempting to create resources. When a provider is installed the Crossplane RBAC Manager automatically creates a *SA* and binds it to a *ClusterRole* with a suitable set of permissions (it's a pretty cool system). + +With the Kubernetes *Provider* this is quite limited in what you can do by default, limiting you to creating *ConfigMaps*, *Secrets* and managing other *Crossplane* resources. But if you want to widen the scope of this you only have to create your own *SA* and attach it to the provider as shown below: + +```yaml +apiVersion: pkg.crossplane.io/v1beta1 +kind: DeploymentRuntimeConfig +metadata: + name: kubernetes +spec: + serviceAccountTemplate: + metadata: + name: provider-k8s-enhanced #--Your SA name here +--- +apiVersion: pkg.crossplane.io/v1 +kind: Provider +metadata: + name: provider-kubernetes +spec: + package: xpkg.upbound.io/upbound/provider-kubernetes:v1.1.0 + runtimeConfigRef: + name: kubernetes #--Links to the Runtime Config above +``` + +With a *Provider* installed and configured, you can created Kubernetes resources by passing manifests nested within the Crossplane *MR* type `object.kubernetes.m.crossplane.io` as shown below: + +```yaml +apiVersion: kubernetes.m.crossplane.io/v1alpha1 #--Namespaced resource +kind: Object +metadata: + name: tfc-example-secret +spec: + forProvider: + manifest: + #--Begin Kubernetes object manifest + apiVersion: v1 + kind: Secret + metadata: + name: tfc-example-secret + namespace: tenant1 + data: + secret_information: c29tZXJ1YmJpc2gK + #--End kubernetes object manifest + providerConfigRef: + name: kubernetes #--Created above + kind: ClusterProviderConfig +``` + +We can see the resource has created and can be looked up using regular Kubernetes verbs: + +```bash +kuebctl get secrets -n tenant1 +NAME TYPE DATA AGE +tfc-example-secret Opaque 1 1m +``` diff --git a/_posts/2025-11-06-crossplane-v2-aws-ecr-integration.md b/_posts/2025-11-06-crossplane-v2-aws-ecr-integration.md new file mode 100644 index 0000000..f43dfc5 --- /dev/null +++ b/_posts/2025-11-06-crossplane-v2-aws-ecr-integration.md @@ -0,0 +1,205 @@ +--- +title: "Crossplane v2 - Integrating With AWS ECR" +layout: post +date: 2025-11-06 +categories: + - "automation" + - "aws" + - "containers" + - "devops" + - "integrations" +tags: + - "automation" + - "aws" + - "cloud" + - "crossplane" + - "ecr" + - "integration" + - "kubernetes" + - "microservices" +--- + +Recently I've been playing a lot with [Crossplane](https://www.crossplane.io/) and I ran up against needing to work with images in a [Private ECR Registry](https://docs.aws.amazon.com/AmazonECR/latest/userguide/Registries.html). + +Crossplane has a built in solution to working with a private image registry using what they call **ImageConfigs** and a classic *ImagePullSecret*, you can see a breakdown of how that works [in the docs here](https://docs.crossplane.io/latest/packages/image-configs/#configuring-a-pull-secret), but as anyone who has worked with private ECR will know, it is particularly allergic to using static secrets and likes to use *STS*. In this post I'll take a look at my suggestions for how to integrate in two scenarios: + +- Installing Crossplane (where the installation image comes from a private *ECR Registry*) +- Installing *Providers* and *Functions* from a private *ECR Registry* + + + +## Assumptions + +As part of this article, I am going to assume that you have already created some *Repositories* in your private *ECR Registry* and managed to get some images in there, I won't be covering how to go about that or the guide will be too long. + +I will be assuming that you have the below *Repositories* and images created in your *ECR Registry* (all pulled from xpkg.crossplane.io originally): + +- crossplane/crossplane:latest +- upbound/provider-family-aws:v2.2.0 +- upbound/provider-aws-s3:v2.1.0 +- upbound/function-patch-and-transform:v0.9.0 + +## Installing Crossplane - Configuring NodeGroup Permissions + +Despite running two different application *Pods*, crossplane actually only pulls a single image as part of it's installation; `crossplane/crossplane` so that's one less thing to worry about. + +We can most easily ensure that the image can be transparently pulled by expanding the scope of the *NodeGroup Instance Role* for the *Nodes* Crossplane will run on to include these additional permissions: + +```json +{ + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Action": [ + "ecr:GetAuthorizationToken" + ], + "Resource": "*" + }, + { + "Effect": "Allow", + "Action": [ + "ecr:BatchCheckLayerAvailability", + "ecr:GetDownloadUrlForLayer", + "ecr:BatchGetImage" + ], + "Resource": [ + "arn:aws:ecr:eu-west-2:1234567890123:crossplane/crossplane" //--Your image repo ARN here + ] + } + ] +} +``` + +These permissions can be created as an additional *IAM Policy*, then bound to your existing *NodeGroup Instance Role* (if you have multiple *NG Instance Roles* you can assign this as you see fit). To determine your current *NodeGroup Instance Role* configuration, use the AWS CLI: + +```bash +aws eks list-nodegroups --cluster-name tfc-cluster1 +{ + "nodegroups": [ + "tooling", + "regular_workloads", + "secure_workloads" + ] +} + +aws eks describe-nodegroup --cluster-name tfc-cluster1 --nodegroup-name regular_workloads | grep nodeRole + "nodeRole": "arn:aws:iam::1234567890123:role/regular_workloads", #--Your NG Instance Role ARN +``` + +## Installing Crossplane - Service Account Considerations + +With this policy in place, the Crossplane image can be pulled, but additional configuration is needed in advance to ensure our *Providers* and *Functions* can be installed. Their installation is handled via the *Crossplane Service Account* which needs to be bound to an existing IAM role with permissions to pull images. + +Your IAM role must have at least the permissions shown below: + +```json +{ + "Version": "2012-10-17", + "Statement": [ + { + "Effect": "Allow", + "Action": [ + "ecr:GetAuthorizationToken" + ], + "Resource": "*" + }, + { + "Effect": "Allow", + "Action": [ + "ecr:BatchCheckLayerAvailability", + "ecr:GetDownloadUrlForLayer", + "ecr:BatchGetImage" + ], + "Resource": [ + "arn:aws:ecr:eu-west-2:1234567890123:upbound/provider-*", //--Wildcarding is to allow pulling + "arn:aws:ecr:eu-west-2:1234567890123:upbound/function-*" //--all Providers and functions + ] + } + ] +} +``` + +Once this role exists, Crossplane can be installed with the below *Helm Values*: + +```yaml +#--helm_values.yaml + +image: + repository: 1234567890123.dkr.ecr.eu-west-2.amazonaws.com/crossplane/crossplane + tag: latest +serviceAccount: + customAnnotations: + eks.amazonaws.com/role-arn: $PROVIDER_AND_FUNCTION_PULL_IAM_ROLE_ARN +``` + +Install with + +```bash +helm install crossplane -n crossplane-system crossplane-stable/crossplane -f helm_values.yaml +``` + +## Installing Providers and Functions + +Once Crossplane is stable, *Providers* and *Functions* can be installed, the below image highlights what we're aiming for: + + + +There is an important gotcha when working with a private registry around provider dependencies, these are not perfectly handled and can lead to some very confusing errors and behavior + +To highlight how this will behave in reality, if we were to apply the below on a fresh Crossplane deployment: + +```yaml +apiVersion: pkg.crossplane.io/v1 +kind: Provider +metadata: + name: provider-aws-s3 +spec: + package: 1234567890123.dkr.ecr.eu-west-2.amazonaws.com/upbound/provider-aws-s3:v2.2.0 +``` + +We will see: + +```bash +kubectl get provider +# NAME INSTALLED HEALTHY PACKAGE AGE +# upbound-provider-family-aws False False xpkg.upbound.io/upbound/upbound-provider-family-aws:v2.2.0 8m +# provider-aws-s3 True True 1234567890123.dkr.ecr.eu-west-2.amazonaws.com/upbound/provider-aws-s3:v2.2.0 9m +``` + +The dependency `upbound-provider-family-aws` contains essential dependencies for the `s3 Provider` to function, as such it has attempted to install automatically. When using a private registry however, this installation will fail, if I remember correctly this is due to a dependency resolution error that occurs when the dependencies have a URI that doesn't match the *Provider* (don't quote me on that, I can't find my source)! + +It is worth pointing out that even if your *Provider* has somehow managed to install and shows as healthy, but the dependencies haven't, failures are still going to occur somewhere. These often manifest in in the form of *ServiceAccount RBAC* errors relating to *ProviderConfigs* and *ProviderConfigUsages*. This stems from the fact that when a *Provider*'s dependencies fail to install, a complete set of *CRD*s underpinning the *Provider* have also not been installed. These *CRD*s are supposed to be part of a *ClusterRole* that has failed to materialise correctly. + +These problems can be pretty confusing, hard to pin down and are really best avoided all together. + +To steer clear of this it is best to install both the *Providers* **AND** their dependencies manually: + +```yaml +#---Install Deps +apiVersion: pkg.crossplane.io/v1 +kind: Provider +metadata: + name: provider-family-aws +spec: + package: 1234567890123.dkr.ecr.eu-west-2.amazonaws.com/upbound/provider-family-aws:v2.2.0 + +#---Install Provider, skipping dependency resolution +apiVersion: pkg.crossplane.io/v1 +kind: Provider +metadata: + name: provider-aws-s3 +spec: + package: 1234567890123.dkr.ecr.eu-west-2.amazonaws.com/upbound/provider-aws-s3:v2.2.0 + skipDependencyResolution: true #--Skip dependency resolution. We have installed manually + #--if unset, deps will still be installed from xpkg.upbound.io +``` + +We can verify all is well with: + +```bash +kubectl get provider +# NAME INSTALLED HEALTHY PACKAGE AGE +# upbound-provider-family-aws True True xpkg.upbound.io/upbound/upbound-provider-family-aws:v2.2.0 12m +# provider-aws-s3 True True 1234567890123.dkr.ecr.eu-west-2.amazonaws.com/upbound/provider-aws-s3:v2.2.0 11m +``` \ No newline at end of file diff --git a/_posts/2025-11-16-crossplane-v2-connecting-to-a-remote-eks-cluster.md b/_posts/2025-11-16-crossplane-v2-connecting-to-a-remote-eks-cluster.md new file mode 100644 index 0000000..fa177bc --- /dev/null +++ b/_posts/2025-11-16-crossplane-v2-connecting-to-a-remote-eks-cluster.md @@ -0,0 +1,149 @@ +--- +title: "Crossplane v2 - Connecting To A Remote EKS Cluster" +layout: post +date: 2025-11-16 +categories: + - "automation" + - "aws" + - "containers" + - "devops" + - "integrations" +tags: + - "automation" + - "aws" + - "cloud" + - "crossplane" + - "eks" + - "integration" + - "kubernetes" + - "microservices" +--- + +[Recently I took a look at managing Kubernetes objects using Crossplane]({% post_url 2025-11-05-crossplane-v2-creating-kubernetes-resources %}). That process was pretty straight forward but it got me thinking about a more complex integration that we sometimes need to do in to in the wild. This is the need to authenticate with with another, remote *EKS Cluster* and either read from or write to it. + +In this scenario the goal is to read/write to a remote *EKS Cluster* from our existing Crossplane deployment. This one is pretty niche, probably not on many people's minds and predictably isn't covered in the documentation...but sooner or later someone is going to want to do it...so here we go! + + + +## Jumping Through Hoops + +Authentication with EKS can be a bit of an involved process, from a high level, what we'll need to do in Crossplane is: + +- Install the *Provider* for **aws-eks** +- Authenticate with the remote *EKS Cluster*, using an appropriate IAM Role +- Obtain credentials from the remote *EKS Cluster*, in the form of a *kubeconfig* +- Install the *Provider* for **kubernetes** and configure it with our *kubeconfig* +- Finally, use the *Kubernetes Provider* to actually manage resources on the remote *EKS Cluster* + +In practice, that process looks like this: + + + +So a lot of moving parts there...but this isn't as scary as it looks. Mostly this is just the same thing that happens when you type `aws eks update-kubeconfig` in to the terminal. How hard could it be... + +## Install and Configure the aws-eks Provider + +First we create a *DeploymentRuntimeConfig* which is linked to an *AWS IAM Role*. This *Role* needs to have permissions to authenticate to the remote *EKS Cluster* and perform whatever management operations you ultimately plan to do. + +We will also need to create a *Provider* using the `aws-eks` package which uses this *RuntimeConfig*, and a *ProviderConfig* configured to use the IRSA *credential source*. + +```yaml +apiVersion: pkg.crossplane.io/v1beta1 +kind: DeploymentRuntimeConfig +metadata: + name: eks +spec: + serviceAccountTemplate: + metadata: + annotations: + eks.amazonaws.com/role-arn: $REMOTE_CLUSTER_IAM_ROLE_ARN +--- +apiVersion: pkg.crossplane.io/v1 +kind: Provider +metadata: + name: provider-aws-eks +spec: + package: xpkg.upbound.io/upbound/provider-aws-eks:v2.2.0 + runtimeConfigRef: + name: eks +--- +apiVersion: pkg.crossplane.io/v1 +kind: ProviderConfig #--Not cluster-wide. Not for tenant consumption +metadata: + name: eks +spec: + credentials: + source: IRSA #--Auth using IAM role from DeploymentRuntimeConfig +``` + +## Authenticate With The Remote EKS Cluster + +The *aws-eks Provider* provides a *Managed Resource* called *ClusterAuth*. It's purpose is to perform *STS* authentication with a remote *EKS Cluster*. Pairing this with Crossplane's `writeConnectionSecretToRef` functionality, we can authenticate and write the output of the authentication process to a secret on our cluster. + +```yaml +apiVersion: eks.aws.upbound.io/v1beta1 +kind: ClusterAuth +metadata: + name: tfc-cluster12 +spec: + forProvider: + clusterName: tfc-cluster12 #--Remote EKS Cluster name + region: eu-west-2 + providerConfigRef: + name: eks + #--Write connection params, including kubeconf out to a Secret + writeConnectionSecretToRef: + name: tfc-cluster12-connection + namespace: crossplane-system +``` + +## Install and Configure kubernetes Provider + +The kubeconfig to authenticate with the remote cluster is now saved as the *key* `kubeconfig` inside the *Secret* `crossplane-system/tfc-cluster12-connection`. Using this, we can install the Kubernetes *Provider* and configure a *ClusterProviderConfig* which uses the kubeconfig for authentication: + +```yaml +apiVersion: pkg.crossplane.io/v1 +kind: Provider +metadata: + name: provider-kubernetes +spec: + package: xpkg.upbound.io/upbound/provider-kubernetes:v1.1.0 +--- +apiVersion: kubernetes.m.crossplane.io/v1alpha1 #--Namespaced endpoint. Allows namespaced MR creation +kind: ClusterProviderConfig #--ClusterWide, one provider for all tenants +metadata: + name: kubernetes +spec: + credentials: #--Retrieved from ClusterAuth writeConnectionSecretToRef + source: Secret + secretRef: + namespace: crossplane-system + name: tfc-cluster12-connection #--Secret name + key: kubeconfig #--key name inside Secret +``` + +## Create A Managed Resource on the Remote EKS Cluster + +With this all configured, we can finally create objects on the remote *EKS Cluster* by linking an object to our Kubernetes *Provider* as shown below: + +```yaml +apiVersion: kubernetes.m.crossplane.io/v1alpha1 #--Namespaced MR +kind: Object +metadata: + name: tfc-example-secret +spec: + forProvider: + manifest: + #--Begin Kubernetes object manifest + apiVersion: v1 + kind: Secret + metadata: + name: tfc-example-secret + namespace: cluster12-tenant1 + data: + secret_information: c29tZXJ1YmJpc2gK + #--End kubernetes object manifest + providerConfigRef: + name: kubernetes #--Created above, authed to remote cluster + kind: ClusterProviderConfig +``` diff --git a/assets/2025-11-05-crossplane-v2-creating-kubernetes-resources/01.png b/assets/2025-11-05-crossplane-v2-creating-kubernetes-resources/01.png new file mode 100644 index 0000000..94280b8 Binary files /dev/null and b/assets/2025-11-05-crossplane-v2-creating-kubernetes-resources/01.png differ diff --git a/assets/2025-11-06-crossplane-v2-aws-ecr-integration/01.png b/assets/2025-11-06-crossplane-v2-aws-ecr-integration/01.png new file mode 100644 index 0000000..94280b8 Binary files /dev/null and b/assets/2025-11-06-crossplane-v2-aws-ecr-integration/01.png differ diff --git a/assets/2025-11-06-crossplane-v2-aws-ecr-integration/02.png b/assets/2025-11-06-crossplane-v2-aws-ecr-integration/02.png new file mode 100644 index 0000000..dbf2283 Binary files /dev/null and b/assets/2025-11-06-crossplane-v2-aws-ecr-integration/02.png differ diff --git a/assets/2025-11-16-crossplane-v2-connecting-to-a-remote-eks-cluster/01.png b/assets/2025-11-16-crossplane-v2-connecting-to-a-remote-eks-cluster/01.png new file mode 100644 index 0000000..94280b8 Binary files /dev/null and b/assets/2025-11-16-crossplane-v2-connecting-to-a-remote-eks-cluster/01.png differ diff --git a/assets/2025-11-16-crossplane-v2-connecting-to-a-remote-eks-cluster/02.png b/assets/2025-11-16-crossplane-v2-connecting-to-a-remote-eks-cluster/02.png new file mode 100644 index 0000000..0dccd2e Binary files /dev/null and b/assets/2025-11-16-crossplane-v2-connecting-to-a-remote-eks-cluster/02.png differ