WIP feat: Implement Required Resources

README.md

@@ -529,16 +529,165 @@ spec:
     }
 ```
 
-### Extra resources
-By defining one or more special `ExtraResources`, you can ask Crossplane to retrieve additional resources from the local cluster 
+### Required resources
+
+By defining one or more "required resources", you can ask Crossplane to retrieve additional resources from the local cluster 
 and make them available to your templates. 
-See the [docs](https://github.com/crossplane/crossplane/blob/main/design/design-doc-composition-functions-extra-resources.md) for more information.
+See the [docs](https://docs.crossplane.io/latest/composition/compositions/#required-resources) for more information.
+
+This feature only works with Crossplane v2. Crossplane v1 must use [Extra Resources](#extra-resources), described in the section below.
+
+There are two ways to request required resources:
+
+One, you can list the resources to retrieve in the `requirements.requiredResources` field
+of the pipeline step:
+
+```yaml
+apiVersion: apiextensions.crossplane.io/v1
+kind: Composition
+metadata:
+  name: example
+spec:
+  compositeTypeRef:
+    apiVersion: example.crossplane.io/v1beta1
+    kind: XR
+  mode: Pipeline
+  pipeline:
+    - step: basic
+      functionRef:
+        name: function-kcl
+        requirements:
+          requiredResources:
+          - requirementName: foo
+            apiVersion: example.com/v1beta1
+            kind: Foo
+            matchLabels:
+              foo: bar
+          - requirementName: bar
+            apiVersion: example.com/v1beta1
+            kind: Bar
+            name: my-bar
+          - requirementName: baz
+            apiVersion: example.m.com/v1beta1
+            kind: Bar
+            name: my-bar
+            namespace: my-baz-ns
+          - requirementName: quux
+            apiVersion: example.m.com/v1beta1
+            kind: Quux
+            matchLabels:
+              baz: quux
+            namespace: my-quux-ns
+      input:
+        apiVersion: krm.kcl.dev/v1alpha1
+        kind: KCLInput
+        spec:
+          source: "..."
+```
+
+Two, the composition can dynamically request resources by returning a special
+`RequiredResources` item:
+
+```yaml
+apiVersion: krm.kcl.dev/v1alpha1
+kind: KCLInput
+spec:
+  source: |
+    # Omit other logic
+    details = {
+        apiVersion: "meta.krm.kcl.dev/v1alpha1"
+        kind: "RequiredResources"
+        requirements = {
+            foo = {
+                apiVersion: "example.com/v1beta1",
+                kind: "Foo",
+                matchLabels: {
+                    "foo": "bar"
+                }
+            },
+            bar = {
+                apiVersion: "example.com/v1beta1",
+                kind: "Bar",
+                name: "my-bar"
+            },
+            baz = {
+                apiVersion: "example.m.com/v1beta1",
+                kind: "Bar",
+                name: "my-bar"
+                namespace: "my-baz-ns"
+            },
+            quux = {
+                apiVersion: "example.m.com/v1beta1",
+                kind: "Quux",
+                matchLabels: {
+                    "baz": "quux"
+                }
+                namespace: "my-quux-ns"
+            }
+        }
+    }
+
+    # Omit other composite logics.
+    items = [
+        details
+        # Omit other return resources.
+    ]
+```
+
+Either way will result in Crossplane retrieving the requested resources and making them available with the following format:
+
+```yaml
+foo:
+- Resource:
+    apiVersion: example.com/v1beta1
+    kind: Foo
+    metadata:
+      labels:
+        foo: bar
+    # Omitted for brevity
+- Resource:
+    apiVersion: example.com/v1beta1
+    kind: Foo
+    metadata:
+      labels:
+        foo: bar
+    # Omit for brevity
+bar:
+- Resource:
+    apiVersion: example.com/v1beta1
+    kind: Bar
+    metadata:
+      name: my-bar
+    # Omitted for brevity
+```
+
+You can access the retrieved resources in your code like this:
 
 > [!NOTE]
-> With ExtraResources, you can fetch cluster-scoped resources, but not namespaced resources such as claims.
-> If you need to get a composite resource via its claim name you can use `matchLabels` with `crossplane.io/claim-name: <claimname>`.
-> Namespace scoped resources can be queried with the `matchNamespace` field. 
-> Leaving the `matchNamespace` field empty or not defining it will query a cluster scoped resource.
+> Crossplane performs an additional reconciliation pass for dynamic required resources.
+> Consequently, during the initial execution, these resources might not be present.
+> It is essential to implement checks to handle this scenario.
+
+```yaml
+apiVersion: krm.kcl.dev/v1alpha1
+kind: KCLInput
+spec:
+  source: |
+    er = option("params")?.requiredResources
+
+    if er?.bar:
+      name = er?.bar[0]?.Resource?.metadata?.name or ""
+    # Omit other logic
+```
+
+### Extra resources
+
+Extra resources are Crossplane v1's mechanism for retrieving additional resources from the local cluster.
+It is deprecated in Crossplane v2.
+
+Unlike required resources, there is no mechanism for requesting extra resources in the
+pipeline step definition. They can only be requested dynamically, by returning a special
+`ExtraResources` item:
 
 ```yaml
 apiVersion: krm.kcl.dev/v1alpha1
@@ -587,7 +736,9 @@ spec:
 ```
 You can retrieve the extra resources either via labels with `matchLabels` or via name with `matchName: somename`.
 
-This will result in Crossplane receiving the requested resources and make them available with the following format.
+See the [docs](https://github.com/crossplane/crossplane/blob/main/design/design-doc-composition-functions-extra-resources.md) for more information.
+
+This will result in Crossplane receiving the requested resources and making them available using the following format:
 
 ```yaml
 foo:
@@ -618,7 +769,7 @@ You can access the retrieved resources in your code like this:
 
 > [!NOTE]
 > Crossplane performs an additional reconciliation pass for extra resources.
-> Consequently, during the initial execution, these resources may be uninitialized.
+> Consequently, during the initial execution, these resources might not be present.
 > It is essential to implement checks to handle this scenario.
 
 ```yaml

examples/default/required_resources/Makefile

@@ -0,0 +1,2 @@
+run:
+	crossplane render --verbose xr.yaml composition.yaml functions.yaml -r --required-resources required_resources.yaml

examples/default/required_resources/README.md

@@ -0,0 +1,62 @@
+# Example Manifests
+
+You can run your function locally and test it using `crossplane render`
+with these example manifests.
+
+```shell
+# Run the function locally
+$ go run . --insecure --debug
+```
+
+```shell
+# Then, in another terminal, call it with these example manifests
+$ crossplane render --verbose xr.yaml composition.yaml functions.yaml -r --required-resources required_resources.yaml
+---
+---
+apiVersion: example.crossplane.io/v1beta1
+kind: XR
+metadata:
+  name: example
+status:
+  conditions:
+  - lastTransitionTime: "2024-01-01T00:00:00Z"
+    message: 'Unready resources: another-awesome-dev-bucket, my-awesome-dev-bucket'
+    reason: Creating
+    status: "False"
+    type: Ready
+---
+apiVersion: example/v1alpha1
+kind: Foo
+metadata:
+  annotations:
+    crossplane.io/composition-resource-name: another-awesome-dev-bucket
+  generateName: example-
+  labels:
+    crossplane.io/composite: example
+  name: another-awesome-dev-bucket
+  ownerReferences:
+  - apiVersion: example.crossplane.io/v1beta1
+    blockOwnerDeletion: true
+    controller: true
+    kind: XR
+    name: example
+    uid: ""
+---
+apiVersion: example/v1alpha1
+kind: Foo
+metadata:
+  annotations:
+    crossplane.io/composition-resource-name: my-awesome-dev-bucket
+  generateName: example-
+  labels:
+    crossplane.io/composite: example
+  name: my-awesome-dev-bucket
+  ownerReferences:
+  - apiVersion: example.crossplane.io/v1beta1
+    blockOwnerDeletion: true
+    controller: true
+    kind: XR
+    name: example
+    uid: ""
+
+```

examples/default/required_resources/composition.yaml

@@ -0,0 +1,53 @@
+apiVersion: apiextensions.crossplane.io/v1
+kind: Composition
+metadata:
+  name: kcl-function
+spec:
+  compositeTypeRef:
+    apiVersion: example.crossplane.io/v1beta1
+    kind: XR
+  mode: Pipeline
+  pipeline:
+    - step: normal
+      functionRef:
+        name: kcl-function
+      input:
+        apiVersion: krm.kcl.dev/v1alpha1
+        kind: KCLInput
+        metadata:
+          annotations:
+            "krm.kcl.dev/default_ready": "True"
+        spec:
+          source: |
+            oxr = option("params").oxr
+            rr = option("params")?.requiredResources
+
+            foo = [{
+              apiVersion: "example/v1alpha1"
+              kind: "Foo"
+              metadata = {
+                name: k.Resource.metadata.name
+              }
+            } for k in rr?.bucket] if rr?.bucket else []
+
+            dxr = {
+                **oxr
+            }
+
+            details = {
+                apiVersion: "meta.krm.kcl.dev/v1alpha1"
+                kind: "RequiredResources"
+                requirements = {
+                    bucket = {
+                        apiVersion: "s3.aws.upbound.io/v1beta1",
+                        kind: "Bucket",
+                        matchLabels: {
+                            "foo": "bar"
+                        }
+                    }
+                }
+            }
+            items = [
+                details
+                dxr
+            ] + foo

examples/default/required_resources/functions.yaml

@@ -0,0 +1,9 @@
+apiVersion: pkg.crossplane.io/v1beta1
+kind: Function
+metadata:
+  name: kcl-function
+  annotations:
+    # This tells crossplane render to connect to the function locally.
+    render.crossplane.io/runtime: Development
+spec:
+  package: xpkg.upbound.io/crossplane-contrib/function-kcl:latest

examples/default/required_resources/required_resources.yaml

@@ -0,0 +1,29 @@
+apiVersion: s3.aws.upbound.io/v1beta1
+kind: Bucket
+metadata:
+  annotations:
+    crossplane.io/external-name: my-awesome-dev-bucket
+  labels:
+    foo: bar
+  name: my-awesome-dev-bucket
+spec:
+  forProvider:
+    region: us-west-1
+status:
+  atProvider:
+    id: random-bucket-id
+---
+apiVersion: s3.aws.upbound.io/v1beta1
+kind: Bucket
+metadata:
+  annotations:
+    crossplane.io/external-name: my-awesome-dev-bucket
+  labels:
+    foo: bar
+  name: another-awesome-dev-bucket
+spec:
+  forProvider:
+    region: us-west-1
+status:
+  atProvider:
+    id: random-bucket-id

examples/default/required_resources/xr.yaml

@@ -0,0 +1,6 @@
+apiVersion: example.crossplane.io/v1beta1
+kind: XR
+metadata:
+  name: example
+spec:
+  count: 1

examples/default/required_resources_namespaced/Makefile

@@ -0,0 +1,2 @@
+run:
+	crossplane render --verbose xr.yaml composition.yaml functions.yaml -r --required-resources required_resources_namespaced.yaml