From bb507baee6c7ad002abb8ddefb03d5c378329450 Mon Sep 17 00:00:00 2001 From: Itay Grudev Date: Thu, 18 Dec 2025 19:41:54 +0200 Subject: [PATCH 1/3] docs( cluster ): Kyverno Chainsaw testing documentation Signed-off-by: Itay Grudev --- charts/cluster/TESTING.md | 104 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 104 insertions(+) create mode 100644 charts/cluster/TESTING.md diff --git a/charts/cluster/TESTING.md b/charts/cluster/TESTING.md new file mode 100644 index 0000000000..ff4b5997a2 --- /dev/null +++ b/charts/cluster/TESTING.md @@ -0,0 +1,104 @@ +# Testing + +This chart uses Kyverno Chainsaw and implements end-to-end (E2E) tests for common features. Not everything is tested because of inadequate tooling — for example, local simulation of Azure and Google Cloud Storage. We do test S3 via MinIO. Our aim is that every critical feature that is technically feasible to test is covered. + +We use a local kind cluster (minikube also works) and provision prerequisites such as the CloudNativePG operator, Prometheus CRDs, and MinIO. Then we run the `chainsaw` utility, which executes the individual tests. It can run tests in parallel, which is essential because some tests take over five minutes to complete. + +## Procedure + +1. Create a kind cluster. + + ```bash + kind delete cluster kind + ``` + +2. Install the CloudNativePG operator + + ```bash + helm dependency update charts/cloudnative-pg + helm upgrade \ + --install \ + --namespace $NAMESPACE \ + --create-namespace \ + --set config.clusterWide=$CLUSTER_WIDE \ + --wait \ + cnpg charts/cloudnative-pg + ``` + +3. Install the Prometheus CRDs (optional, but required for monitoring tests) + + ```bash + helm repo add prometheus-community https://prometheus-community.github.io/helm-charts + helm install prometheus-crds prometheus-community/prometheus-operator-crds + ``` + +4. Install MinIO (optional, but required for backup/recovery tests). + + ```bash + helm repo add minio-operator https://operator.min.io + helm upgrade \ + --install \ + --namespace minio-system \ + --create-namespace \ + --wait \ + operator minio-operator/operator + + helm upgrade \ + --install \ + --namespace minio \ + --create-namespace \ + --wait \ + --values ./.github/minio.yaml \ + tenant minio-operator/tenant + ``` + +5. Install Kyverno Chainsaw + + Refer to the [Kyverno Chainsaw Installation](https://kyverno.io/blog/2023/12/12/kyverno-chainsaw-the-ultimate-end-to-end-testing-tool/#install-chainsaw) documentation for platform specific instructions. + + You can also install Kyverno Chainsaw from source if you have _Go_ installed: + + ```bash + go install github.com/kyverno/chainsaw@latest + ``` + +6. Run the tests + + To run the whole test suite: + + ```bash + chainsaw test charts/cluster + ``` + + To run a specific test, specify its directory path. Example: + + ```bash + chainsaw test charts/cluster/test/postgresql-cluster-configuration + ``` + +## Test structure + +We are only going to outline the test structure here. Refer to the [Kyverno Chainsaw](https://kyverno.github.io/chainsaw/latest/quick-start/) documentation for full reference and capabilities. +The tests are located in the `test` directory. Each test has its own subdirectory. Because clusters take time to provision, where it makes sense tests should be combined and executed sequentially on the same cluster. One exception is critical functionality such as backup/restore, which should be tested independently. + +Inside each test there is a `chainsaw-test.yaml` file that outlines the steps of that particular test. Here are some tips for writing tests for CloudNativePG: + +* Whenever you're adding new features, make sure at the very least to update the `postgresql-cluster-configuration` test that verifies that all non-default configuration options are passed and applied correctly. We're unlikely to merge PRs that don't have their own test or update this one. +* Always manually uninstall Helm chart resources to speed up cleanup. +* Where applicable, and for steps likely to fail, add `catch` statements. For example: + + ```yaml + catch: + - describe: + apiVersion: batch/v1 + kind: Job + - describe: + apiVersion: postgresql.cnpg.io/v1 + kind: Cluster + - podLogs: + selector: batch.kubernetes.io/job-name=your-job-name + ``` + + This will substantially help with debugging later. +* Provide useful step descriptions to aid in understanding test failures. +* Use reasonable test timeouts so tests fail if something isn't finished after 5–10 minutes. Aim for tests to complete within 10 minutes. From ebb18a823b21ac0d6afd4d5dfeafe46b04ffcc63 Mon Sep 17 00:00:00 2001 From: Itay Grudev Date: Fri, 19 Dec 2025 09:54:52 +0000 Subject: [PATCH 2/3] Fixed errata kind create cluster --- charts/cluster/TESTING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/charts/cluster/TESTING.md b/charts/cluster/TESTING.md index ff4b5997a2..28102c8151 100644 --- a/charts/cluster/TESTING.md +++ b/charts/cluster/TESTING.md @@ -9,7 +9,7 @@ We use a local kind cluster (minikube also works) and provision prerequisites su 1. Create a kind cluster. ```bash - kind delete cluster kind + kind delete cluster ``` 2. Install the CloudNativePG operator From c19c2baebe8a24c5a66ed67b331b8d915232ef3b Mon Sep 17 00:00:00 2001 From: Itay Grudev Date: Fri, 19 Dec 2025 09:56:06 +0000 Subject: [PATCH 3/3] Update TESTING.md --- charts/cluster/TESTING.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/charts/cluster/TESTING.md b/charts/cluster/TESTING.md index 28102c8151..8fd12f4607 100644 --- a/charts/cluster/TESTING.md +++ b/charts/cluster/TESTING.md @@ -9,7 +9,7 @@ We use a local kind cluster (minikube also works) and provision prerequisites su 1. Create a kind cluster. ```bash - kind delete cluster + kind create cluster ``` 2. Install the CloudNativePG operator