From 52c1ad8ad1aff73ab388360ea6c2e8864cc37914 Mon Sep 17 00:00:00 2001 From: Anastasia Alexadrova Date: Tue, 3 Mar 2026 21:05:37 +0100 Subject: [PATCH 01/13] K8SPG-911 Documented TDE --- docs/assets/fragments/vault-enable-kv.txt | 41 +++ .../fragments/vault-generate-tls-certs.txt | 158 +++++++++ docs/assets/fragments/vault-install-tls.txt | 197 +++++++++++ docs/encryption-disable.md | 54 +++ docs/encryption-setup.md | 322 ++++++++++++++++++ docs/encryption.md | 44 +++ docs/operator.md | 111 +++++- mkdocs-base.yml | 3 + 8 files changed, 929 insertions(+), 1 deletion(-) create mode 100644 docs/assets/fragments/vault-enable-kv.txt create mode 100644 docs/assets/fragments/vault-generate-tls-certs.txt create mode 100644 docs/assets/fragments/vault-install-tls.txt create mode 100644 docs/encryption-disable.md create mode 100644 docs/encryption-setup.md create mode 100644 docs/encryption.md diff --git a/docs/assets/fragments/vault-enable-kv.txt b/docs/assets/fragments/vault-enable-kv.txt new file mode 100644 index 00000000..2877bf80 --- /dev/null +++ b/docs/assets/fragments/vault-enable-kv.txt @@ -0,0 +1,41 @@ +At this step you need to configure Vault and enable secrets engine within it. To do so you must first authenticate in Vault. + +When you started Vault, it generates and starts with a [root token :octicons-link-external-16:](https://developer.hashicorp.com/vault/docs/concepts/tokens) that provides full access to Vault. Use this token to authenticate. + +Run the following command on a leader node. The remaining ones will synchronize from the leader. + +1. Extract the Vault root token from the file where you saved the init response output: + + ```bash + cat ${WORKDIR}/vault-init | jq -r ".root_token" + ``` + + ??? example "Sample output" + + ```{.text .no-copy} + hvs.*************Jg9r + ``` + +2. Connect to Vault Pod: + + ```bash + kubectl exec -it vault-0 -n $NAMESPACE -- /bin/sh + ``` + +3. Authenticate in Vault with this token: + + ```bash + vault login hvs.*************Jg9r + ``` + +4. Enable the secrets engine at the mount path. The following command enables KV secrets engine v2 at the `tde` mount-path: + + ```bash + vault secrets enable --version=2 -path=tde kv + ``` + + ??? example "Sample output" + + ```{.text .no-copy} + Success! Enabled the kv secrets engine at: tde/ + ``` \ No newline at end of file diff --git a/docs/assets/fragments/vault-generate-tls-certs.txt b/docs/assets/fragments/vault-generate-tls-certs.txt new file mode 100644 index 00000000..a58576f6 --- /dev/null +++ b/docs/assets/fragments/vault-generate-tls-certs.txt @@ -0,0 +1,158 @@ +## Generate TLS certificates + +To use TLS, you'll need the following certificates: + +* A private key for the Vault server +* A certificate for the Vault server signed by the Kubernetes CA +* The Kubernetes CA certificate + +These files store sensitive information. Make sure to keep them in a secure location. + +### Generate the private key + +Generate a private key for the Vault server: + +```bash +openssl genrsa -out ${WORKDIR}/vault.key 2048 +``` + +### Create the Certificate Signing Request (CSR) + +A Certificate Signing Request (CSR) is a file that contains information about your server and the certificate you need. You create it using your private key, and then submit it to Kubernetes to get a certificate signed by the Kubernetes Certificate Authority (CA). The signed certificate proves your server's identity and enables secure TLS connections. + +1. Create the Certificate Signing Request configuration file: + + Specify the certificate details that Kubernetes needs to sign your certificate: + + * **Request settings** (`[req]`): References the sections for certificate extensions and distinguished name. The distinguished name section is left empty as Kubernetes will populate it automatically. + * **Certificate extensions** (`[v3_req]`): Defines how the certificate can be used. `serverAuth` allows the certificate for server authentication, while `keyUsage` specifies the cryptographic operations the certificate supports (non-repudiation, digital signature, and key encipherment). + * **Subject Alternative Names** (`[alt_names]`): Lists all DNS names and IP addresses where your Vault service can be accessed. This includes the service name, fully qualified domain names (FQDNs) for different Kubernetes DNS contexts (namespace-scoped, cluster-scoped with `.svc`, and fully qualified with `.svc.cluster.local`), and the localhost IP address. + + ```bash + cat > "${WORKDIR}/csr.conf" <<'EOF' + [req] + default_bits = 2048 + prompt = no + encrypt_key = yes + default_md = sha256 + distinguished_name = kubelet_serving + req_extensions = v3_req + [ kubelet_serving ] + O = system:nodes + CN = system:node:*.vault.svc.cluster.local + [ v3_req ] + basicConstraints = CA:FALSE + keyUsage = nonRepudiation, digitalSignature, keyEncipherment, dataEncipherment + extendedKeyUsage = serverAuth, clientAuth + subjectAltName = @alt_names + [alt_names] + DNS.1 =*.vault-internal + DNS.2 = *.vault-standby + DNS.3 =*.vault-internal.vault.svc.cluster.local + DNS.4 = *.vault-standby.vault.svc.cluster.local + DNS.5 =*.vault + DNS.6 = vault.vault.svc.cluster.local + IP.1 = 127.0.0.1 + EOF + ``` + +2. Generate the CSR. The following command creates the Certificate Signing Request file using your private key and the configuration file. + + The `-subj` parameter specifies the distinguished name directly: the Common Name (CN) identifies your Vault service using the Kubernetes node naming convention (`system:node:${SERVICE}.${NAMESPACE}.svc`), and the Organization (O) field is set to `system:nodes`, which Kubernetes requires to recognize and sign the certificate. The command combines these subject fields with the certificate extensions defined in the configuration file to produce the complete CSR. + + ```bash + openssl req -new -key $WORKDIR/vault.key \ + -subj "/CN=system:node:${SERVICE}.${NAMESPACE}.svc;/O=system:nodes" \ + -out $WORKDIR/server.csr -config $WORKDIR/csr.conf + ``` + +### Issue the certificate + +To get your certificate signed by Kubernetes, you need to submit the CSR through the Kubernetes API. The CSR file you generated with OpenSSL must be wrapped in a Kubernetes CertificateSigningRequest resource. + +1. Create the CSR YAML file to send it to Kubernetes: + + This YAML file creates a Kubernetes CertificateSigningRequest object that contains your CSR. The file embeds the base64-encoded CSR content and specifies: + + * The signer name (`kubernetes.io/kubelet-serving`) that tells Kubernetes which CA should sign the certificate + * The groups field (`system:authenticated`) that identifies who can approve this CSR + * The certificate usages that define how the certificate can be used (digital signature, key encipherment, and server authentication) + + ```bash + cat > $WORKDIR/csr.yaml < Approved,Issued + ``` + +### Retrieve the certificates + +After Kubernetes approves and signs your CSR, you need to retrieve the signed certificate and the Kubernetes CA certificate. These certificates are required to configure TLS for your Vault server. + +1. Retrieve the signed certificate from the CertificateSigningRequest object. The certificate is base64-encoded in Kubernetes, so you decode it and save it to a file. + + ```bash + kubectl get csr ${CSR_NAME} -o jsonpath='{.status.certificate}' | base64 -d > $WORKDIR/vault.crt + ``` + +2. Retrieve Kubernetes CA certificate: + + This command retrieves the Kubernetes cluster's Certificate Authority (CA) certificate from your `kubeconfig` file. The CA certificate is needed to verify that the signed certificate is valid and was issued by the Kubernetes CA. The command uses `kubectl config view` with flags to get the raw, flattened configuration and extract the CA certificate data, which is also base64-encoded. + + ```bash + kubectl config view \ + --raw \ + --minify \ + --flatten \ + -o jsonpath='{.clusters[].cluster.certificate-authority-data}' \ + | base64 -d > ${WORKDIR}/vault.ca + ``` + +### Store certificates in Kubernetes secrets + +Create a TLS secret in Kubernetes to store the certificates and key: + +```bash +kubectl create secret generic ${SECRET_NAME_VAULT} \ + --namespace ${NAMESPACE} \ + --from-file=vault.key=$WORKDIR/vault.key \ + --from-file=vault.crt=$WORKDIR/vault.crt \ + --from-file=vault.ca=$WORKDIR/vault.ca +``` + diff --git a/docs/assets/fragments/vault-install-tls.txt b/docs/assets/fragments/vault-install-tls.txt new file mode 100644 index 00000000..44891061 --- /dev/null +++ b/docs/assets/fragments/vault-install-tls.txt @@ -0,0 +1,197 @@ +## Install Vault with TLS + +For this setup, we install Vault in Kubernetes using the [Helm 3 package manager :octicons-link-external-16:](https://helm.sh/) in High Availability (HA) mode with Raft storage backend and with TLS enabled. + +1. Add and update the Vault Helm repository: + + ```bash + helm repo add hashicorp https://helm.releases.hashicorp.com + helm repo update + ``` + +2. Install Vault with TLS enabled: + + ```bash + helm upgrade --install ${SERVICE} hashicorp/vault \ + --disable-openapi-validation \ + --version ${VAULT_HELM_VERSION} \ + --namespace ${NAMESPACE} \ + --set "global.enabled=true" \ + --set "global.tlsDisable=false" \ + --set "global.platform=kubernetes" \ + --set server.extraEnvironmentVars.VAULT_CACERT=/vault/userconfig/${SECRET_NAME_VAULT}/vault.ca \ + --set "server.extraEnvironmentVars.VAULT_TLSCERT=/vault/userconfig/${SECRET_NAME_VAULT}/vault.crt" \ + --set "server.extraEnvironmentVars.VAULT_TLSKEY=/vault/userconfig/${SECRET_NAME_VAULT}/vault.key" \ + --set "server.volumes[0].name=userconfig-${SECRET_NAME_VAULT}" \ + --set "server.volumes[0].secret.secretName=${SECRET_NAME_VAULT}" \ + --set "server.volumes[0].secret.defaultMode=420" \ + --set "server.volumeMounts[0].mountPath=/vault/userconfig/${SECRET_NAME_VAULT}" \ + --set "server.volumeMounts[0].name=userconfig-${SECRET_NAME_VAULT}" \ + --set "server.volumeMounts[0].readOnly=true" \ + --set "server.ha.enabled=true" \ + --set "server.ha.replicas=3" \ + --set "server.ha.raft.enabled=true" \ + --set "server.ha.raft.setNodeId=true" \ + --set-string "server.ha.raft.config=cluster_name = \"vault-integrated-storage\" + ui = true + listener \"tcp\" { + tls_disable = 0 + address = \"[::]:8200\" + cluster_address = \"[::]:8201\" + tls_cert_file = \"/vault/userconfig/${SECRET_NAME_VAULT}/vault.crt\" + tls_key_file = \"/vault/userconfig/${SECRET_NAME_VAULT}/vault.key\" + tls_client_ca_file = \"/vault/userconfig/${SECRET_NAME_VAULT}/vault.ca\" + } + storage \"raft\" { + path = \"/vault/data\" + } + disable_mlock = true + service_registration \"kubernetes\" {}" + ``` + + This command does the following: + + * Installs HashiCorp Vault in High Availability (HA) mode with secure TLS enabled in your Kubernetes cluster. + * Configures Vault pods to use certificates from a Kubernetes Secret via volume mounts for secure HTTPS communication between Vault and clients. + * Sets up Raft as the backend storage with three replicas for fault tolerance, and configures the Vault TCP listener to enforce TLS with your specified certificate files. + + + ??? example "Sample output" + + ```{.text .no-copy} + NAME: vault + LAST DEPLOYED: Wed Aug 20 12:55:38 2025 + NAMESPACE: vault + STATUS: deployed + REVISION: 1 + NOTES: + Thank you for installing HashiCorp Vault! + + Now that you have deployed Vault, you should look over the docs on using + Vault with Kubernetes available here: + + https://developer.hashicorp.com/vault/docs + ``` + +4. Retrieve the Pod name where Vault is running: + + ```bash + kubectl -n $NAMESPACE get pod -l app.kubernetes.io/name=${SERVICE} -o jsonpath='{range .items[*]}{.metadata.name}{"\n"}{end}' + ``` + + ??? example "Sample output" + + ```{.text .no-copy} + vault-0 + vault-1 + vault-2 + ``` + +## Initialize and unseal Vault + +1. After Vault is installed, you need to initialize it. Run the following command to initialize the first pod: + + ```bash + kubectl exec -it pod/vault-0 -n $NAMESPACE -- vault operator init -key-shares=1 -key-threshold=1 -format=json > ${WORKDIR}/vault-init + ``` + + The command does the following: + + * Connects to the Vault Pod + * Initializes Vault server with TLS enabled + * Creates 1 unseal key share which is required to unseal the server + * Outputs the init response to a local file. The file includes unseal keys and root token. + +2. Vault is started in a sealed state. In this state Vault can access the storage but it cannot decrypt data. In order to use Vault, you need to unseal it. + + Retrieve the unseal key from the file: + + ```bash + unsealKey=$(jq -r ".unseal_keys_b64[]" < ${WORKDIR}/vault-init) + ``` + + Now, unseal Vault. Run the following command: + + ```bash + kubectl exec -it pod/vault-0 -n $NAMESPACE -- vault operator unseal "$unsealKey" + ``` + + ??? example "Sample output" + + ```{.text .no-copy} + Key Value + --- ----- + Seal Type shamir + Initialized true + Sealed false + Total Shares 1 + Threshold 1 + Version 1.19.0 + Build Date 2025-03-04T12:36:40Z + Storage Type raft + Cluster Name vault-integrated-storage + Cluster ID ed275c91-e227-681b-5aaa-f7a9fc19e37e + Removed From Cluster false + HA Enabled true + HA Cluster + HA Mode active + Active Since 2025-12-15T13:36:42.542059059Z + Raft Committed Index 37 + Raft Applied Index 37 + ``` + +3. Add the remaining Pods to the Vault cluster. If you have another secret name, replace the `vault-secret` with your value in the following for loop: + + ```bash + for POD in vault-1 vault-2; do + kubectl -n "$NAMESPACE" exec $POD -- sh -c ' + vault operator raft join -address=https://${HOSTNAME}.vault-internal:8200 \ + -leader-ca-cert="$(cat /vault/userconfig/vault-secret/vault.ca)" \ + -leader-client-cert="$(cat /vault/userconfig/vault-secret/vault.crt)" \ + -leader-client-key="$(cat /vault/userconfig/vault-secret/vault.key)" \ + https://vault-0.vault-internal:8200; + ' + done + ``` + + The command connects to each Vault Pod (`vault-1` and `vault-2`) and issues the `vault operator raft join` command, which: + * Joins the Pods to the Vault Raft cluster, enabling HA mode. + * Uses the necessary TLS certificates to securely connect to the cluster leader (`vault-0`). + * Ensures all nodes participate in the Raft consensus and share storage responsibilities. + + ??? example "Sample output" + + ```{.text .no-copy} + Key Value + --- ----- + Joined true + ``` + +4. Unseal the remaining Pods. Use this for loop: + + ```bash + for POD in vault-1 vault-2; do + kubectl -n "$NAMESPACE" exec $POD -- sh -c " + vault operator unseal \"$unsealKey\" + " + done + ``` + + ??? example "Expected output" + + ```{.text .no-value} + Key Value + --- ----- + Seal Type shamir + Initialized true + Sealed true + Total Shares 1 + Threshold 1 + Unseal Progress 0/1 + Unseal Nonce n/a + Version 1.19.0 + Build Date 2025-03-04T12:36:40Z + Storage Type raft + Removed From Cluster false + HA Enabled true + ``` diff --git a/docs/encryption-disable.md b/docs/encryption-disable.md new file mode 100644 index 00000000..a0d5226d --- /dev/null +++ b/docs/encryption-disable.md @@ -0,0 +1,54 @@ +# Disable encryption + +Disabling `pg_tde` (Transparent Data Encryption) is generally not recommended, as it removes an important layer of security that protects your data at rest. However, if you must disable encryption, this guide walks you through the steps. + +!!! important + + To properly disable encryption in the Operator, you must follow a specific sequence and modify your Custom Resource (CR) twice. Attempting to disable everything in a single step will not work: the Operator needs to drop the `pg_tde` extension before you remove the Vault (key provider) configuration. + + Failing to follow the steps in this tutorial in order will result in errors, because removing the Vault configuration before the extension is dropped prevents the Operator from cleaning up properly. + +1. Unencrypt **all encrypted databases** in your database. Connect to the primary database Pod as the `postgres` user, connect to each encrypted database and run the following command to unencrypt **every** encrypted table + + ```sql + ALTER TABLE SET ACCESS METHOD heap; + ``` + + Exit the Pod. + +2. Edit the Custom Resource and set the `extensions.pg_tde.enabled` option to `false`. + + ```yaml + spec: + extensions: + pg_tde: + enabled: false + ``` + +3. Apply the changes: + + ```bash + kubectl apply -f deploy/cr.yaml -n $NAMESPACE + ``` + + This command triggers the rolling restart of your database Pods. As a result, the Operator runs `DROP EXTENSION pg_tde` in all databases. + +4. Run the `CHECKPOINT` command in PostgreSQL. It forces an immediate checkpoint to flush all dirty pages to disk and update all datafiles and indexes. Connect to the Connect to the primary database Pod as the `postgres` user and run: + + ```sql + CHECKPOINT; + ``` + + This flushes data to disk in all databases. + + Exit the Pod. + +5. Update the Custom Resource again and remove all vault-related configuration from `extensions.pg_tde` section. +6. Apply the changes: + + ```bash + kubectl apply -f deploy/cr.yaml -n $NAMESPACE + ``` + + This triggers another rolling restart of the database Pods. + diff --git a/docs/encryption-setup.md b/docs/encryption-setup.md new file mode 100644 index 00000000..b6d3cd23 --- /dev/null +++ b/docs/encryption-setup.md @@ -0,0 +1,322 @@ +# Configure data-at-rest encryption with HashiCorp Vault + +This document guides you through setting up data-at-rest encryption with `pg_tde` and HashiCorp Vault as the key provider. To learn more about data-at-rest encryption and how it works, see [Data-at-rest encryption](encryption.md). + +## Assumptions + +1. This guide is provided as a best effort and builds upon procedures described in the official Vault documentation. Since Vault's setup steps may change in future releases, this document may become outdated; we cannot guarantee ongoing accuracy or responsibility for such changes. For the most up-to-date and reliable information, please always refer to [the official Vault documentation](https://developer.hashicorp.com/vault/tutorials/kubernetes/kubernetes-minikube-tls#kubernetes-minikube-tls). +2. In the following sections we deploy the Vault server in High Availability (HA) mode on Kubernetes via Helm with TLS enabled. The HA setup uses Raft storage backend and consists of 3 replicas for redundancy. Using Helm is not mandatory. Any supported Vault deployment (on-premises, in the cloud, or a managed Vault service) works as long as the Operator can reach it. +3. This guide uses Vault Helm chart version 0.30.0. You may want to change it to the required version by setting the `VAULT_HELM_VERSION` variable. + +## Prerequisites + +To configure data-at-rest encryptio, you need the following: + +* `kubectl`- Kubernetes command-line interface +* `helm` - Helm package manager +* `jq` - JSON processor +* The Operator and Percona Distribution for PostgreSQL installed. + +## Prepare your environment + +1. Create the namespaces for Vault and the database cluster. If you have installed the Operator and Percona Distribution for PostgreSQL, you don't need to create a namespace for them. + + * For Vault server: + + ```bash + kubectl create namespace vault + ``` + + * For Percona Distribution for PostgreSQL cluster: + + ```bash + kubectl create namespace pg + ``` + +2. Export the namespaces and other variables as environment variables to simplify further configuration: + + ```bash + export NAMESPACE="vault" + export CLUSTER_NAMESPACE="pg" + export VAULT_HELM_VERSION="0.30.0" + export SERVICE="vault" + export CSR_NAME="vault-csr" + export SECRET_NAME_VAULT="vault-secret" + export POLICY_NAME="tde-policy" + export WORKDIR="/tmp/vault" + ``` + +3. Create a working directory for configuration files: + + ```bash + mkdir -p $WORKDIR + ``` + +---8<--- "vault-generate-tls-certs.txt" + +---8<--- "vault-install-tls.txt" + +## Configure Vault + +---8<--- "vault-enable-kv.txt" + +1. (Optional) You can also enable audit. This is not mandatory, but useful: + + ```bash + vault audit enable file file_path=/vault/vault-audit.log + ``` + + ??? example "Expected output" + + ``` {.text .no-copy} + Success! Enabled the file audit device at: file/ + ``` + +## Create a non-root token + +Using the root token for authentication is not recommended, as it poses significant security risks. Instead, you should create a dedicated, non-root token for the Operator to use when accessing Vault. The permissions for this token are controlled by an access policy. Before you create a token you must first create the access policy. + +1. Create a policy for accessing the kv engine path and define the required permissions in the `capabilities` parameter: + + ```bash + kubectl -n "$NAMESPACE" exec vault-0 -- sh -c ' + vault policy write '"$POLICY_NAME"' - << "EOF" + path "tde/data/*" { + capabilities = ["read", "create", "update", "list"] + } + path "tde/metadata/*" { + capabilities = ["read", "list"] + } + path "sys/internal/ui/mounts/*" { + capabilities = ["read"] + } + path "sys/mounts/*" { + capabilities = ["read"] + } + EOF + ' + ``` + +2. Create the AppRole authentication method in Vault. This method allows the Operator to programmatically authenticate to Vault with a role and secret, rather than using a human token. + + ```bash + kubectl -n "${NAMESPACE}" exec pod/vault-0 -- vault auth enable approle + kubectl -n "${NAMESPACE}" exec pod/vault-0 -- vault write auth/approle/role/tde-role policies="${POLICY_NAME}" + ``` + +3. Now create a token with a policy. + + ```bash + kubectl -n "${NAMESPACE}" exec pod/vault-0 -- vault token create -policy="${POLICY_NAME}" -format=json > "${WORKDIR}/vault-token.json" + ``` + +4. Export the non-root token as an environment variable: + + ```bash + export NEW_TOKEN=$(jq -r '.auth.client_token' "${WORKDIR}/vault-token.json") + ``` + +5. Verify the token: + + ```bash + echo "New Vault Token: $NEW_TOKEN" + ``` + + ??? example "Sample output" + + ```{.text .no-copy} + hvs.CAESINO******************************************T2Y + ``` + +## Create a Secret for Vault + +To enable Vault for the Operator, create a Secret object for it using the Vault token and the path to TLS certificates. Note that you must create the Secret in the namespace where the Operator and the database cluster is running. + +For the following command specify the token and the path to the `ca.cert` file (this is `vault.ca` in our example): + +```bash +kubectl create secret generic cluster1-vault --from-literal=token=$NEW_TOKEN --from-file=ca.crt=${WORKDIR}/vault.ca -n $CLUSTER_NAMESPACE +``` + +Check that the Secret is created: + +```bash +kubectl get secret -n $CLUSTER_NAMESPACE +``` + +## Configure `pg_tde` in the Custom Resource manifest + +Now, enable the `pg_tde` extension for your cluster and configure Vault as the key provider. For this you need the following information: + +* * A Vault server name and port. If Vault is deployed in a separate namespace, use the fully qualified name in the format `..svc.cluster.local`. +* The Secret name with the Vault token +* The Secret name with the CA certificate. In our example, the Vault token and the CA certificate are in the same Secret that you created earlier +* The secrets mount path + +!!! note + + Applying the changes for a running cluster will trigger rolling restart of the database Pods. + +1. Edit the `deploy/cr.yaml` file and specify the following: + + * `extensions.pg_tde` - set to `true` + * Add Vault-specific options to the `extensions.pg_tde.vault` section. + + The example configuration looks like this: + + ```yaml + spec: + .... + extensions: + pg_tde: + enabled: true + vault: + host: https://vault.vault.svc.cluster.local:8200 + mountPath: tde + tokenSecret: + name: cluster1-vault + key: token + caSecret: + name: cluster1-vault + key: ca.crt + ``` + +2. Apply the configuration: + + ```bash + kubectl apply -f deploy/cr.yaml -n $CLUSTER_NAMESPACE + ``` + +3. Check the pg_tde status: + + ```bash + kubectl get pg cluster1 -n $CLUSTER_NAMESPACE + ``` + + ??? example "Expected output" + + ```{.yaml .no-copy} + status: + conditions: + ..... + - lastTransitionTime: "2026-03-04T13:29:51Z" + message: pg_tde is enabled in PerconaPGCluster + observedGeneration: 1 + reason: Enabled + status: "True" + type: PGTDEEnabled + ``` + +## Verify the encryption + +Check that the encryption is enabled. To do that, create a table in PostgreSQL using the `tde_heap` access method. To learn more, refer to the [Table Access Methods and pg_tde :octicons-link-external-16:](https://docs.percona.com/pg-tde/index/table-access-method.html) documentation. + +1. Find the primary Pod in your cluster and export it as an environment variable: + + ```bash + export PRIMARY_POD=$(kubectl get pods -n "$CLUSTER_NAMESPACE" \ + -l postgres-operator.crunchydata.com/role=primary \ + -o jsonpath='{.items[0].metadata.name}') + ``` + +2. Verify the Pod: + + ```bash + echo $PRIMARY_POD + ``` + + ??? example "Sample output" + + ```{.text .no-copy} + cluster1-instance1-btdf-0 4/4 Running 0 41m + ``` + +3. Execute into the primary PostgreSQL Pod as the `postgres` user and establish the `psql` session. + + ```bash + kubectl -n $CLUSTER_NAMESPACE exec -it $PRIMARY_POD -- psql + ``` + + ??? example "Sample output" + + ```{.text .no-copy} + psql (17.7 - Percona Server for PostgreSQL 17.7.1) + Type "help" for help. + + postgres=# + ``` + +4. Inside the Pod, create a table: + + ```sql + CREATE TABLE secure_data ( + id INTEGER GENERATED ALWAYS AS IDENTITY PRIMARY KEY, + name TEXT, + amount NUMERIC(10,2), + created_at DATE + ) USING tde_heap; + ``` + +5. Insert some sample data: + + ```sql + INSERT INTO secure_data (name, amount, created_at) VALUES + ('Alice', 1234.56, '2025-08-01'), + ('Bob', 7890.12, '2025-08-10'), + ('Charlie', 345.67, '2025-08-19'); + ``` + +6. Verify if the table is encrypted: + + ```sql + SELECT pg_tde_is_encrypted( + 'secure_data' + ); + ``` + + ??? example "Expected output" + + ``` + pg_tde_is_encrypted + --------------------- + t + (1 row) + ``` + + +## Troubleshooting + +If you encounter issues during the setup, use the following troubleshooting tips: + +1. **Certificate Signing Request (CSR) issues**: If you have problems with the CSR, manually delete it and recreate it: + + ```bash + kubectl delete csr vault-csr || true + ``` + + Then recreate and re-approve it in Kubernetes following the steps in the [Issue the certificate](#issue-the-certificate) section. + +2. **Vault policy issues**: Check the mount points and permissions. Ensure that: + + * The mount path in your policy matches the path where you enabled the secrets engine + * The policy has the required capabilities (`create`, `read`, `update`, `list`) for the paths your application needs + * You have included the `sys/internal/ui/mounts/` and `sys/mounts/*` paths + +3. **Mount point conflicts**: If you encounter issues with a mount point in Vault, you cannot reuse it. You need to: + + * Provide a new mount path when enabling the secrets engine + * Update your access policy to include the new mount path + * Update the `mount_point` value in your Custom Resource configuration to match the new mount path + +4. Verify that you reference the correct secret name in your Custom Resource. + + +## Clean up + +After you finish the setup and ensure everything works as expected, you can clean up the temporary files: + +```bash +rm -rf $WORKDIR +``` + +Disabling `pg_tde` (Transparent Data Encryption) is generally not recommended, as it removes an important layer of security that protects your data at rest. However, if you must disable encryption, you can follow the steps in the [Disable encryption](encryption-disable.md) tutorial. diff --git a/docs/encryption.md b/docs/encryption.md new file mode 100644 index 00000000..f10ca2a0 --- /dev/null +++ b/docs/encryption.md @@ -0,0 +1,44 @@ +# Data-at-rest encryption + +!!! admonition + + This feature is in tech preview stage. + +Data-at-rest encryption ensures that data stored on disk remains protected even if the underlying storage is compromised. This process is transparent to your applications, meaning you don't need to change your application code. If an unauthorized user gains access to the storage, they can't read the data files. + +The Operator supports transparent data encryption (TDE) via the [pg_tde :octicons-link-external-16:](https://docs.percona.com/pg-tde/index.html) extension. When enabled, `pg_tde` encrypts user data in tables, indexes, and temporary tables on disk so that data remains unreadable without the proper encryption keys, even if someone gains access to the storage. + +This feature is available with Percona Distribution for PostgreSQL 17 and above. + +To store encryption keys, the Operator uses a key management system (KMS).The Operator currently supports HashiCorp Vault as the key value storage engine (KV v2). Support of KMIP and other key providers will be added in future releases. WAL encryption is not yet supported. + +## How it works + +When you enable `pg_tde` and provide Vault configuration, the Operator automates the setup: + +1. Adds `pg_tde` to `shared_preload_libraries` so the extension loads at startup. +2. Mounts the Vault token and CA certificate secrets into the database containers. +3. Creates the `pg_tde` extension wth the `CREATE EXTENSION ` command in all databases. +4. Registers Vault as the key provider, creates a global encryption key and sets it as a default key using the functions provided by `pg_tde`. + +The Operator tracks the pg_tde configuration via a hash and records it in the `status.pgTDERevision` condition. The Operator uses the hash to detect changes and reconfigure the `pg_tde` when needed. + +To see the pg_tde configuration status, run the `kubectl get pg -n -o yaml` command and look for the `pgTDERevision` condition. + +The global key name is determined by the cluster's `metadata.uid`, so it changes if you delete and recreate the cluster. `pg_tde` handles this like key rotation as long as both old and new keys remain accessible (for example, you deleted and recreated the cluster without removing PVCs). + +## Implementation specifics + +1. `pg_tde` is available with PostgreSQL 17 and above. +2. Vault must use a **KV secrets engine v2** for the mount path. +3. You can configure Vault to communicate with the Operator with and without TLS. +4. The Operator does not assume anything about the contents of your secrets; you specify the secret names and keys in the Custom Resource. + +## Known limitations + +1. WAL encryption is not yet supported. It will be added in future releases. +2. Only HashiCorp Vault key provider is currently supported. Other providers and the support of KMIP protocol are planned to be added in future releases. + +## Next steps + +[Configure data-at-rest encryption](encryption-setup.md){.md-button} diff --git a/docs/operator.md b/docs/operator.md index 97520de0..2d465bf8 100755 --- a/docs/operator.md +++ b/docs/operator.md @@ -2112,7 +2112,9 @@ The [Kubernetes secret :octicons-link-external-16:](https://kubernetes.io/docs/c ### `extensions.builtin.pg_stat_monitor` -Enable or disable [pg_stat_monitor :octicons-link-external-16:](https://docs.percona.com/pg-stat-monitor/index.html) PostgreSQL extension. +Enable or disable [pg_stat_monitor :octicons-link-external-16:](https://docs.percona.com/pg-stat-monitor/index.html) PostgreSQL extension. + +This option is deprecated. Use the `extensions.pg_stat_monitor` option instead. | Value type | Example | | ---------- | ------- | @@ -2122,6 +2124,8 @@ Enable or disable [pg_stat_monitor :octicons-link-external-16:](https://docs.per Enable or disable [pg_stat_statements :octicons-link-external-16:](https://www.postgresql.org/docs/current/pgstatstatements.html) PostgreSQL extension. +This option is deprecated. Use the `extensions.pg_stat_statements` option instead. + | Value type | Example | | ---------- | ------- | | :material-toggle-switch-outline: boolean | `false` | @@ -2130,6 +2134,8 @@ Enable or disable [pg_stat_statements :octicons-link-external-16:](https://www.p Enable or disable [PGAudit :octicons-link-external-16:](https://www.pgaudit.org/) PostgreSQL extension. +This option is deprecated. Use the `extensions.pg_audit` option instead. + | Value type | Example | | ---------- | ------- | | :material-toggle-switch-outline: boolean | `true` | @@ -2138,6 +2144,8 @@ Enable or disable [PGAudit :octicons-link-external-16:](https://www.pgaudit.org/ Enable or disable [pgvector :octicons-link-external-16:](https://github.com/pgvector/pgvector) PostgreSQL extension. **This extension is not compatible with PostgreSQL 12!** +This option is deprecated. Use the `extensions.pgvector` option instead. + | Value type | Example | | ---------- | ------- | | :material-toggle-switch-outline: boolean | `false` | @@ -2146,10 +2154,110 @@ Enable or disable [pgvector :octicons-link-external-16:](https://github.com/pgve Enable or disable [pg_repack :octicons-link-external-16:](https://github.com/reorg/pg_repack) PostgreSQL extension. +This option is deprecated. Use the `extensions.pg_repack` option instead. + +| Value type | Example | +| ---------- | ------- | +| :material-toggle-switch-outline: boolean | `false` | + +### `extensions.pg_stat_monitor` + +Enable or disable [pg_stat_monitor :octicons-link-external-16:](https://docs.percona.com/pg-stat-monitor/index.html) PostgreSQL extension. + +| Value type | Example | +| ---------- | ------- | +| :material-toggle-switch-outline: boolean | `true` | + +### `extensions.pg_stat_statements` + +Enable or disable [pg_stat_statements :octicons-link-external-16:](https://www.postgresql.org/docs/current/pgstatstatements.html) PostgreSQL extension. + +| Value type | Example | +| ---------- | ------- | +| :material-toggle-switch-outline: boolean | `false` | + +### `extensions.pg_audit` + +Enable or disable [PGAudit :octicons-link-external-16:](https://www.pgaudit.org/) PostgreSQL extension. + +| Value type | Example | +| ---------- | ------- | +| :material-toggle-switch-outline: boolean | `true` | + +### `extensions.pgvector` + +Enable or disable [pgvector :octicons-link-external-16:](https://github.com/pgvector/pgvector) PostgreSQL extension. **This extension is not compatible with PostgreSQL 12!** + +| Value type | Example | +| ---------- | ------- | +| :material-toggle-switch-outline: boolean | `false` | + +### `extensions.pg_repack` + +Enable or disable [pg_repack :octicons-link-external-16:](https://github.com/reorg/pg_repack) PostgreSQL extension. + +| Value type | Example | +| ---------- | ------- | +| :material-toggle-switch-outline: boolean | `false` | + +### `extensions.pg_tde` + +Enable or disable [pg_rtde :octicons-link-external-16:](https://docs.percona.com/pg-tde/index.html) PostgreSQL extension. Read more about it in [Data-at-rest encryption](encryption.md). + +This extension is compatible **with Percona Distribution for PostgreSQL 17 and above**. + | Value type | Example | | ---------- | ------- | | :material-toggle-switch-outline: boolean | `false` | +### `extensions.pg_tde.vault.host` + +The Vault server name and port. If Vault is deployed in a separate namespace, use the fully qualified name in the format `..svc.cluster.local`. Use the HTTPS protocol for encrypted communication with TLS and HTTP protocol for communication without TLS. + +| Value type | Example | +| ---------- | ------- | +| :material-code-string: string | `https://vault-service:8200` | + +### `extensions.pg_tde.vault.mountPath` + +The secrets mount path in Vault. + +| Value type | Example | +| ---------- | ------- | +| :material-code-string: string | `tde` | + +### `extensions.pg_tde.vault.tokenSecret.name` + +The name of the Secrets object that stores a token to access Vault. + +| Value type | Example | +| ---------- | ------- | +| :material-code-string: string | `pg-tde-vault-secret` | + +### `extensions.pg_tde.vault.tokenSecret.key` + +Specifies a token to use for accessing Vault. + +| Value type | Example | +| ---------- | ------- | +| :material-code-string: string | `token` | + +### `extensions.pg_tde.vault.caSecret.name` + +The name of the Secrets object that stores TLS certificates for secure communication with Vault. You can use the same Secret for storing a token and TLS certificates. + +| Value type | Example | +| ---------- | ------- | +| :material-code-string: string | `pg-tde-vault-secret` | + +### `extensions.pg_tde.vault.caSecret.key` + +Specifies the CA certificate to provide to Vault during authentication. + +| Value type | Example | +| ---------- | ------- | +| :material-code-string: string | `ca.crt` | + ### `extensions.custom.name` Name of the PostgreSQL custom extension. @@ -2166,3 +2274,4 @@ Version of the PostgreSQL custom extension. | ---------- | ------- | | :material-code-string: string | `1.6.1` | + diff --git a/mkdocs-base.yml b/mkdocs-base.yml index 47786d95..9432aec0 100644 --- a/mkdocs-base.yml +++ b/mkdocs-base.yml @@ -202,6 +202,9 @@ nav: - "Anti-affinity and tolerations": constraints.md - "Labels and annotations": annotations.md - "Transport encryption (TLS/SSL)": TLS.md + - Data-at-rest encryption: + - About data-at-rest encryption: encryption.md + - Configure data-at-rest encryption with HashiCorp Vault: encryption-setup.md - "Telemetry": telemetry.md - reconciliation-concurrency.md - Management: From d0dddd34a48d21d60222fa3310b8ae1837f28446 Mon Sep 17 00:00:00 2001 From: Anastasia Alexandrova Date: Thu, 5 Mar 2026 10:16:37 +0100 Subject: [PATCH 02/13] Update docs/encryption-setup.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- docs/encryption-setup.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/encryption-setup.md b/docs/encryption-setup.md index b6d3cd23..7daf1736 100644 --- a/docs/encryption-setup.md +++ b/docs/encryption-setup.md @@ -148,7 +148,7 @@ kubectl get secret -n $CLUSTER_NAMESPACE Now, enable the `pg_tde` extension for your cluster and configure Vault as the key provider. For this you need the following information: -* * A Vault server name and port. If Vault is deployed in a separate namespace, use the fully qualified name in the format `..svc.cluster.local`. +* A Vault server name and port. If Vault is deployed in a separate namespace, use the fully qualified name in the format `..svc.cluster.local`. * The Secret name with the Vault token * The Secret name with the CA certificate. In our example, the Vault token and the CA certificate are in the same Secret that you created earlier * The secrets mount path From dcbacd6f486894f41fb64c4d6c3d9ac287f62eb0 Mon Sep 17 00:00:00 2001 From: Anastasia Alexandrova Date: Thu, 5 Mar 2026 10:16:56 +0100 Subject: [PATCH 03/13] Update docs/encryption-setup.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- docs/encryption-setup.md | 10 +--------- 1 file changed, 1 insertion(+), 9 deletions(-) diff --git a/docs/encryption-setup.md b/docs/encryption-setup.md index 7daf1736..e7b962f9 100644 --- a/docs/encryption-setup.md +++ b/docs/encryption-setup.md @@ -97,19 +97,11 @@ Using the root token for authentication is not recommended, as it poses signific ' ``` -2. Create the AppRole authentication method in Vault. This method allows the Operator to programmatically authenticate to Vault with a role and secret, rather than using a human token. - - ```bash - kubectl -n "${NAMESPACE}" exec pod/vault-0 -- vault auth enable approle - kubectl -n "${NAMESPACE}" exec pod/vault-0 -- vault write auth/approle/role/tde-role policies="${POLICY_NAME}" - ``` - -3. Now create a token with a policy. +2. Now create a token with a policy. ```bash kubectl -n "${NAMESPACE}" exec pod/vault-0 -- vault token create -policy="${POLICY_NAME}" -format=json > "${WORKDIR}/vault-token.json" ``` - 4. Export the non-root token as an environment variable: ```bash From 6bc501d347840e8c9aec4e07b584ba3195aa8c6c Mon Sep 17 00:00:00 2001 From: Anastasia Alexandrova Date: Thu, 5 Mar 2026 10:17:37 +0100 Subject: [PATCH 04/13] Update docs/operator.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- docs/operator.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/operator.md b/docs/operator.md index 2d465bf8..090854f1 100755 --- a/docs/operator.md +++ b/docs/operator.md @@ -2202,7 +2202,7 @@ Enable or disable [pg_repack :octicons-link-external-16:](https://github.com/reo ### `extensions.pg_tde` -Enable or disable [pg_rtde :octicons-link-external-16:](https://docs.percona.com/pg-tde/index.html) PostgreSQL extension. Read more about it in [Data-at-rest encryption](encryption.md). +Enable or disable [pg_tde :octicons-link-external-16:](https://docs.percona.com/pg-tde/index.html) PostgreSQL extension. Read more about it in [Data-at-rest encryption](encryption.md). This extension is compatible **with Percona Distribution for PostgreSQL 17 and above**. From 88b602b16645c940ffa8677e650c72ebcccf99c0 Mon Sep 17 00:00:00 2001 From: Anastasia Alexandrova Date: Thu, 5 Mar 2026 10:18:06 +0100 Subject: [PATCH 05/13] Update docs/encryption.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- docs/encryption.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/encryption.md b/docs/encryption.md index f10ca2a0..ec27cfb2 100644 --- a/docs/encryption.md +++ b/docs/encryption.md @@ -21,9 +21,9 @@ When you enable `pg_tde` and provide Vault configuration, the Operator automates 3. Creates the `pg_tde` extension wth the `CREATE EXTENSION ` command in all databases. 4. Registers Vault as the key provider, creates a global encryption key and sets it as a default key using the functions provided by `pg_tde`. -The Operator tracks the pg_tde configuration via a hash and records it in the `status.pgTDERevision` condition. The Operator uses the hash to detect changes and reconfigure the `pg_tde` when needed. +The Operator tracks the pg_tde configuration via a hash and exposes its state through a `PGTDEEnabled` condition in `status.conditions`. The Operator uses the hash to detect changes and reconfigure the `pg_tde` when needed. -To see the pg_tde configuration status, run the `kubectl get pg -n -o yaml` command and look for the `pgTDERevision` condition. +To see the pg_tde configuration status, run the `kubectl get pg -n -o yaml` command and look for the condition with `type: PGTDEEnabled` under `status.conditions`. The global key name is determined by the cluster's `metadata.uid`, so it changes if you delete and recreate the cluster. `pg_tde` handles this like key rotation as long as both old and new keys remain accessible (for example, you deleted and recreated the cluster without removing PVCs). From cac3cf55b17839d3975586fb672a7cacaa9dedd9 Mon Sep 17 00:00:00 2001 From: Anastasia Alexandrova Date: Thu, 5 Mar 2026 10:18:38 +0100 Subject: [PATCH 06/13] Update docs/encryption-setup.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- docs/encryption-setup.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/encryption-setup.md b/docs/encryption-setup.md index e7b962f9..3de232f0 100644 --- a/docs/encryption-setup.md +++ b/docs/encryption-setup.md @@ -298,7 +298,7 @@ If you encounter issues during the setup, use the following troubleshooting tips * Provide a new mount path when enabling the secrets engine * Update your access policy to include the new mount path - * Update the `mount_point` value in your Custom Resource configuration to match the new mount path + * Update the `mountPath` value in your Custom Resource configuration to match the new mount path 4. Verify that you reference the correct secret name in your Custom Resource. From f15a7391a1dc42b99baca99622f9fae554e0115c Mon Sep 17 00:00:00 2001 From: Anastasia Alexandrova Date: Thu, 5 Mar 2026 10:19:03 +0100 Subject: [PATCH 07/13] Update docs/encryption.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- docs/encryption.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/encryption.md b/docs/encryption.md index ec27cfb2..0be6237f 100644 --- a/docs/encryption.md +++ b/docs/encryption.md @@ -10,7 +10,7 @@ The Operator supports transparent data encryption (TDE) via the [pg_tde :octicon This feature is available with Percona Distribution for PostgreSQL 17 and above. -To store encryption keys, the Operator uses a key management system (KMS).The Operator currently supports HashiCorp Vault as the key value storage engine (KV v2). Support of KMIP and other key providers will be added in future releases. WAL encryption is not yet supported. +To store encryption keys, the Operator uses a key management system (KMS). The Operator currently supports HashiCorp Vault as the key value storage engine (KV v2). Support of KMIP and other key providers will be added in future releases. WAL encryption is not yet supported. ## How it works From 9b0e256884a6bc1e87bf0fb1132918523ab347af Mon Sep 17 00:00:00 2001 From: Anastasia Alexandrova Date: Thu, 5 Mar 2026 10:28:58 +0100 Subject: [PATCH 08/13] Update docs/encryption-disable.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- docs/encryption-disable.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/encryption-disable.md b/docs/encryption-disable.md index a0d5226d..3c4dff6f 100644 --- a/docs/encryption-disable.md +++ b/docs/encryption-disable.md @@ -33,7 +33,7 @@ Disabling `pg_tde` (Transparent Data Encryption) is generally not recommended, a This command triggers the rolling restart of your database Pods. As a result, the Operator runs `DROP EXTENSION pg_tde` in all databases. -4. Run the `CHECKPOINT` command in PostgreSQL. It forces an immediate checkpoint to flush all dirty pages to disk and update all datafiles and indexes. Connect to the Connect to the primary database Pod as the `postgres` user and run: +4. Run the `CHECKPOINT` command in PostgreSQL. It forces an immediate checkpoint to flush all dirty pages to disk and update all datafiles and indexes. Connect to the primary database Pod as the `postgres` user and run: ```sql CHECKPOINT; From 6046bd101a3e0a5c212cf7a688d3a81bca8cbb49 Mon Sep 17 00:00:00 2001 From: Anastasia Alexandrova Date: Thu, 5 Mar 2026 10:29:24 +0100 Subject: [PATCH 09/13] Update docs/encryption-setup.md Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> --- docs/encryption-setup.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/encryption-setup.md b/docs/encryption-setup.md index 3de232f0..ee1632b1 100644 --- a/docs/encryption-setup.md +++ b/docs/encryption-setup.md @@ -10,7 +10,7 @@ This document guides you through setting up data-at-rest encryption with `pg_tde ## Prerequisites -To configure data-at-rest encryptio, you need the following: +To configure data-at-rest encryption, you need the following: * `kubectl`- Kubernetes command-line interface * `helm` - Helm package manager From 211e01efcbc2dc9e14530d78d9a184d1cf631437 Mon Sep 17 00:00:00 2001 From: Anastasia Alexadrova Date: Thu, 5 Mar 2026 10:49:32 +0100 Subject: [PATCH 10/13] Updated after the review, applied Copilot suggestions --- docs/assets/fragments/vault-install-tls.txt | 8 +++---- docs/encryption-disable.md | 25 +++++++++++++-------- docs/encryption-setup.md | 2 +- docs/encryption.md | 2 +- docs/operator.md | 4 ++-- 5 files changed, 24 insertions(+), 17 deletions(-) diff --git a/docs/assets/fragments/vault-install-tls.txt b/docs/assets/fragments/vault-install-tls.txt index 44891061..81bff4a4 100644 --- a/docs/assets/fragments/vault-install-tls.txt +++ b/docs/assets/fragments/vault-install-tls.txt @@ -73,7 +73,7 @@ For this setup, we install Vault in Kubernetes using the [Helm 3 package manager https://developer.hashicorp.com/vault/docs ``` -4. Retrieve the Pod name where Vault is running: +3. Retrieve the Pod name where Vault is running: ```bash kubectl -n $NAMESPACE get pod -l app.kubernetes.io/name=${SERVICE} -o jsonpath='{range .items[*]}{.metadata.name}{"\n"}{end}' @@ -179,15 +179,15 @@ For this setup, we install Vault in Kubernetes using the [Helm 3 package manager ??? example "Expected output" - ```{.text .no-value} + ```{.text .no-copy} Key Value --- ----- Seal Type shamir Initialized true - Sealed true + Sealed false Total Shares 1 Threshold 1 - Unseal Progress 0/1 + Unseal Progress 1/1 Unseal Nonce n/a Version 1.19.0 Build Date 2025-03-04T12:36:40Z diff --git a/docs/encryption-disable.md b/docs/encryption-disable.md index 3c4dff6f..d133fb8f 100644 --- a/docs/encryption-disable.md +++ b/docs/encryption-disable.md @@ -4,11 +4,17 @@ Disabling `pg_tde` (Transparent Data Encryption) is generally not recommended, a !!! important - To properly disable encryption in the Operator, you must follow a specific sequence and modify your Custom Resource (CR) twice. Attempting to disable everything in a single step will not work: the Operator needs to drop the `pg_tde` extension before you remove the Vault (key provider) configuration. + To properly disable encryption in the Operator, you must follow a specific sequence and modify your Custom Resource (CR) twice. Attempting to disable everything in a single step will not work: the Operator needs to drop the `pg_tde` extension before you remove the key provider configuration. Failing to follow the steps in this tutorial in order will result in errors, because removing the Vault configuration before the extension is dropped prevents the Operator from cleaning up properly. -1. Unencrypt **all encrypted databases** in your database. Connect to the primary database Pod as the `postgres` user, connect to each encrypted database and run the following command to unencrypt **every** encrypted table +1. Export the namespace where your database cluster is deployed as an environment variable. Replace the `` placeholder with your value: + + ```bash + export CLUSTER_NAMESPACE= + ``` + +2. Unencrypt **all encrypted databases** in your database. Connect to the primary database Pod as the `postgres` user, connect to each encrypted database and run the following command to unencrypt **every** encrypted table ```sql ALTER TABLE SET ACCESS METHOD heap; @@ -16,7 +22,7 @@ Disabling `pg_tde` (Transparent Data Encryption) is generally not recommended, a Exit the Pod. -2. Edit the Custom Resource and set the `extensions.pg_tde.enabled` option to `false`. +3. Edit the Custom Resource and set the `extensions.pg_tde.enabled` option to `false`. ```yaml spec: @@ -25,15 +31,15 @@ Disabling `pg_tde` (Transparent Data Encryption) is generally not recommended, a enabled: false ``` -3. Apply the changes: +4. Apply the changes: ```bash - kubectl apply -f deploy/cr.yaml -n $NAMESPACE + kubectl apply -f deploy/cr.yaml -n $CLUSTER_NAMESPACE ``` This command triggers the rolling restart of your database Pods. As a result, the Operator runs `DROP EXTENSION pg_tde` in all databases. -4. Run the `CHECKPOINT` command in PostgreSQL. It forces an immediate checkpoint to flush all dirty pages to disk and update all datafiles and indexes. Connect to the primary database Pod as the `postgres` user and run: +5. Run the `CHECKPOINT` command in PostgreSQL. It forces an immediate checkpoint to flush all dirty pages to disk and update all datafiles and indexes. Connect to the primary database Pod as the `postgres` user and run: ```sql CHECKPOINT; @@ -43,11 +49,12 @@ Disabling `pg_tde` (Transparent Data Encryption) is generally not recommended, a Exit the Pod. -5. Update the Custom Resource again and remove all vault-related configuration from `extensions.pg_tde` section. -6. Apply the changes: +6. Update the Custom Resource again and remove all vault-related configuration from `extensions.pg_tde` section. + +7. Apply the changes: ```bash - kubectl apply -f deploy/cr.yaml -n $NAMESPACE + kubectl apply -f deploy/cr.yaml -n $CLUSTER_NAMESPACE ``` This triggers another rolling restart of the database Pods. diff --git a/docs/encryption-setup.md b/docs/encryption-setup.md index ee1632b1..7baf2a80 100644 --- a/docs/encryption-setup.md +++ b/docs/encryption-setup.md @@ -220,7 +220,7 @@ Check that the encryption is enabled. To do that, create a table in PostgreSQL u ??? example "Sample output" ```{.text .no-copy} - cluster1-instance1-btdf-0 4/4 Running 0 41m + cluster1-instance1-btdf-0 ``` 3. Execute into the primary PostgreSQL Pod as the `postgres` user and establish the `psql` session. diff --git a/docs/encryption.md b/docs/encryption.md index 0be6237f..b2a6bdd0 100644 --- a/docs/encryption.md +++ b/docs/encryption.md @@ -18,7 +18,7 @@ When you enable `pg_tde` and provide Vault configuration, the Operator automates 1. Adds `pg_tde` to `shared_preload_libraries` so the extension loads at startup. 2. Mounts the Vault token and CA certificate secrets into the database containers. -3. Creates the `pg_tde` extension wth the `CREATE EXTENSION ` command in all databases. +3. Creates the `pg_tde` extension with the `CREATE EXTENSION pg_tde;` command in all databases. 4. Registers Vault as the key provider, creates a global encryption key and sets it as a default key using the functions provided by `pg_tde`. The Operator tracks the pg_tde configuration via a hash and exposes its state through a `PGTDEEnabled` condition in `status.conditions`. The Operator uses the hash to detect changes and reconfigure the `pg_tde` when needed. diff --git a/docs/operator.md b/docs/operator.md index 090854f1..82d559ec 100755 --- a/docs/operator.md +++ b/docs/operator.md @@ -2228,7 +2228,7 @@ The secrets mount path in Vault. ### `extensions.pg_tde.vault.tokenSecret.name` -The name of the Secrets object that stores a token to access Vault. +The name of the Secret object that stores a token to access Vault. | Value type | Example | | ---------- | ------- | @@ -2244,7 +2244,7 @@ Specifies a token to use for accessing Vault. ### `extensions.pg_tde.vault.caSecret.name` -The name of the Secrets object that stores TLS certificates for secure communication with Vault. You can use the same Secret for storing a token and TLS certificates. +The name of the Secret object that stores TLS certificates for secure communication with Vault. You can use the same Secret for storing a token and TLS certificates. | Value type | Example | | ---------- | ------- | From 81b96dbfcb6488e54c586ca450481bda74035a8e Mon Sep 17 00:00:00 2001 From: Anastasia Alexadrova Date: Thu, 5 Mar 2026 15:04:55 +0100 Subject: [PATCH 11/13] Added info about backups and restores with pg-tde --- docs/encryption.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/encryption.md b/docs/encryption.md index b2a6bdd0..b5ed189c 100644 --- a/docs/encryption.md +++ b/docs/encryption.md @@ -27,6 +27,8 @@ To see the pg_tde configuration status, run the `kubectl get pg - The global key name is determined by the cluster's `metadata.uid`, so it changes if you delete and recreate the cluster. `pg_tde` handles this like key rotation as long as both old and new keys remain accessible (for example, you deleted and recreated the cluster without removing PVCs). +With `pg_tde` enabled you can make backups and restores as usual. Note that for the restore the Operator must have the access to the encryption key that was used to encrypt the backup data. + ## Implementation specifics 1. `pg_tde` is available with PostgreSQL 17 and above. From ff6b36a80b0018313f0555b02c333ec3836b302f Mon Sep 17 00:00:00 2001 From: Anastasia Alexadrova Date: Tue, 10 Mar 2026 09:57:02 +0100 Subject: [PATCH 12/13] Updated info about PGTDEEnabled condition Split text into sections for scannability --- docs/encryption.md | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-) diff --git a/docs/encryption.md b/docs/encryption.md index b5ed189c..3bfa6f70 100644 --- a/docs/encryption.md +++ b/docs/encryption.md @@ -21,13 +21,25 @@ When you enable `pg_tde` and provide Vault configuration, the Operator automates 3. Creates the `pg_tde` extension with the `CREATE EXTENSION pg_tde;` command in all databases. 4. Registers Vault as the key provider, creates a global encryption key and sets it as a default key using the functions provided by `pg_tde`. -The Operator tracks the pg_tde configuration via a hash and exposes its state through a `PGTDEEnabled` condition in `status.conditions`. The Operator uses the hash to detect changes and reconfigure the `pg_tde` when needed. +## Status and conditions -To see the pg_tde configuration status, run the `kubectl get pg -n -o yaml` command and look for the condition with `type: PGTDEEnabled` under `status.conditions`. +The Operator tracks the `pg_tde` configuration via a hash and exposes its state through a `PGTDEEnabled` condition in `status.conditions`. The Operator uses the hash to detect changes and reconfigure the `pg_tde` when needed. + +To see the `pg_tde` configuration status, run: + +```bash +kubectl get pg -n -o yaml +``` + +Look for the condition with `type: PGTDEEnabled` under `status.conditions`. + +The `PGTDEEnabled` condition indicates that the `pg_tde` extension is created in all databases and added to `shared_preload_libraries`. Note that the condition can be `True` and the cluster status `Ready` even when there are issues with token or key provider configuration. However, the Operator logs these errors and if you encounter issues with encryption, check the Operator logs for details. + +## Global key handling The global key name is determined by the cluster's `metadata.uid`, so it changes if you delete and recreate the cluster. `pg_tde` handles this like key rotation as long as both old and new keys remain accessible (for example, you deleted and recreated the cluster without removing PVCs). -With `pg_tde` enabled you can make backups and restores as usual. Note that for the restore the Operator must have the access to the encryption key that was used to encrypt the backup data. +With `pg_tde` enabled you can make backups and restores as usual. For restore, the Operator must have access to the encryption key that was used to encrypt the backup data. ## Implementation specifics From 34675a02736f3375a77fa62a2802e56112e2c412 Mon Sep 17 00:00:00 2001 From: Anastasia Alexadrova Date: Mon, 16 Mar 2026 15:01:56 +0100 Subject: [PATCH 13/13] Added a specifics about tde use with standby clusters --- docs/encryption-disable.md | 22 ++++++++++------------ docs/encryption.md | 3 +++ 2 files changed, 13 insertions(+), 12 deletions(-) diff --git a/docs/encryption-disable.md b/docs/encryption-disable.md index d133fb8f..f0fbe116 100644 --- a/docs/encryption-disable.md +++ b/docs/encryption-disable.md @@ -20,9 +20,17 @@ Disabling `pg_tde` (Transparent Data Encryption) is generally not recommended, a ALTER TABLE SET ACCESS METHOD heap; ``` +3. Run the `CHECKPOINT` command in PostgreSQL. It forces an immediate checkpoint to flush all dirty pages to disk and update all datafiles and indexes. Connect to the primary database Pod as the `postgres` user and run: + + ```sql + CHECKPOINT; + ``` + + This flushes data to disk in all databases. + Exit the Pod. -3. Edit the Custom Resource and set the `extensions.pg_tde.enabled` option to `false`. +4. Edit the Custom Resource and set the `extensions.pg_tde.enabled` option to `false`. ```yaml spec: @@ -31,7 +39,7 @@ Disabling `pg_tde` (Transparent Data Encryption) is generally not recommended, a enabled: false ``` -4. Apply the changes: +5. Apply the changes: ```bash kubectl apply -f deploy/cr.yaml -n $CLUSTER_NAMESPACE @@ -39,16 +47,6 @@ Disabling `pg_tde` (Transparent Data Encryption) is generally not recommended, a This command triggers the rolling restart of your database Pods. As a result, the Operator runs `DROP EXTENSION pg_tde` in all databases. -5. Run the `CHECKPOINT` command in PostgreSQL. It forces an immediate checkpoint to flush all dirty pages to disk and update all datafiles and indexes. Connect to the primary database Pod as the `postgres` user and run: - - ```sql - CHECKPOINT; - ``` - - This flushes data to disk in all databases. - - Exit the Pod. - 6. Update the Custom Resource again and remove all vault-related configuration from `extensions.pg_tde` section. 7. Apply the changes: diff --git a/docs/encryption.md b/docs/encryption.md index 3bfa6f70..5e4edeaf 100644 --- a/docs/encryption.md +++ b/docs/encryption.md @@ -47,6 +47,9 @@ With `pg_tde` enabled you can make backups and restores as usual. For restore, t 2. Vault must use a **KV secrets engine v2** for the mount path. 3. You can configure Vault to communicate with the Operator with and without TLS. 4. The Operator does not assume anything about the contents of your secrets; you specify the secret names and keys in the Custom Resource. +5. If you are using a [standby cluster](standby.md), you must [configure `pg_tde`](encryption-setup.md) and the key provider on both the source (primary) and the standby clusters. This means you need to enable the extension and set up the key provider in each cluster’s Custom Resource. This configuration is essential for the standby to be able to write and access encrypted data from the source. + + Initially, the Operator uses the key provider configuration from the source cluster to write data on the standby. If the standby cluster is promoted to become the new primary, it will generate its own key provider configuration. The data previously written remains accessible, provided that the proper key provider setup was completed on the standby before promotion. In summary, both the source and standby clusters require correct `pg_tde` and key provider configuration for seamless operation and failover. ## Known limitations