When Hybrid Manager (HM) is deployed across multiple locations, Transparent Data Encryption (TDE) behaves differently depending on the cluster architecture you provision. This page describes what happens when you supply a customer-managed key per location, and what to plan for when you scale a cluster from a single region to multiple regions. The guidance applies to every supported key management provider — HashiCorp Vault, AWS KMS, Azure Key Vault, Google Cloud KMS — and to the passphrase option.
Read this page in addition to the multi-DC deployment guide and the Enabling KMS for TDE index.
How TDE protects your data
TDE uses two keys:
A customer-managed key (KEK). You register this in HM as a customer key. It lives in your key management service (KMS) provider or for the passphrase option, is derived from a passphrase you supply. HM never sees the key material itself. HM only invokes wrap/unwrap operations against your KMS.
A data encryption key (DEK). HM generates this once per Postgres cluster at cluster creation. In a distributed high-availability deployment, each data group generates and holds its own DEK. The DEK is what actually encrypts your on-disk data and write-ahead log files. Postgres unwraps the DEK using your KMS at each start-up and keeps the plaintext DEK in memory to encrypt reads and writes. DEKs never leave the cluster (or, for PGD, the data group) that generated them.
Two consequences follow that matter for multi-DC planning:
- Once a cluster's on-disk encryption file is written, only a KEK that can decrypt that file will let the cluster start.
- Whichever KEK produced the file is the KEK that must remain available for the lifetime of that cluster's data.
How TDE and cross-data-center replication interact
Whether the customer key registered in a second location has to share key material with the source's key, or is free to be genuinely different, depends on how data reaches that location and on what the operator does with the encryption file after the copy:
Physical streaming replication (PSR). The source's data directory is copied byte-for-byte via Postgres base-backup, including the encryption file that stores the wrapped DEK. The replica keeps that file as-is, so only a KEK compatible with the source's KEK can unwrap it. This is the transport used by replica clusters, and it is why the replica's own customer key must resolve to compatible key material.
Adding a data group to a distributed high-availability (PGD) cluster. A physical join (
bdr_init_physical) copies the source group's data directory in full, including the encryption file that stores the wrapped DEK — the same mechanism a PSR replica uses. The joining group starts Postgres using its own locally configured wrap/unwrap commands, exactly as at every other startup; there's no separate operator step that reads the DEK with the source group's KEK and re-wraps it under the target's. This means the copiedkey.binmust be unwrappable by the joining group's own KEK, so the joining group's KEK must resolve to key material compatible with the source group's KEK — the same requirement that applies to a PSR replica cluster.How PGD data groups stay in sync after they're up. Changes are decoded into row images inside the source's Postgres instance, after the storage layer has already decrypted the page, and streamed to the destination over mutually authenticated Transport Layer Security (TLS). The destination's Postgres instance receives the row images and its own storage layer encrypts them under its own DEK before writing to disk. Wrapped keys never leave a location, and the two locations don't exchange any key material at this stage. This ongoing replication path doesn't care whether the two groups' KEKs share material — but it only runs after the joining group has already started successfully, which does require compatible material (see above).
Note
The takeaway for planning: only the seed data group's key.bin is generated fresh, wrapped directly under its own configured KEK. Every data group added later via physical join, and every PSR replica, receives a physically copied key.bin and can only start if its own configured KEK resolves to key material compatible with the KEK that produced that file. There's currently no re-wrap step anywhere in the pipeline that would let a joined group or a replica end up under genuinely different key material.
Known limitation: cross-vault same-material lookups
The recommended pattern — one customer-key record per location, all resolving to the same underlying KMS material — is required for both PSR replicas and PGD groups added by physical join. There's currently a code-level rough edge in how that same-material case is resolved on Azure: each decrypt attempt first tries the key name/version embedded in the copied file's metadata, which only ever exists in the source vault, so it predictably fails before falling back to the locally configured key. The fallback does recover correctly (as long as the material genuinely matches), but this failed first attempt repeats on every Postgres restart, not just the initial join, because nothing rewrites the file's embedded metadata afterward. Granting the joining location's identity decrypt access on the source location's key vault has no effect on this failure mode — the client used for both attempts is always bound to the locally configured vault, so the source vault is never actually contacted.
Missing decrypt permission on the joining group's own configured key is the most common cause of a joining pod stuck in the bootstrap phase — see PGD data group stuck with key unwrap command produced no output in the troubleshooting guide.
Replica clusters across data centers
A replica cluster (from the Replica Clusters tab) is a physical streaming replica of a primary cluster. When the replica is created, it's bootstrapped from the primary using Postgres's base-backup mechanism, which copies the primary's data directory in full, including the encryption file that stores the wrapped DEK.
This has an important implication:
- The KEK that protects a replica cluster's data at rest is the primary's KEK, not the customer key you configure on the replica. The replica's customer key is registered per location for access-control purposes, but the underlying key material must be compatible with the primary's KEK so that the replica's Postgres instance can start.
In practice, "compatible" means one of the following:
The same underlying key material referenced from the replica's location. Examples:
- AWS KMS: use a multi-region replica key whose replicas are present in every location that hosts a replica cluster.
- Azure Key Vault: use a key vault reachable from every location, or replicate the same key material into a per-region vault.
- Google Cloud KMS: use a multi-region or global key ring reachable from every location.
- HashiCorp Vault: replicate the Transit engine key across regions.
- Passphrase: use the same passphrase everywhere.
A single KEK reachable from all locations. If the same customer-managed key is accessible from every region that hosts a replica cluster, you can register it as one customer key per location in HM (each row is per-location metadata) and the replica clusters will be able to start.
Recommendations
Register a customer key in every location that will host a replica cluster, and make sure they resolve to compatible key material. HM lets you configure a different customer key per replica in the create request; use this to represent per-location access control, not to swap the underlying key material.
Don't expect the replica's own customer key to become the effective KEK after bootstrap. HM doesn't re-wrap the encryption file on the replica after base-backup. The primary's KEK remains the key that must be able to unwrap the file for the life of the replica cluster.
Plan KEK rotation as a per-cluster operation across all locations. Rotating the KEK on the primary rewrites the primary's on-disk encryption file, but this change doesn't propagate over streaming replication. If you rotate the primary's KEK, you must run an equivalent rotation on each replica so their Postgres instances can continue to start.
Cross-cloud multi-DC isn't supported. A multi-DC HM deployment requires the same provider or on-premises family across all locations, and TDE inherits that constraint. Your KMS must be reachable from every location that hosts a replica cluster.
Distributed high-availability (PGD) clusters across data centers
A distributed high-availability cluster is composed of independent data groups. Each data group is a separate Postgres cluster that participates in the cluster's cross-group replication. In Hybrid Manager, every data group added after the seed is joined by a physical join (bdr_init_physical), which copies the source group's data directory in full, including the wrapped-DEK file, the same way a PSR replica is bootstrapped from its primary:
| Data group | How key.bin is created on the target | KEK required for the group to start |
|---|---|---|
| Seed (first) data group | Freshly generated at cluster creation, wrapped by the group's own configured KEK | The group's own configured KEK |
Any group added later (physical join via bdr_init_physical) | Copied from the source group by base-backup; not re-wrapped | A KEK compatible with the source group's KEK (same material) |
Unlike a fresh seed group, a group added by physical join has no operator-driven re-wrap step. The joining group's own configured KEK is what Postgres uses to unwrap the copied file at every startup, so it only starts successfully if that KEK resolves to key material compatible with whatever KEK wrapped the file at the source.
Recommendations
Register a customer key in every location that will host a data group, and select the appropriate key per data group under Security on the Data Groups tab. Each row is per-location metadata for access control — it isn't a way to swap the underlying key material.
Per-region key sovereignty (genuinely different key material per data group) isn't currently supported. A data group added by physical join must resolve to the same underlying key material as the group it joined from, for the same reason a PSR replica must be compatible with its primary — see How TDE and cross-data-center replication interact. Configuring a joining group with genuinely different material results in the group permanently failing to start Postgres.
Use the same compatibility patterns as replica clusters to give each location its own key record: a multi-region/global key from your KMS provider reachable from every location, or the same underlying material imported separately into a per-location key vault/key ring (registered as a distinct customer key per location in HM). See Replica clusters across data centers for provider-specific examples. Note the current rough edge described above for the per-location-import pattern on Azure.
KEK rotation on one data group is local to that group. Rotating a data group's KEK rewrites that group's on-disk encryption file only. Other data groups continue with their own KEKs unchanged.
Scaling from single region to multi-region
Adding a replica cluster to an existing single-region cluster works the same way as creating a replica cluster alongside a primary at initial cluster creation. The new replica is bootstrapped from the primary using base-backup, so the primary's KEK becomes the effective KEK for the new replica. Register a compatible customer key in the target location before adding the replica.
Adding a data group to an existing distributed high-availability cluster works the same way as a PSR replica bootstrap: the physical join (bdr_init_physical) copies the source group's key.bin as-is, with no re-wrap step. The new group's configured KEK must resolve to key material compatible with the source group's KEK, and only the new group's own KEK needs to be reachable from the new group's location — there's no separate access requirement on the source group's key, since the join never actually contacts the source's key vault. See Distributed high-availability (PGD) clusters across data centers for the full comparison.
Key rotation
Note
HM doesn't currently expose a built-in workflow to rotate the customer-managed key (KEK) of an existing cluster. Rotation is a customer-driven activity that you plan and execute against your own KMS provider, where the on-disk wrapped key must also be rewritten against the running Postgres instances.
Three distinct scenarios are commonly called "key rotation." They have very different requirements.
1. Automatic rotation inside your KMS (transparent)
Most cloud KMS providers can rotate the underlying cryptographic material of a customer-managed key on a schedule (for example, AWS KMS annual auto-rotation on customer-managed keys). In these cases the key's identifier (Amazon Resource Name (ARN), resource name, key URI) doesn't change, and prior versions of the material remain usable for decrypt. No action is required on the HM side. Your Postgres clusters continue to unwrap their on-disk encryption file transparently.
If you're only relying on your KMS provider's built-in rotation, there's nothing further to plan.
2. Replacing one customer key with a different customer key
If you want to move a cluster's encryption from one customer-managed key to a different one (for example, to migrate to a new HSM, a new region's key, or to retire an old key after a compromise), this isn't currently possible through HM without manual customer intervention. The Hybrid Manager console doesn't expose a rotate-key action.
If you need to perform this rotation, plan for it as a customer-side operation coordinated with your Postgres administrator. The high-level shape is:
- Register the new customer key in every HM location that hosts data for the cluster.
- On every Postgres pod that holds an on-disk encryption file for the cluster (primary and every replica for a replica-cluster topology; every data group's nodes for a distributed high-availability topology), run the Postgres-provided key rotation operation to rewrite the on-disk wrapped key under the new KEK. This is a per-pod operation and doesn't propagate over streaming replication.
- Verify each pod can restart cleanly using the new key before removing access to the old key.
- Keep the old KMS key available and accessible until every pod has been rotated and successfully restarted. Deleting the old key prematurely leaves any un-rotated pod unable to start.
For step 2, follow the guidance for your Postgres distribution (EDB Postgres Advanced Server or EDB Postgres Extended Server) for the exact command that rewrites the on-disk wrapped key. Contact EDB support if you require assistance planning or executing rotation across a multi-DC deployment.
3. Rotation across a multi-DC deployment per-topology
Replica clusters (PSR): the primary's KEK is the KEK of record for the entire cluster (see Replica clusters across data centers). If you rotate on the primary, you must also rotate on every replica. If you rotate a replica but not the primary, the replica's on-disk encryption file — copied from the primary at bootstrap — remains under the primary's KEK, so the replica's new KEK doesn't actually take effect until the file itself is rewritten by the rotation operation on that replica's pod. Plan the rotation as: primary first, then each replica in turn.
Distributed high-availability (PGD) clusters: each data group's KEK is independent. You can rotate one group's KEK without touching the others. Rotate each group's KEK on the pods of that group only.
What to prepare before rotating
- Register the new customer key in every affected location in HM before you begin.
- Verify the new key is reachable from every pod that will need to unwrap under it (the workload identity or credentials attached to the pod must have decrypt permission on the new key).
- Take a fresh backup of the cluster.
- Plan a maintenance window; each pod will need to restart under the new key.
- Don't delete the old key in your KMS provider until every pod has been rotated and has successfully restarted at least once under the new key.
Summary
| Scenario | Can each location use a different KEK material? |
|---|---|
| Primary cluster with replica cluster(s) in different locations | No. All locations must resolve to compatible key material. The primary's KEK is the effective KEK for every replica. |
| Distributed high-availability cluster with data groups in different locations, each configured with its own customer key | Only the seed group's material is independent. Every group added later by physical join must resolve to key material compatible with the group it joined from — the same requirement as a PSR replica. There's no re-wrap step; the joining group's own configured KEK is what Postgres uses to unwrap the copied file at every startup. |
| Adding a replica cluster to a single-region cluster later | Same rule as above, the primary's KEK protects the new replica's data at rest. |
| Adding a data group to a distributed high-availability cluster later | No. The new group's key.bin is copied as-is from the source group with no re-wrap; the new group's KEK must be compatible with the source's. Only the new group's own KEK needs to be reachable from its location — the physical join never contacts the source's key vault. |
| KEK rotation on the primary of a cluster with replicas | Must be run on the primary and repeated on each replica. |
| KEK rotation on a distributed high-availability data group | Local to that group only. |