WarehousePG Copy command reference

The whpg-copy command copies objects from databases in a source WarehousePG (WHPG) cluster to databases in a destination WarehousePG cluster.

Synopsis

whpg-copy <subcommand> [global-options]

Subcommands

Copy

Copy data from one WarehousePG (WHPG) cluster to another.

whpg-copy copy
   -s | --src-url <url> 
   -d | --dst-url <url> 
   [-c | --config-file <path>]
   [--src-db <database_name>]
   [--dst-db <database_name>]
   [-i | --include-table <schema.table>]
   [-e | --exclude-table <schema.table>]
   [--compression <true|false>]
   [-p | --through-partition-leaves <true|false>]
   [--target-mode <append|skip-existing|truncate>]
   [--validate-method <none|count|checksum>]
   [--port-range <min>-<max>]
   [--workers <number>]
   [--timeout <seconds>]
   [--dry-run]

The subcommand copy supports the following options:

Source and destination clusters and databases

• --src-url <url> or -s <url>: Connection string for the source database. It follows the format postgres://[user@]host[:port][/dbname].
• --dst-url <url> or -d <url>: Connection string for the destination database. It follows the format postgres://[user@]host[:port][/dbname].
• --src-db <DB_NAME>: The source database. It overrides the database name specified in --src-url.
• --dst-db <DB_NAME>: The destination database. It overrides the database name specified in --dst-url.

Relations to copy

You can selectively include or exclude tables. If you don't provide this option, the utility copies all user tables in the source database.

• --include-table <table> or -i <table>: Specifies a table to include. Can be used multiple times. Use the format schema.table to specify the relations. If you are using special characters, follow PostgreSQL's qualified identifier rules to quote them.
• --exclude-table <table> or -e <table>: Specifies a table to exclude. Can be used multiple times. Uses the same format as --include-table.

Other options

Customize the data transfer behavior with these options:

• --config-file <path> or -c <path>: Specify configuration options via a TOML configuration file. See whpg-copy configuration file for the full list of options. Note that command line options will override the configuration file.

• --target-mode <mode>: Determines how to handle existing tables on the destination. The supported options are:

  • append (Default): Inserts data into existing tables.
  • truncate: Truncates the destination table before copying.
  • skip-existing: Skips the copy operation if the table already exists.

• --compression <bool>: Enables or disables ZSTD compression during data transfer. Default is true.

• --through-partition-leaves <bool> or -p <bool>: If true (default), copies data directly between leaf partitions in parallel. If false, data goes through the specified root/intermediate partition table.

• --validate-method <method>: Validation to perform after copying. The supported options are:

  • none (Default): No validation.
  • count: Compares row counts.
  • checksum: Calculates and compares data hashes.

• --dry-run: If set, performs a trial run without modifying data on the destination.

• --timeout <seconds>: Connection timeout in seconds. Default is 0 (never times out).

• --workers <number>: Specifies the number of concurrent worker tasks. Default is 4. Use the keys + or - to dynamically increase or decrease the number of parallel workers during the copy process.

Diagnose

Identify blocked connections or routing issues between WHPG clusters.

whpg-copy diagnose
   -s | --src-url <url>
   - d | --dst-url <url>
   [--port-range <min-max>]
   [--timeout <seconds>]

The subcommand diagnose supports the following options:

• --src-url <url> or -s <url>: Connection string for the source database. It follows the format postgres://[user[:password]@]host[:port][/dbname].
• --dst-url <url> or -d <url>: Connection string for the destination database. It follows the format postgres://[user[:password]@]host[:port][/dbname].
• --port-range <min-max>: Range of ports available to scan sequentially until an available port is found.
• --timeout <seconds>: Connection timeout in seconds. Default is 0 (never times out).

Config-example

Generate a sample TOML-formatted configuration file to use with the whpg-copy copy command.

whpg-copy config-example

See whpg-copy configuration file for the full list of all parameters and values.

Version

Display the current version of the whpg-copy utility.

whpg-copy version

Global options

The following options can be used with any of the whpg-copy subcommands:

• --help: Displays command help.