Configuring and running the EDB DMS Reader

Suggest edits

Create migration credentials to enable the EDB DMS Reader on your auxiliary machine to connect to the Hybrid Manager for migrations. You need one set of credentials per destination Hybrid Manager project.

For example, if you are using a single Hybrid Manager project for all migrations, you only need one set of credentials on your auxiliary machine.

  1. On the Hybrid Manager Portal, select the project you are using for your migration activities.

  2. Within your project, select Migrate > Credentials.

  3. Select Create Migration Credential, enter a name for your credentials, and select Create Migration Credential.

  4. After the credentials are generated, select Download Credential.

    Note

    This is the only opportunity you have to download the credentials. If you don't download them at this time, you will have to revoke them and create new credentials.

  5. Unzip the downloaded zip file and copy the generated folder to the host where the reader is installed.

Configuring the reader

  1. Ensure your machine has Java/OpenJDK 17 or later installed.

  2. Open the EDB DMS Reader located in /opt/cdcreader/run-cdcreader.sh and ensure you have write permissions.

  3. Set the variables according to your environment and uncomment the edited lines. See parameters for further guidance. The script is reproduced below.

#!/bin/bash -e
# run_cdcreader.sh
#
# This script provides a convenient place to specify
# environment variables used to configure the
# EDB Data Migration Service Reader.
#
# After env exports, `java` is called to start the
# software.

##########################################
# DMS Reader General Configuration       #
##########################################

# This ID is used to identify DMS Reader
# and is specified by the user.
#export DBCONFIG_ID=

# This is the DMS backend service used by the Reader
# Consult your system administrators
#export RW_SERVICE_HOST=https://transporter-rw-service.yourdomain.hcp

# You need to create migration credentials in EDB Postgres AI platform
# and set these fields with the path of the credential files
#export TLS_PRIVATE_KEY_PATH=$HOME/credentials/client-key.pem
#export TLS_CERTIFICATE_PATH=$HOME/credentials/client-cert.pem
#export TLS_CA_PATH=$HOME/credentials/int.crt
#export APICURIOREQUEST_CLIENT_KEYSTORE_LOCATION=$HOME/credentials/client.keystore.p12
#export APICURIOREQUEST_TRUSTSTORE_LOCATION=$HOME/credentials/int.truststore.p12
#export KAFKASECURITY_CLIENT_KEYSTORE_LOCATION=$HOME/credentials/client.keystore.p12
#export KAFKASECURITY_TRUSTSTORE_LOCATION=$HOME/credentials/int.truststore.p12

##########################################
# DMS Reader Source DB Configuration     #
##########################################

# A sample configuration to create a single postgres database connection:
#export DBCONFIG_DATABASES_0__TYPE=POSTGRES
#export DBCONFIG_DATABASES_0__HOSTNAME=localhost
#export DBCONFIG_DATABASES_0__PORT=5432
# The CATALOG is the database name
#export DBCONFIG_DATABASES_0__CATALOG=source
#export DBCONFIG_DATABASES_0__USERNAME=postgres
# The password env can be set without specifying it here
# but the env structure looks like this
#export DBCONFIG_DATABASES_0__PASSWORD=password

# You can increase the index to configure more than
# one database for the DMS Reader
#export DBCONFIG_DATABASES_1__TYPE=ORACLE
#export DBCONFIG_DATABASES_1__HOSTNAME=localhost
#export DBCONFIG_DATABASES_1__PORT=1521
# The CATALOG is the database name
#export DBCONFIG_DATABASES_1__CATALOG=ORCLCDB/ORCLPDB1
#export DBCONFIG_DATABASES_1__USERNAME=oracle
# The password env can be set without specifying it here
# but the env structure looks like this
#export DBCONFIG_DATABASES_1__PASSWORD=password

##########################################
# Optional Parameters Below              #
##########################################

# FIPS mode can be enabled by setting the FIPS_MODE env variable to true
#export FIPS_MODE=true

# Configure logging
# Generic loglevel
#export QUARKUS_LOG_LEVEL=DEBUG
# Loglevel for a single package
#export QUARKUS_LOG_CATEGORY__COM_ENTERPRISEDB__LEVEL=DEBUG

cd "$(dirname "$0")"

java -jar quarkus-run.jar

Parameters

DMS Reader general configuration

These parameters define the information the EDB DMS Reader requires to access the Hybrid Manager instance and project that you are using for the migration.

DBCONFIG_ID

This is the name you assign to identify a source. This name will later appear as a source in the Migrate > Sources section of the EDB Postgres AI Console.

Consider the following ID guidelines:

  • The maximum character length for the ID is 255 characters.
  • You can use lowercase and uppercase characters, numbers, underscores(_) and hyphens(-) for the ID. Other special characters are not supported.
  • The ID must be unique within a project. Do not use the same ID for two or more sources in the same project.

RW_SERVICE_HOST

Specifies the URL of the service that will host the migration. Set RW_SERVICE_HOST to the domain name or host associated with the Hybrid Manager ingress.

Derive RW_SERVICE_HOST from the TRANSPORTER_RW_SERVICE_DOMAIN_NAME that was assigned by the administrators or installers of your Hybrid Manager instance via the values.yaml file. Alternatively, derive RW_SERVICE_HOST from the URL you use to access the Hybrid Manager Console.

Example: If the URL for the Hybrid Manager Console is https://portal.foo-bar.enterprise.network, the RW_SERVICE_HOST will be https://transporter-rw-service.foo-bar.enterprise.network/transporter where you replace portal with transporter-rw-service and add /transporter at the end of the endpoint.

TLS_PRIVATE_KEY_PATH

Directory path to the client-key.pem private key you downloaded from the EDB Postgres AI Console.

The HTTP client of the EDB DMS Reader uses it to perform mTLS authentication with the transporter-rw-service.

TLS_CERTIFICATE_PATH

Directory path to the X509 client-cert.pem certificate you downloaded from the EDB Postgres AI Console.

The HTTP client of the EDB DMS Reader uses it to perform mTLS authentication with the transporter-rw-service.

TLS_CA_PATH

Directory path to the int.cert Certificate Authority you downloaded from the EDB Postgres AI Console.

It signs the certificate configured in TLS_CERTIFICATE_PATH.

APICURIOREQUEST_CLIENT_KEYSTORE_LOCATION

File path to the client-keystore.p12 keystore location file you downloaded from the Hybrid Manager. It is created from the private key and certificate configured in TLS_PRIVATE_KEY_PATH and TLS_CERTIFICATE_PATH.

The Apicurio client uses it to perform mTLS authentication with the transporter-rw-service.

APICURIOREQUEST_TRUSTSTORE_LOCATION

Created from the Certificate Authority configured in TLS_CA_PATH.

The Apicurio client uses it to perform mTLS authentication with the transporter-rw-service.

KAFKASECURITY_CLIENT_KEYSTORE_LOCATION

File path to the client-keystore.p12 keystore location file you downloaded from the Hybrid Manager (same path as APICURIOREQUEST_CLIENT_KEYSTORE_LOCATION).

This will be required for Kafka client authentication later when the Hybrid Manager creates Kafka pods to perform the migration.

KAFKASECURITY_TRUSTSTORE_LOCATION

File path to the int.truststore.p12 location file you downloaded from the Hybrid Manager (same path as APICURIOREQUEST_TRUSTSTORE_LOCATION).

This will be required for Kafka client authentication later when the Hybrid Manager creates Kafka pods to perform the migration.

DMS Reader source database configuration

These parameters (DBCONFIG_DATABASES section) define a list of source database information you require for the EDB DMS Reader to be able to read the correct source database information for the migration.

You can configure the EDB DMS Reader to migrate multiple databases. The DBCONFIG_DATABASES_0__TYPE section delimits the information for the first database. You can use DBCONFIG_DATABASES_1__TYPE to provide data for a second database. Add more sections to the EDB DMS Reader (DBCONFIG_DATABASES_2__TYPE, DBCONFIG_DATABASES_3__TYPE) by increasing the index manually.

DBCONFIG_DATABASES_0__TYPE

This is the source database type. EDB DMS reader supports ORACLE and POSTGRES.

DBCONFIG_DATABASES_0__HOSTNAME

The hostname of the source database.

Note

For RDS databases, ensure the DBCONFIG_DATABASES_0__HOSTNAME value points to the RDS endpoint you can find under the Connectivity & security in the AWS Console. A redacted example is foo_database.cb7asw1xxxxx.eu-west-0.rds.amazonaws.com.

DBCONFIG_DATABASES_0__PORT

The port of the source database.

DBCONFIG_DATABASES_0__CATALOG

The database name in the source database server.

DBCONFIG_DATABASES_0__USERNAME

The database username of the source database.

DBCONFIG_DATABASES_0__PASSWORD

The password for the database username of the source database.

FIPS_MODE

If you need to comply with FIPS standards, enable FIPS mode for the DMS Reader by setting the environment variable to true: export FIPS_MODE=true.

Running the EDB DMS Reader

  1. Start the migration:

    cd /opt/cdcreader
./run-cdcreader.sh

  2. Go to the Hybrid Manager Console, and verify that a source with the DBCONFIG_ID= name is displayed in Migrate > Sources.

You can select this source for your migration.

← Prev

Preparing AWS RDS source databases

↑ Up

Getting started

Next →

Configuring and running the EDB DMS Writer

Could this page be better? Report a problem or suggest an addition!