Pipelines PGFS with Google Cloud Storage

Overview: Google Cloud Storage

PGFS uses the gs: prefix to indicate a Google Cloud Storage bucket.

The general syntax for using GCS is:

select pgfs.create_storage_location(
               'storage_location_name',
               'gs://bucket_name',
               credentials => '{}'
       );

credentials

The credentials argument is optional. It takes a value in JSON format and offers the following settings.

OptionDescription
google_application_credentialsPath to the application credentials file
google_service_account_key_filePath to the service account key file

See the Google Cloud documentation for more information on how to manage service account keys.

You can also set up these options via the equivalent environment variables to facilitate authentication in managed environments such as Google Kubernetes Engine.

Example: private GCS bucket

SELECT pgfs.create_storage_location('edb_ai_example_images', 'gs://my-company-ai-images',
                                    credentials => '{"google_service_account_key_file": "/var/run/gcs.json"}'
       );

Example: authentication in GKE

Ensure that the GOOGLE_APPLICATION_CREDENTIALS or the GOOGLE_SERVICE_ACCOUNT_KEY_FILE environment variable is set on your PostgreSQL pod. Then, PGFS picks them up.

SELECT pgfs.create_storage_location('edb_ai_example_images', 'gs://my-company-ai-images');

Could this page be better? Report a problem or suggest an addition!