Using Nvidia NIM models in the Nvidia cloud

To use a Nvidia NIM that's hosted in Nvidia's cloud, you first need to select a model to use. This tutorial uses the Nvidia NIM model llama-3.3-70b-instruct.

Prerequisites

• An Nvidia NGC account. (If you don't have one, you can create one here.)

Configuring the Nvidia cloud

1. Select a model

Choose a model from Nvidia's model library. This example uses the llama-3.3-70b-instruct model.

2. Generate an API Key

You can create API keys from the model’s page.

To generate an API key, select Get API Key. If you haven't logged in, you're prompted to log in. If you haven't signed up, you're invited to do so. You'll receive an API key that you can use to interact with the model. This example refers to this key as the <NIM API KEY>.

Integrating the model with AI Accelerator

1. Enable AI Accelerator in EDB Postgres AI

With EDB Postgres Advanced Server, EDB Postgres Extended Server, or a community Postgres instance running, connect to the database and enable the AI Accelerator extension:

CREATE EXTENSION aidb CASCADE;

2. Register the model

SELECT aidb.create_model( \
    'my_nim_llm', \
    'nim_completions', \
    '{"model": "meta/llama-3.3-70b-instruct"}', \
    '{"api_key":"<NIM API KEY>"}'::JSONB \
);

3. Run the model

To interact with the model, execute the following query:

SELECT aidb.decode_text('my_nim_llm', 'Tell me a short, one sentence story');

                                          decode_text                                       
    ----------------------------------------------------------------------------------------
     As the clock struck midnight, a single tear fell from the porcelain doll's glassy eye.

Your output may vary. You've successfully used Nvidia NIM models, running on Nvidia's cloud, integrated with AI Accelerator.