Monitoring with Hybrid Manager v1.3

With Hybrid Manager (HM), you can collect a wide range of metrics (host metrics, Kubernetes metrics, Postgres database data, query data, and logging data). You can visualize the collected data with the HM console or Grafana instance to analyze trends, configure alerting, improve the efficiency of databases, perform troubleshooting, and much more.

Understanding the monitoring architecture

HM has three main internal pipelines that provide monitoring capabilities:

• Prometheus and Thanos collect metrics.
• Loki collects logs.
• Stats Collector collects query data.

All these services come with their own dedicated stores, which means that the collected data is kept in HM. These services and stores are deployed by default.

These collected metrics become available to you through two interfaces:

• The Grafana dashboard.
• The HM console, for example, in the Estate tab or Cluster view.

Enabling database monitoring

The monitoring services that come with HM require access to the databases you want to monitor, as well as some extensions to enable Stats Collector.

• For HM-managed clusters (created with the HM), the default setup provides access to the monitoring stack.

• For external self-managed clusters, each of the databases must be registered with the HM agent to enable data collection. By default, the agent provides the configuration to collect Prometheus, Thanos, and Loki data. Additionally, the agent provides configuration parameters so you can control whether to enable or disable the extensions required for Stats Collector (collection of query data), namely edb_wait_states_data, edb_stat_monitor, edb_query_advisor, and pg_stat_statements.

• For external cloud service provider (CSP)-managed databases, such as AWS RDS, HM needs to get access to the database through a Kubernetes secret resource to enable data collection. See Monitoring CSP-managed databases for more information. Query data with Stats Collector isn't currently available for CSP-managed databases.

Monitoring data ingestion and scope

Because of the different levels of access HM has to the variety of cluster types, the way that data is ingested and the scope of collected data varies.

• For HM-managed clusters (created with HM), the databases push host metrics, Postgres metrics, cluster metrics, query metrics, data required for recommendations, and logs to the corresponding Prometheus, Thanos, Loki, and Stats Collector stores.

• For external self-managed databases, the agent pushes host metrics, Postgres metrics, cluster metrics, query metrics, data required for recommendations, and logs to the corresponding Prometheus, Thanos, and Stats Collector stores at configurable intervals.

• For external CSP-managed databases, such as AWS RDS, HM leverages the HM-to-database connection to pull host and Postgres metrics to the central stores. Query data with Stats Collector isn't currently available for CSP-managed databases.

For more information, see Metrics support scope.

Monitoring refresh frequency

The refresh intervals for the metrics that are collected by HM vary depending on the metric and database type. This difference is due to the varying methods of data collection and integration of Prometheus/Thanos, Loki, and Stats Collector services.

See Metrics refresh intervals to understand how often metrics are updated and how to customize the frequency.

Connecting data into external tools

Currently, EDB doesn't support plugging metrics and log data collected by HM into external tools or enterprise systems.

Related topics

• Enabling monitoring ► Install the agent or connect your database to HM to enable monitoring on external databases.
• Metrics refresh intervals ► Understand how often metrics are updated and how to customize the frequency.
• Monitoring with Grafana ► Use Grafana to monitor your databases.
• Monitoring with HM console ► Use the HM console to monitor your databases.
• Notifications ► Configure notifications.