Observability dashboards and alerts configuration

Connectivity Link includes starting points for monitoring your Connectivity Link deployment with ready-to-use example dashboards and alerts. You can customize these dashboards and alerts to fit your environment.

Dashboards are organized with different metrics for different use cases.

Platform engineer Grafana dashboard

The platform engineer dashboard displays the following details:

  • Policy compliance and governance.

  • Resource consumption

  • Error rates

  • Request latency and throughput

  • Multi-window, multi-burn alert templates for API error rates and latency

  • Multicluster split

Application developer Grafana dashboard

The application developer dashboard is less focused on policies than the platform engineer dashboard and is more focused on APIs and applications. For example:

  • Request latency and throughput per API

  • Total requests and error rates by API path

Business user Grafana dashboard

The business user dashboard includes the following details:

  • Requests per second per API

  • Increase or decrease in rates of API usage over specified times

Grafana dashboards available to import

The Connectivity Link example dashboards are uploaded to the Grafana dashboards website. You can import the following dashboards into your Grafana deployment on OpenShift Container Platform:

Table 8. Connectivity Link example dashboards in Grafana
Name Dashboard ID

App Developer dashboard

21538

Platform Engineer dashboard

20982

Business User dashboard

20981

DNS Operator dashboard

22695

Import dashboards in Grafana

You can manually select and important dashboards to Grafana to conduct rapid prototyping or emergency troubleshooting, test community dashboards, or perfect a dashboard that you intend to automate for another team.

Important

You must perform these steps on each OpenShift Container Platform cluster that you want to use Connectivity Link on.

Prerequisites
  • You configured your monitoring stack and other observability resources as needed.

  • You installed Connectivity Link.

  • You have administrator access to a running OpenShift Container Platform cluster.

Procedure

Workflow to import dashboards automatically in OpenShift Container Platform

Automating the import of observability dashboards can give you more consistency, version control, and operational velocity. Automation gives you the benefits of monitoring-as-code, and helps keep Operators updated, clusters identical, and supports multi-tenancy.

  • You can use a GrafanaDashboard resource to reference a ConfigMap.

    Data sources are configured as template variables, automatically integrating with your existing data sources. The metrics for these dashboards are sourced from Prometheus.

    Important

    For some example dashboard panels to work correctly, HTTPRoute objects in Connectivity Link must include a service and deployment label with a value that matches the name of the service and deployment being routed to. For example, service=my-app and deployment=my-app. This allows low-level Istio and Envoy metrics to be added to Gateway API state metrics.

  • If you do not want to use the GUI, you can automate dashboard provisioning in Grafana by adding JSON files to a ConfigMap object that you must mount at /etc/grafana/provisioning/dashboards.

Prometheus alert configuration reference

You can configure Prometheus alerts in OpenShift Container Platform is a proactive way to tune alerts so that you can ensure platform stability. For example, you can set alert triggers for automated incident detection, usage, and cluster health.

  • You can integrate the Connectivity Link example alerts into Prometheus as PrometheusRule resources, and then adjust the alert thresholds to suit your specific operational needs.

  • For details on how to configure Prometheus alerts, see Configuring alerts and notifications for user workload monitoring.

  • Service Level Objective (SLO) alerts generated by using the Sloth GitHub project are also included. You can use these alerts to integrate with the SLO Grafana dashboard, which uses generated labels to comprehensively overview your SLOs.