Configuring the Cluster Observability Operator to monitor a service

The Cluster Observability Operator is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production. These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process.

For more information about the support scope of Red Hat Technology Preview features, see Technology Preview Features Support Scope.

You can monitor metrics for a service by configuring monitoring stacks managed by the Cluster Observability Operator (COO).

To test monitoring a service, follow these steps:

  • Deploy a sample service that defines a service endpoint.

  • Create a ServiceMonitor object that specifies how the service is to be monitored by the COO.

  • Create a MonitoringStack object to discover the ServiceMonitor object.

Deploying a sample service for Cluster Observability Operator

This configuration deploys a sample service named prometheus-coo-example-app in the user-defined ns1-coo project. The service exposes the custom version metric.

Prerequisites

  • You have access to the cluster as a user with the cluster-admin cluster role or as a user with administrative permissions for the namespace.

Procedure

  1. Create a YAML file named prometheus-coo-example-app.yaml that contains the following configuration details for a namespace, deployment, and service:

    1. apiVersion: v1
    2. kind: Namespace
    3. metadata:
    4.   name: ns1-coo
    5. ---
    6. apiVersion: apps/v1
    7. kind: Deployment
    8. metadata:
    9.   labels:
    10.     app: prometheus-coo-example-app
    11.   name: prometheus-coo-example-app
    12.   namespace: ns1-coo
    13. spec:
    14.   replicas: 1
    15.   selector:
    16.     matchLabels:
    17.       app: prometheus-coo-example-app
    18.   template:
    19.     metadata:
    20.       labels:
    21.         app: prometheus-coo-example-app
    22.     spec:
    23.       containers:
    24.       - image: ghcr.io/rhobs/prometheus-example-app:0.4.1
    25.         imagePullPolicy: IfNotPresent
    26.         name: prometheus-coo-example-app
    27. ---
    28. apiVersion: v1
    29. kind: Service
    30. metadata:
    31.   labels:
    32.     app: prometheus-coo-example-app
    33.   name: prometheus-coo-example-app
    34.   namespace: ns1-coo
    35. spec:
    36.   ports:
    37.   - port: 8080
    38.     protocol: TCP
    39.     targetPort: 8080
    40.     name: web
    41.   selector:
    42.     app: prometheus-coo-example-app
    43.   type: ClusterIP

  2. Save the file.

  3. Apply the configuration to the cluster by running the following command:

    1. $ oc apply -f prometheus-coo-example-app.yaml

  4. Verify that the pod is running by running the following command and observing the output:

    1. $ oc -n -ns1-coo get pod

    Example output

    1. NAME                                      READY     STATUS    RESTARTS   AGE
    2. prometheus-coo-example-app-0927545cb7-anskj   1/1       Running   0          81m

Specifying how a service is monitored by Cluster Observability Operator

To use the metrics exposed by the sample service you created in the “Deploying a sample service for Cluster Observability Operator” section, you must configure monitoring components to scrape metrics from the /metrics endpoint.

You can create this configuration by using a ServiceMonitor object that specifies how the service is to be monitored, or a PodMonitor object that specifies how a pod is to be monitored. The ServiceMonitor object requires a Service object. The PodMonitor object does not, which enables the MonitoringStack object to scrape metrics directly from the metrics endpoint exposed by a pod.

This procedure shows how to create a ServiceMonitor object for a sample service named prometheus-coo-example-app in the ns1-coo namespace.

Prerequisites

  • You have access to the cluster as a user with the cluster-admin cluster role or as a user with administrative permissions for the namespace.

  • You have installed the Cluster Observability Operator.

  • You have deployed the prometheus-coo-example-app sample service in the ns1-coo namespace.

    The prometheus-coo-example-app sample service does not support TLS authentication.

Procedure

  1. Create a YAML file named example-coo-app-service-monitor.yaml that contains the following ServiceMonitor object configuration details:

    1. apiVersion: monitoring.rhobs/v1alpha1
    2. kind: ServiceMonitor
    3. metadata:
    4.   labels:
    5.     k8s-app: prometheus-coo-example-monitor
    6.   name: prometheus-coo-example-monitor
    7.   namespace: ns1-coo
    8. spec:
    9.   endpoints:
    10.   - interval: 30s
    11.     port: web
    12.     scheme: http
    13.   selector:
    14.     matchLabels:
    15.       app: prometheus-coo-example-app

    This configuration defines a ServiceMonitor object that the MonitoringStack object will reference to scrape the metrics data exposed by the prometheus-coo-example-app sample service.

  2. Apply the configuration to the cluster by running the following command:

    1. $ oc apply -f example-app-service-monitor.yaml

  3. Verify that the ServiceMonitor resource is created by running the following command and observing the output:

    1. $ oc -n ns1-coo get servicemonitor

    Example output

    1. NAME                         AGE
    2. prometheus-coo-example-monitor   81m

Creating a MonitoringStack object for the Cluster Observability Operator

To scrape the metrics data exposed by the target prometheus-coo-example-app service, create a MonitoringStack object that references the ServiceMonitor object you created in the “Specifying how a service is monitored for Cluster Observability Operator” section. This MonitoringStack object can then discover the service and scrape the exposed metrics data from it.

Prerequisites

  • You have access to the cluster as a user with the cluster-admin cluster role or as a user with administrative permissions for the namespace.

  • You have installed the Cluster Observability Operator.

  • You have deployed the prometheus-coo-example-app sample service in the ns1-coo namespace.

  • You have created a ServiceMonitor object named prometheus-coo-example-monitor in the ns1-coo namespace.

Procedure

  1. Create a YAML file for the MonitoringStack object configuration. For this example, name the file example-coo-monitoring-stack.yaml.

  2. Add the following MonitoringStack object configuration details:

    Example MonitoringStack object

    1. apiVersion: monitoring.rhobs/v1alpha1
    2. kind: MonitoringStack
    3. metadata:
    4.   name: example-coo-monitoring-stack
    5.   namespace: ns1-coo
    6. spec:
    7.   logLevel: debug
    8.   retention: 1d
    9.   resourceSelector:
    10.     matchLabels:
    11.       k8s-app: prometheus-coo-example-monitor

  3. Apply the MonitoringStack object by running the following command:

    1. $ oc apply -f example-coo-monitoring-stack.yaml

  4. Verify that the MonitoringStack object is available by running the following command and inspecting the output:

    1. $ oc -n ns1-coo get monitoringstack

    Example output

    1. NAME                         AGE
    2. example-coo-monitoring-stack   81m