Enabling alert routing for user-defined projects

In OKD 4, a cluster administrator can enable alert routing for user-defined projects. This process consists of two general steps:

• Enable alert routing for user-defined projects to use the default platform Alertmanager instance or, optionally, a separate Alertmanager instance only for user-defined projects.

• Grant users permission to configure alert routing for user-defined projects.

After you complete these steps, developers and other users can configure custom alerts and alert routing for their user-defined projects.

Understanding alert routing for user-defined projects

As a cluster administrator, you can enable alert routing for user-defined projects. With this feature, you can allow users with the alert-routing-edit role to configure alert notification routing and receivers for user-defined projects. These notifications are routed by the default Alertmanager instance or, if enabled, an optional Alertmanager instance dedicated to user-defined monitoring.

Users can then create and configure user-defined alert routing by creating or editing the AlertmanagerConfig objects for their user-defined projects without the help of an administrator.

After a user has defined alert routing for a user-defined project, user-defined alert notifications are routed as follows:

• To the alertmanager-main pods in the openshift-monitoring namespace if using the default platform Alertmanager instance.

• To the alertmanager-user-workload pods in the openshift-user-workload-monitoring namespace if you have enabled a separate instance of Alertmanager for user-defined projects.

Enabling the platform Alertmanager instance for user-defined alert routing

You can allow users to create user-defined alert routing configurations that use the main platform instance of Alertmanager.

Prerequisites

• You have access to the cluster as a user with the cluster-admin cluster role.

• You have installed the OpenShift CLI (oc).

Procedure

1. Edit the cluster-monitoring-config ConfigMap object:

   $ oc -n openshift-monitoring edit configmap cluster-monitoring-config

2. Add enableUserAlertmanagerConfig: true in the alertmanagerMain section under data/config.yaml:

   apiVersion: v1
   kind: ConfigMap
   metadata:
     name: cluster-monitoring-config
     namespace: openshift-monitoring
   data:
     config.yaml: |
       alertmanagerMain:
         enableUserAlertmanagerConfig: true (1)

   1	Set the enableUserAlertmanagerConfig value to true to allow users to create user-defined alert routing configurations that use the main platform instance of Alertmanager.

3. Save the file to apply the changes.

Enabling a separate Alertmanager instance for user-defined alert routing

In some clusters, you might want to deploy a dedicated Alertmanager instance for user-defined projects, which can help reduce the load on the default platform Alertmanager instance and can better separate user-defined alerts from default platform alerts. In these cases, you can optionally enable a separate instance of Alertmanager to send alerts for user-defined projects only.

Prerequisites

• You have access to the cluster as a user with the cluster-admin cluster role.

• You have enabled monitoring for user-defined projects in the cluster-monitoring-config config map for the openshift-monitoring namespace.

• You have installed the OpenShift CLI (oc).

Procedure

1. Edit the user-workload-monitoring-config ConfigMap object:

   $ oc -n openshift-user-workload-monitoring edit configmap user-workload-monitoring-config

2. Add enabled: true and enableAlertmanagerConfig: true in the alertmanager section under data/config.yaml:

   apiVersion: v1
   kind: ConfigMap
   metadata:
     name: user-workload-monitoring-config
     namespace: openshift-user-workload-monitoring
   data:
     config.yaml: |
       alertmanager:
         enabled: true (1)
         enableAlertmanagerConfig: true (2)

   1	Set the enabled value to true to enable a dedicated instance of the Alertmanager for user-defined projects in a cluster. Set the value to false or omit the key entirely to disable the Alertmanager for user-defined projects. If you set this value to false or if the key is omitted, user-defined alerts are routed to the default platform Alertmanager instance.
   2	Set the enableAlertmanagerConfig value to true to enable users to define their own alert routing configurations with AlertmanagerConfig objects.

3. Save the file to apply the changes. The dedicated instance of Alertmanager for user-defined projects starts automatically.

Verification

• Verify that the user-workload Alertmanager instance has started:

  # oc -n openshift-user-workload-monitoring get alertmanager

  Example output

  NAME            VERSION   REPLICAS   AGE
  user-workload   0.24.0    2          100s

Granting users permission to configure alert routing for user-defined projects

You can grant users permission to configure alert routing for user-defined projects.

Prerequisites

• You have access to the cluster as a user with the cluster-admin cluster role.

• You have enabled monitoring for user-defined projects in the cluster-monitoring-config config map for the openshift-monitoring namespace.

• The user account that you are assigning the role to already exists.

• You have installed the OpenShift CLI (oc).

Procedure

• Assign the alert-routing-edit cluster role to a user in the user-defined project:

  $ oc -n <namespace> adm policy add-role-to-user alert-routing-edit <user> (1)

  1	For <namespace>, substitute the namespace for the user-defined project, such as ns1. For <user>, substitute the username for the account to which you want to assign the role.

Additional resources

• Enabling monitoring for user defined projects

• Creating alert routing for user-defined projects

Next steps

• Managing alerts