Configuring the Reporting Operator

Metering is a deprecated feature. Deprecated functionality is still included in OKD and continues to be supported; however, it will be removed in a future release of this product and is not recommended for new deployments.

For the most recent list of major functionality that has been deprecated or removed within OKD, refer to the Deprecated and removed features section of the OKD release notes.

The Reporting Operator is responsible for collecting data from Prometheus, storing the metrics in Presto, running report queries against Presto, and exposing their results via an HTTP API. Configuring the Reporting Operator is primarily done in your MeteringConfig custom resource.

Securing a Prometheus connection

When you install metering on OKD, Prometheus is available at https://prometheus-k8s.openshift-monitoring.svc:9091/.

To secure the connection to Prometheus, the default metering installation uses the OKD certificate authority (CA). If your Prometheus instance uses a different CA, you can inject the CA through a config map. You can also configure the Reporting Operator to use a specified bearer token to authenticate with Prometheus.

Procedure

  • Inject the CA that your Prometheus instance uses through a config map. For example:

    1. spec:
    2.   reporting-operator:
    3.     spec:
    4.       config:
    5.         prometheus:
    6.           certificateAuthority:
    7.             useServiceAccountCA: false
    8.             configMap:
    9.               enabled: true
    10.               create: true
    11.               name: reporting-operator-certificate-authority-config
    12.               filename: "internal-ca.crt"
    13.               value: |
    14.                 -----BEGIN CERTIFICATE-----
    15.                 (snip)
    16.                 -----END CERTIFICATE-----

    Alternatively, to use the system certificate authorities for publicly valid certificates, set both useServiceAccountCA and configMap.enabled to false.

  • Specify a bearer token to authenticate with Prometheus. For example:

  1. spec:
  2.   reporting-operator:
  3.     spec:
  4.       config:
  5.         prometheus:
  6.           metricsImporter:
  7.             auth:
  8.               useServiceAccountToken: false
  9.               tokenSecret:
  10.                 enabled: true
  11.                 create: true
  12.                 value: "abc-123"

Exposing the reporting API

On OKD the default metering installation automatically exposes a route, making the reporting API available. This provides the following features:

  • Automatic DNS

  • Automatic TLS based on the cluster CA

Also, the default installation makes it possible to use the OpenShift service for serving certificates to protect the reporting API with TLS. The OpenShift OAuth proxy is deployed as a sidecar container for the Reporting Operator, which protects the reporting API with authentication.

Using OpenShift Authentication

By default, the reporting API is secured with TLS and authentication. This is done by configuring the Reporting Operator to deploy a pod containing both the Reporting Operator’s container, and a sidecar container running OpenShift auth-proxy.

To access the reporting API, the Metering Operator exposes a route. Once that route has been installed, you can run the following command to get the route’s hostname.

  1. $ METERING_ROUTE_HOSTNAME=$(oc -n openshift-metering get routes metering -o json | jq -r '.status.ingress[].host')

Next, set up authentication using either a service account token or basic authentication with a username and password.

Authenticate using a service account token

With this method, you use the token in the Reporting Operator’s service account, and pass that bearer token to the Authorization header in the following command:

  1. $ TOKEN=$(oc -n openshift-metering serviceaccounts get-token reporting-operator)
  2. curl -H "Authorization: Bearer $TOKEN" -k "https://$METERING_ROUTE_HOSTNAME/api/v1/reports/get?name=[Report Name]&namespace=openshift-metering&format=[Format]"

Be sure to replace the name=[Report Name] and format=[Format] parameters in the URL above. The format parameter can be json, csv, or tabular.

Authenticate using a username and password

Metering supports configuring basic authentication using a username and password combination, which is specified in the contents of an htpasswd file. By default, a secret containing empty htpasswd data is created. You can, however, configure the reporting-operator.spec.authProxy.htpasswd.data and reporting-operator.spec.authProxy.htpasswd.createSecret keys to use this method.

Once you have specified the above in your MeteringConfig resource, you can run the following command:

  1. $ curl -u testuser:password123 -k "https://$METERING_ROUTE_HOSTNAME/api/v1/reports/get?name=[Report Name]&namespace=openshift-metering&format=[Format]"

Be sure to replace testuser:password123 with a valid username and password combination.

Manually Configuring Authentication

To manually configure, or disable OAuth in the Reporting Operator, you must set spec.tls.enabled: false in your MeteringConfig resource.

This also disables all TLS and authentication between the Reporting Operator, Presto, and Hive. You would need to manually configure these resources yourself.

Authentication can be enabled by configuring the following options. Enabling authentication configures the Reporting Operator pod to run the OpenShift auth-proxy as a sidecar container in the pod. This adjusts the ports so that the reporting API isn’t exposed directly, but instead is proxied to via the auth-proxy sidecar container.

  • reporting-operator.spec.authProxy.enabled

  • reporting-operator.spec.authProxy.cookie.createSecret

  • reporting-operator.spec.authProxy.cookie.seed

You need to set reporting-operator.spec.authProxy.enabled and reporting-operator.spec.authProxy.cookie.createSecret to true and reporting-operator.spec.authProxy.cookie.seed to a 32-character random string.

You can generate a 32-character random string using the following command.

  1. $ openssl rand -base64 32 | head -c32; echo.

Token authentication

When the following options are set to true, authentication using a bearer token is enabled for the reporting REST API. Bearer tokens can come from service accounts or users.

  • reporting-operator.spec.authProxy.subjectAccessReview.enabled

  • reporting-operator.spec.authProxy.delegateURLs.enabled

When authentication is enabled, the Bearer token used to query the reporting API of the user or service account must be granted access using one of the following roles:

  • report-exporter

  • reporting-admin

  • reporting-viewer

  • metering-admin

  • metering-viewer

The Metering Operator is capable of creating role bindings for you, granting these permissions by specifying a list of subjects in the spec.permissions section. For an example, see the following advanced-auth.yaml example configuration.

  1. apiVersion: metering.openshift.io/v1
  2. kind: MeteringConfig
  3. metadata:
  4.   name: "operator-metering"
  5. spec:
  6.   permissions:
  7.     # anyone in the "metering-admins" group can create, update, delete, etc any
  8.     # metering.openshift.io resources in the namespace.
  9.     # This also grants permissions to get query report results from the reporting REST API.
  10.     meteringAdmins:
  11.     - kind: Group
  12.       name: metering-admins
  13.     # Same as above except read only access and for the metering-viewers group.
  14.     meteringViewers:
  15.     - kind: Group
  16.       name: metering-viewers
  17.     # the default serviceaccount in the namespace "my-custom-ns" can:
  18.     # create, update, delete, etc reports.
  19.     # This also gives permissions query the results from the reporting REST API.
  20.     reportingAdmins:
  21.     - kind: ServiceAccount
  22.       name: default
  23.       namespace: my-custom-ns
  24.     # anyone in the group reporting-readers can get, list, watch reports, and
  25.     # query report results from the reporting REST API.
  26.     reportingViewers:
  27.     - kind: Group
  28.       name: reporting-readers
  29.     # anyone in the group cluster-admins can query report results
  30.     # from the reporting REST API. So can the user bob-from-accounting.
  31.     reportExporters:
  32.     - kind: Group
  33.       name: cluster-admins
  34.     - kind: User
  35.       name: bob-from-accounting
  37.   reporting-operator:
  38.     spec:
  39.       authProxy:
  40.         # htpasswd.data can contain htpasswd file contents for allowing auth
  41.         # using a static list of usernames and their password hashes.
  42.         #
  43.         # username is 'testuser' password is 'password123'
  44.         # generated htpasswdData using: `htpasswd -nb -s testuser password123`
  45.         # htpasswd:
  46.         #   data: |
  47.         #     testuser:{SHA}y/2sYAj5yrQIN4TL0YdPdmGNKpc=
  48.         #
  49.         # change REPLACEME to the output of your htpasswd command
  50.         htpasswd:
  51.           data: |
  52.             REPLACEME

Alternatively, you can use any role which has rules granting get permissions to reports/export. This means get access to the export sub-resource of the Report resources in the namespace of the Reporting Operator. For example: admin and cluster-admin.

By default, the Reporting Operator and Metering Operator service accounts both have these permissions, and their tokens can be used for authentication.

Basic authentication with a username and password

For basic authentication you can supply a username and password in the reporting-operator.spec.authProxy.htpasswd.data field. The username and password must be the same format as those found in an htpasswd file. When set, you can use HTTP basic authentication to provide your username and password that has a corresponding entry in the htpasswdData contents.