Configuring the log curator

You can configure log retention time. That is, you can specify how long the default Elasticsearch log store keeps indices by configuring a separate retention policy for each of the three log sources: infrastructure logs, application logs, and audit logs. For instructions, see Configuring log retention time.

Optionally, to remove Elasticsearch indices that use the data model from OKD 4.4 and earlier, you can also use the Elasticsearch Curator. The following sections explain how to use the Elasticsearch Curator.

Configuring the Curator schedule

You can specify the schedule for Curator using the Cluster Logging custom resource created by the OpenShift Logging installation.

Prerequisites

• Cluster logging and Elasticsearch must be installed.

Procedure

To configure the Curator schedule:

1. Edit the ClusterLogging custom resource in the openshift-logging project:

   $ oc edit clusterlogging instance

   apiVersion: "logging.openshift.io/v1"
   kind: "ClusterLogging"
   metadata:
     name: "instance"
   ...
     curation:
       curator:
         schedule: 30 3 * * * (1)
       type: curator

   1	Specify the schedule for Curator in cron format.

   The time zone is set based on the host node where the Curator pod runs.

Configuring Curator index deletion

You can configure Elasticsearch Curator to delete Elasticsearch data that uses the data model prior to OKD version 4.5. You can configure per-project and global settings. Global settings apply to any project not specified. Per-project settings override global settings.

Prerequisites

• Cluster logging must be installed.

Procedure

To delete indices:

1. Edit the OKD custom Curator configuration file:

   $ oc edit configmap/curator

2. Set the following parameters as needed:

   config.yaml: |
     project_name:
       action
         unit:value

   The available parameters are:

   Table 1. Project options

   Variable Name	Description
   project_name	The actual name of a project, such as myapp-devel. For OKD operations logs, use the name .operations as the project name.
   action	The action to take, currently only delete is allowed.
   unit	The period to use for deletion, days, weeks, or months.
   value	The number of units.

   Table 2. Filter options

   Variable Name	Description
   .defaults	Use .defaults as the project_name to set the defaults for projects that are not specified.
   .regex	The list of regular expressions that match project names.
   pattern	The valid and properly escaped regular expression pattern enclosed by single quotation marks.

For example, to configure Curator to:

• Delete indices in the myapp-dev project older than 1 day

• Delete indices in the myapp-qe project older than 1 week

• Delete operations logs older than 8 weeks

• Delete all other projects indices after they are 31 days old

• Delete indices older than 1 day that are matched by the ^project\..+\-dev.*$ regex

• Delete indices older than 2 days that are matched by the ^project\..+\-test.*$ regex

Use:

  config.yaml: |
    .defaults:
      delete:
        days: 31
    .operations:
      delete:
        weeks: 8
    myapp-dev:
      delete:
        days: 1
    myapp-qe:
      delete:
        weeks: 1
    .regex:
      - pattern: '^project\..+\-dev\..*$'
        delete:
          days: 1
      - pattern: '^project\..+\-test\..*$'
        delete:
          days: 2