Troubleshooting the log curator

You can use information in this section for debugging log curation. Curator is used to remove data that is in the Elasticsearch index format prior to OKD 4.6, and will be removed in a later release.

Troubleshooting log curation

You can use information in this section for debugging log curation. For example, if curator is in a failed state, but the log messages do not provide a reason, you could increase the log level and trigger a new job, instead of waiting for another scheduled run of the cron job.

Prerequisites

• Cluster logging and Elasticsearch must be installed.

Procedure

To enable the Curator debug log and trigger next Curator iteration manually:

1. Enable debug log of Curator:

   $ oc set env cronjob/curator CURATOR_LOG_LEVEL=DEBUG CURATOR_SCRIPT_LOG_LEVEL=DEBUG

   Specify the log level:

   • CRITICAL. Curator displays only critical messages.

   • ERROR. Curator displays only error and critical messages.

   • WARNING. Curator displays only error, warning, and critical messages.

   • INFO. Curator displays only informational, error, warning, and critical messages.

   • DEBUG. Curator displays only debug messages, in addition to all of the above.

     The default value is INFO.

     Cluster logging uses the OKD custom environment variable CURATOR_SCRIPT_LOG_LEVEL in OKD wrapper scripts (run.sh and convert.py). The environment variable takes the same values as CURATOR_LOG_LEVEL for script debugging, as needed.

2. Trigger next curator iteration:

   $ oc create job --from=cronjob/curator <job_name>

3. Use the following commands to control the cron job:

   • Suspend a cron job:

     $ oc patch cronjob curator -p '{"spec":{"suspend":true}}'

   • Resume a cron job:

     $ oc patch cronjob curator -p '{"spec":{"suspend":false}}'

   • Change a cron job schedule:

     $ oc patch cronjob curator -p '{"spec":{"schedule":"0 0 * * *"}}' (1)

     1	The schedule option accepts schedules in cron format.