This document mainly introduces the log management of the Frontend (FE) process.

This document is applicable to Doris versions 2.1.4 and later.

Log Categories

When starting the FE process using sh bin/start_fe.sh --daemon, the following types of log files will be generated in the FE log directory:

• fe.log

  FE process running log. The main log file for FE. All levels of FE process logs (DEBUG, INFO, WARN, ERROR, etc.) will be printed to this log file.

• fe.warn.log

  FE process running log. Only prints logs at WARN level and above. The content in fe.warn.log is a subset of the fe.log log content. It is mainly used to quickly view warning or error level logs.

• fe.audit.log

  Audit log. Used to record all database operation records executed through this FE node. This includes SQL, DDL, DML statements, etc.

• fe.out

  Used to receive standard output stream and error data stream logs. For example, output from echo commands in start scripts, or other log information not captured by the log4j framework. Usually serves as a supplement to the running log. In rare cases, you may need to check the content of fe.out for more information.

• fe.gc.log

  GC log of the FE JVM. The behavior of this log is controlled by the JVM startup option JAVA_OPTS in fe.conf.

Log Configuration

Includes configuring the log storage path, retention time, retention count, size, etc.

The following configuration items are configured in the fe.conf file.

::: note Starting from version 3.0.2, the default value of sys_log_mode configuration is changed to AYSNC. :::

tip

sys_log_roll_num 控制的是一天的保留日志数量，而不是总数量，需要配合 sys_log_delete_age 共同确定总保留日志数量。 sys_log_roll_num controls the number of retained logs per day, not the total number. It needs to be combined with sys_log_delete_age to determine the total number of retained logs.

Enable DEBUG Log

The Debug level log of FE can be enabled by modifying the configuration file or through the interface or API during runtime.

• Enable through configuration file

  Add the configuration item sys_log_verbose_modules in fe.conf. For example:

  # Only open DEBUG log for "org.apache.doris.catalog.Catalog"
  sys_log_verbose_modules=org.apache.doris.catalog.Catalog
  # Open DEBUG log for all classes in "org.apache.doris.catalog"
  sys_log_verbose_modules=org.apache.doris.catalog
  # Open DEBUG log for all classes in "org"
  sys_log_verbose_modules=org

  Add the configuration item and restart the FE node to take effect.

• Enable through FE UI interface

  You can modify the log level at runtime through the UI interface. No need to restart the FE node. Open the FE node’s http port in the browser (default is 8030) and log in to the UI interface. Then click on the Log tab in the top navigation bar.

  

  In the Add input box, you can enter the package name or specific class name to open the corresponding Debug log. For example, entering org.apache.doris.catalog.Catalog will open the Debug log of the Catalog class:

  

  You can also enter the package name or specific class name in the Delete input box to close the corresponding Debug log.

  :::note The modification here will only affect the log level of the corresponding FE node. It will not affect the log level of other FE nodes. :::

• Modify through API

  You can also modify the log level at runtime through the following API. No need to restart the FE node.

  curl -X POST -uuser:passwd fe_host:http_port/rest/v1/log?add_verbose=org.apache.doris.catalog.Catalog

  Where the username and password are the root or admin users logged into Doris. The add_verbose parameter specifies the package name or class name for enabling Debug log. If successful, it will return:

  {
      "msg": "success", 
      "code": 0, 
      "data": {
          "LogConfiguration": {
              "VerboseNames": "org,org.apache.doris.catalog.Catalog", 
              "AuditNames": "slow_query,query,load", 
              "Level": "INFO"
          }
      }, 
      "count": 0
  }

  You can also close Debug log through the following API:

  curl -X POST -uuser:passwd fe_host:http_port/rest/v1/log?del_verbose=org.apache.doris.catalog.Catalog

  The del_verbose parameter specifies the package name or class name for closing Debug log.

Log Configuration for k8s

In some cases, the FE process is deployed through a container environment (such as k8s). All logs need to be output through standard output stream instead of files.

In this case, you can start the FE process in the foreground and output all logs to the standard output stream by using the command sh bin/start_fe.sh --console.

To distinguish different types of logs in the same standard output stream, a different prefix will be added before each log. For example:

RuntimeLogger 2024-06-24 00:05:21,522 INFO (main|1) [DorisFE.start():158] Doris FE starting...
RuntimeLogger 2024-06-24 00:05:21,530 INFO (main|1) [FrontendOptions.analyzePriorityCidrs():194] configured prior_cidrs value: 172.20.32.136/24
RuntimeLogger 2024-06-24 00:05:21,535 INFO (main|1) [FrontendOptions.initAddrUseIp():101] local address: /172.20.32.136.
RuntimeLogger 2024-06-24 00:05:21,740 INFO (main|1) [ConsistencyChecker.initWorkTime():106] consistency checker will work from 23:00 to 23:00
RuntimeLogger 2024-06-24 00:05:21,889 ERROR (main|1) [Util.report():128] SLF4J: Class path contains multiple SLF4J bindings.

The meanings of different prefixes are as follows:

• StdoutLogger: Logs in the standard output stream, corresponding to the content in fe.out.
• StderrLogger: Logs in the standard error stream, corresponding to the content in fe.out.
• RuntimeLogger: Logs in fe.log.
• AuditLogger: Logs in fe.audit.log.
• No prefix: Logs in fe.gc.log.

Additionally, there is an additional configuration parameter for the container environment: