Log Levels

In this section we define and describe the meaning of the log levelsin ArangoDB’s log messages. The log levels are, from most to least severe:

• FATAL
• ERROR
• WARN
• INFO
• DEBUG
• TRACEFor each log topic one can configure the lowest level which is actually logged.For example, if one sets the log level to ERROR for some log topic,one only sees messages of level ERROR and above (ERROR and FATAL).

See an example of how toconfigure log levelsin the Administration chapter.

FATAL

Fatal errors are the most severe errors and only occur if a service or applicationcan not recover safely from an abnormal state, which forces it to shut down.

Typically, a fatal error only occurs once in the process lifetime,so if the log file is tied to the process, this is typicallythe last message in the log. There might be a few exceptions to thisrule, where it makes more sense to keep the server running, for exampleto be able to diagnose the problem better.

We reserve this error type for the following events:

• crucial files/folders are missing or inaccessible during startup
• overall application or system failure with a serious danger ofdata corruption or loss (the following shutdown is intended to preventpossible or further data loss)Recommendation:Fatal errors should be investigated immediately by a system administrator.

ERROR

If a problem is encountered which is fatal to some operation, but not forthe service or the application as a whole, then an error is logged.

Reasons for log entries of this severity are for example:

• missing data
• a required file can’t be opened
• incorrect connection strings
• missing servicesIf some operation is automatically retried and eventually succeeds,no error will be written to the log. Therefore, if an error is logged thenit should be taken seriously as it may require user intervention to solve.

Note that in any distributed system, temporary failures of network connectionsor certain servers or services can and will happen. Most systems will toleratesuch failures and retry for some time, but will eventually run out of patience,give up and fail the operation one level up.

Recommendation:A system administrator should be notified automatically to investigate the error.By filtering the log to look at errors (and other logged events above)one can determine the error frequency and quickly identify the initial failurethat might have resulted in a cascade of additional errors.

WARN

A warning is triggered by anything that can potentially causeapplication oddities, but from which the system recovers automatically.

Examples of events which lead to warnings:

• switching from a primary to backup server
• retrying an operation
• missing secondary data
• things running inefficiently(in particular slow queries and bad system settings)Certain warnings are logged at startup time only, such as startup optionvalues which lie outside the recommended range.

These might be problems, or might not. For example, expected transientenvironmental conditions such as short loss of network or databaseconnectivity are logged as warnings, not errors. Viewing a log filteredto show only warnings and errors may give quick insight into earlyhints at the root cause of subsequent errors.

Recommendation:Can mostly be ignored but can give hints for inefficiencies orfuture problems.

INFO

Info messages are generally useful information to log to betterunderstand what state the system is in. One will usually want tohave info messages available but does usually not care about themunder normal circumstances.

Informative messages are logged in events like:

• successful initialization
• services starting or stopping
• successful completion of significant transactions
• configuration assumptionsViewing log entries of severity info and above should give a quick overviewof major state changes in the process providing top-level context forunderstanding any warnings or errors that also occur. Logging info levelmessages and above will usually not spam anything beyond good readability.

Recommendation:Usually good to have, but one does not have to look at info level messagesunder normal circumstances.

DEBUG

Information which is helpful to ArangoDB developers as well as to otherpeople like system administrators to diagnose an application or serviceis logged as debug message.

Debug messages make software much more maintainable but require somediligence because the value of individual debug statements may changeover time as programs evolve. The best way to achieve this is by gettingthe development team in the habit of regularly reviewing logs as a standardpart of troubleshooting reported issues. We encourage our teams toprune out messages that no longer provide useful context and to addmessages where needed to understand the context of subsequent messages.

Recommendation:Debug level messages are usually switched off, but one can switch them onto investigate problems.

TRACE

Trace messages produce a lot of output and are usually only needed byArangoDB developers to debug problems in the source code.

Recommendation:Trace level logging should generally stay disabled.