12.1 Logs

We want to build web applications that can keep track of events which have occurred throughout execution, combining them all into one place for easy access later on, when we inevitably need to perform debugging or optimization tasks. Go provides a simple log package which we can use to help us implement simple logging functionality. Logs can be printed using Go’s fmt package, called inside error handling functions for general error logging. Go’s standard package only contains basic functionality for logging, however. There are many third party logging tools that we can use to supplement it if your needs are more sophisticated (tools similar to log4j and log4cpp, if you’ve ever had to deal with logging in Java or C++). A popular and fully featured, open-source logging tool in Go is the seelog logging framework. Let’s take a look at how we can use seelog to perform logging in our Go applications.

Introduction to seelog

Seelog is a logging framework for Go that provides some simple functionality for implementing logging tasks such as filtering and formatting. Its main features are as follows:

  • Dynamic configuration via XML; you can load configuration parameters dynamically without recompiling your program
  • Supports hot updates, the ability to dynamically change the configuration without the need to restart the application
  • Supports multi-output streams that can simultaneously pipe log output to multiple streams, such as a file stream, network flow, etc.

  • Support for different log outputs

    • Command line output
    • File Output
    • Cached output
    • Support log rotate
    • SMTP Mail

The above is only a partial list of seelog’s features. To fully take advantage of all of seelog’s functionality, have a look at its official wiki which thoroughly documents what you can do with it. Let’s see how we’d use seelog in our projects:

First install seelog:

  1. go get -u github.com/cihub/seelog

Then let’s write a simple example:

  1. package main
  3. import log "github.com/cihub/seelog"
  5. func main() {
  6.     defer log.Flush()
  7.     log.Info("Hello from Seelog!")
  8. }

Compile and run the program. If you see a Hello from seelog in your application log, seelog has been successfully installed and is running operating normally.

Custom log processing with seelog

Seelog supports custom log processing. The following code snippet is based on the its custom log processing part of its package:

  1. package logs
  3. import (
  4.     "errors"
  5.     "fmt"
  6.     seelog "github.com/cihub/seelog"
  7.     "io"
  8. )
  10. var Logger seelog.LoggerInterface
  12. func loadAppConfig() {
  13.     appConfig := `
  14. <seelog minlevel="warn">
  15.     <outputs formatid="common">
  16.         <rollingfile type="size" filename="/data/logs/roll.log" maxsize="100000" maxrolls="5"/>
  17.         <filter levels="critical">
  18.             <file path="/data/logs/critical.log" formatid="critical"/>
  19.             <smtp formatid="criticalemail" senderaddress="astaxie@gmail.com" sendername="ShortUrl API" hostname="smtp.gmail.com" hostport="587" username="mailusername" password="mailpassword">
  20.                 <recipient address="xiemengjun@gmail.com"/>
  21.             </smtp>
  22.         </filter>
  23.     </outputs>
  24.     <formats>
  25.         <format id="common" format="%Date/%Time [%LEV] %Msg%n" />
  26.         <format id="critical" format="%File %FullPath %Func %Msg%n" />
  27.         <format id="criticalemail" format="Critical error on our server!\n    %Time %Date %RelFile %Func %Msg \nSent by Seelog"/>
  28.     </formats>
  29. </seelog>
  30. `
  31.     logger, err := seelog.LoggerFromConfigAsBytes([]byte(appConfig))
  32.     if err != nil {
  33.         fmt.Println(err)
  34.         return
  35.     }
  36.     UseLogger(logger)
  37. }
  39. func init() {
  40.     DisableLog()
  41.     loadAppConfig()
  42. }
  44. // DisableLog disables all library log output
  45. func DisableLog() {
  46.     Logger = seelog.Disabled
  47. }
  49. // UseLogger uses a specified seelog.LoggerInterface to output library log.
  50. // Use this func if you are using Seelog logging system in your app.
  51. func UseLogger(newLogger seelog.LoggerInterface) {
  52.     Logger = newLogger
  53. }

The above implements the three main functions:

  • DisableLog

Initializes a global variable Logger with seelog disabled, mainly in order to prevent the logger from being repeatedly initialized

  • LoadAppConfig

Initializes the configuration settings of seelog according to a configuration file. In our example we are reading the configuration from an in-memory string, but of course, you can read it from an XML file also. Inside the configuration, we set up the following parameters:

  • Seelog

The minlevel parameter is optional. If configured, logging levels which are greater than or equal to the specified level will be recorded. The optional maxlevel parameter is similarly used to configure the maximum logging level desired.

  • Outputs

Configures the output destination. In our particular case, we channel our logging data into two output destinations. The first is a rolling log file where we continuously save the most recent window of logging data. The second destination is a filtered log which records only critical level errors. We additionally configure it to alert us via email when these types of errors occur.

  • Formats

Defines the various logging formats. You can use custom formatting, or predefined formatting -a full list of predefined formats can be found on seelog’s wiki

  • UseLogger

Set the current logger as our log processor

Above, we’ve defined and configured a custom log processing package. The following code demonstrates how we’d use it:

  1. package main
  3. import (
  4.     "net/http"
  5.     "project/logs"
  6.     "project/configs"
  7.     "project/routes"
  8. )
  10. func main() {
  11.     addr, _ := configs.MainConfig.String("server", "addr")
  12.     logs.Logger.Info("Start server at:%v", addr)
  13.     err := http.ListenAndServe(addr, routes.NewMux())
  14.     logs.Logger.Critical("Server err:%v", err)
  15. }

Email notifications

The above example explains how to set up email notifications with seelog. As you can see, we used the following smtp configuration:

  1. <smtp formatid="criticalemail" senderaddress="astaxie@gmail.com" sendername="ShortUrl API" hostname="smtp.gmail.com" hostport="587" username="mailusername" password="mailpassword">
  2.     <recipient address="xiemengjun@gmail.com"/>
  3. </smtp>

We set the format of our alert messages through the criticalemail configuration, providing our mail server parameters to be able to receive them. We can also configure our notifier to send out alerts to additional users using the recipient configuration. It’s a simple matter of adding one line for each additional recipient.

To test whether or not this code is working properly, you can add a fake critical message to your application like so:

  1. logs.Logger.Critical("test Critical message")

Don’t forget to delete it once you’re done testing, or when your application goes live, your inbox may be flooded with email notifications.

Now, whenever our application logs a critical message while online, you and your specified recipients will receive a notification email. You and your team can then process and remedy the situation in a timely manner.

Using application logs

When it comes to logs, each application’s use-case may vary. For example, some people use logs for data analysis purposes, others for performance optimization. Some logs are used to analyze user behavior and how people interact with your website. Of course, there are logs which are simply used to record application events as auxiliary data for finding problems.

As an example, let’s say we need to track user attempts at logging into our system. This involves recording both successful and unsuccessful login attempts into our log. We’d typically use the “Info” log level to record these types of events, rather than something more serious like “warn”. If you’re using a linux-type system, you can conveniently view all unsuccessful login attempts from the log using the grep command like so:

  1. # cat /data/logs/roll.log | grep "failed login"
  2. 2012-12-11 11:12:00 WARN : failed login attempt from 11.22.33.44 username password

This way, we can easily find the appropriate information in our application log, which can help us to perform statistical analysis if needed. In addition, we also need to consider the size of logs generated by high-traffic web applications. These logs can sometimes grow unpredictably. To resolve this issue, we can set seelog up with the logrotate configuration to ensure that single log files do not consume excessive disk space.

Summary

In this section, we’ve learned the basics of seelog and how to build a custom logging system with it. We saw that we can easily configure seelog into as powerful a log processing system as we need, using it to supply us with reliable sources of data for analysis. Through log analysis, we can optimize our system and easily locate the sources of problems when they arise. In addition, seelog ships with various default log levels. We can use the minlevel configuration in conjunction with a log level to easily set up tests or send automated notification messages.