13.4 Logging and configuration

The importance of logging and configuration

Previously in the book, we saw that event logging plays a very important role in application development. With adequate logging, we can record crucial information that can later be dissected for debugging and optimization purposes. In the section where we looked at the seelog logging utility, we saw that it had settings for various log level gradations, which can be essential for program development and deployment; we can set the logging level lower in a development environment, while setting it high in production so that we can mask extraneous information when we are trying to debug our application.

Setting up the server configuration module for deploying an application involves a number of different server settings. For example, we typically need to provide information regarding database configuration, listening ports, etc., via the configuration file. Setting up a centralized configuration file allows us the flexibility of deploying the application to different machines and connecting to remote databases, if needed.

The Beego logging system

The Beego logger’s design borrows ideas from seelog and provides similar functionality in terms of setting logging levels. Beego’s system is, however, more lightweight and makes use of the Go’s log.Logger interface. By default, logs are output to os.Stdout, but users can implement this interface through beego.SetLogger to customize this. A detailed example of an implemented interface can be seen below:

// Log levels for controlling the logging output.
const (
    LevelTrace = iota
    LevelDebug
    LevelInfo
    LevelWarning
    LevelError
    LevelCritical
)
// logLevel controls the global log level used by the logger.
var level = LevelTrace
// LogLevel returns the global log level and can be used in
// a custom implementations of the logger interface.
func Level() int {
    return level
}
// SetLogLevel sets the global log level used by the simple
// logger.
func SetLevel(l int) {
    level = l
}

This section implements the above log grading system. The default level is set to Trace and users can customize grading levels using SetLevel.

// logger references the used application logger.
var BeeLogger = log.New(os.Stdout, "", log.Ldate|log.Ltime)
// SetLogger sets a new logger.
func SetLogger(l *log.Logger) {
    BeeLogger = l
}
// Trace logs a message at trace level.
func Trace(v ...interface{}) {
    if level <= LevelTrace {
        BeeLogger.Printf("[T] %v\n", v)
    }
}
// Debug logs a message at debug level.
func Debug(v ...interface{}) {
    if level <= LevelDebug {
        BeeLogger.Printf("[D] %v\n", v)
    }
}
// Info logs a message at info level.
func Info(v ...interface{}) {
    if level <= LevelInfo {
        BeeLogger.Printf("[I] %v\n", v)
    }
}
// Warning logs a message at warning level.
func Warn(v ...interface{}) {
    if level <= LevelWarning {
        BeeLogger.Printf("[W] %v\n", v)
    }
}
// Error logs a message at error level.
func Error(v ...interface{}) {
    if level <= LevelError {
        BeeLogger.Printf("[E] %v\n", v)
    }
}
// Critical logs a message at critical level.
func Critical(v ...interface{}) {
    if level <= LevelCritical {
        BeeLogger.Printf("[C] %v\n", v)
    }
}

The code snippet above initializes a BeeLogger object by default, outputting logs to os.Stdout. As mentioned, users can implement beego.SetLogger to customize the logger’s output. BeeLogger implements six functions:

• Trace (record general information, for example:)
  • “Entered parse function validation block”
  • “Validation: entered second ‘if’”
  • “Dictionary ‘Dict’ is empty. Using default value”
• Debug (debugging information, for example:)
  • “Web page requested: http://somesite.com Params = ‘…’”
  • “Response generated. Response size: 10000. Sending.”
  • “New file received. Type: PNG Size: 20000”
• Info (printing general information, for example:)
  • “Web server restarted”
  • “Hourly statistics: Requested pages: 12345 Errors: 123…”
  • “Service paused. Waiting for ‘resume’ call”
• Warn (warning messages, for example:)
  • “Cache corrupted for file = ‘test.file’. Reading from back-end”
  • “Database 192.168.0.7/DB not responding. Using backup 192.168.0.8/DB”
  • “No response from statistics server. Statistics not sent”
• Error (error messages, for example:)
  • “Internal error. Cannot process request# 12345 Error:….”
  • “Cannot perform login: credentials DB not responding”
• Critical (fatal errors, for example:)
  • “Critical panic received:…. Shutting down”
  • “Fatal error:… App is shutting down to prevent data corruption or loss”

You can see that each of these levels has a specific purpose. For instance if we set the logging level to Warn (level=LevelWarning), at the time of deployment, all of the lower level logs (Trace, Debug, Info) will not output anything.

Beego configuration design

For processing configuration information, Beego implements a key=value file parser which reads information formatted similarly to ini configuration files. The parser reads the configuration data and saves it to a map. Finally, it calls several functions for retrieving the value’s datatype (int, string, etc). The detailed implementation can be seen below:

Define some global constants for the ini configuration file:

var (
    bComment = []byte{'#'}
    bEmpty   = []byte{}
    bEqual   = []byte{'='}
    bDQuote  = []byte{'"'}
)

Defines the format of the configuration file:

// A Config represents the configuration.
type Config struct {
    filename string
    comment  map[int][]string  // id: []{comment, key...}; id 1 is for main comment.
    data     map[string]string // key: value
    offset   map[string]int64  // key: offset; for editing.
    sync.RWMutex
}

Defines a function for parsing the file. The process begins by opening the file, then reading it line by line and parsing comments, blank lines and key=value data:

// ParseFile creates a new Config and parses the file configuration from the
// named file.
func LoadConfig(name string) (*Config, error) {
    file, err := os.Open(name)
    if err != nil {
        return nil, err
    }
    cfg := &Config{
        file.Name(),
        make(map[int][]string),
        make(map[string]string),
        make(map[string]int64),
        sync.RWMutex{},
    }
    cfg.Lock()
    defer cfg.Unlock()
    defer file.Close()
    var comment bytes.Buffer
    buf := bufio.NewReader(file)
    for nComment, off := 0, int64(1); ; {
        line, _, err := buf.ReadLine()
        if err == io.EOF {
            break
        }
        if bytes.Equal(line, bEmpty) {
            continue
        }
        off += int64(len(line))
        if bytes.HasPrefix(line, bComment) {
            line = bytes.TrimLeft(line, "#")
            line = bytes.TrimLeftFunc(line, unicode.IsSpace)
            comment.Write(line)
            comment.WriteByte('\n')
            continue
        }
        if comment.Len() != 0 {
            cfg.comment[nComment] = []string{comment.String()}
            comment.Reset()
            nComment++
        }
        val := bytes.SplitN(line, bEqual, 2)
        if bytes.HasPrefix(val[1], bDQuote) {
            val[1] = bytes.Trim(val[1], `"`)
        }
        key := strings.TrimSpace(string(val[0]))
        cfg.comment[nComment-1] = append(cfg.comment[nComment-1], key)
        cfg.data[key] = strings.TrimSpace(string(val[1]))
        cfg.offset[key] = off
    }
    return cfg, nil
}

Below are a number of functions the parser uses for reading the configuration file. The return value is determined as either a bool, int, float64 or string:

// Bool returns the boolean value for a given key.
func (c *Config) Bool(key string) (bool, error) {
    return strconv.ParseBool(c.data[key])
}
// Int returns the integer value for a given key.
func (c *Config) Int(key string) (int, error) {
    return strconv.Atoi(c.data[key])
}
// Float returns the float value for a given key.
func (c *Config) Float(key string) (float64, error) {
    return strconv.ParseFloat(c.data[key], 64)
}
// String returns the string value for a given key.
func (c *Config) String(key string) string {
    return c.data[key]
}

Application guide

The following function is an example of an application I used to fetch json data from a remote url address:

func GetJson() {
    resp, err := http.Get(beego.AppConfig.String("url"))
    if err != nil {
        beego.Critical("http get info error")
        return
    }
    defer resp.Body.Close()
    body, err := ioutil.ReadAll(resp.Body)
    err = json.Unmarshal(body, &AllInfo)
    if err != nil {
        beego.Critical("error:", err)
    }
}

Beego’s Critical() logging function is called to report any errors which may occur in the GetJson() function. beego.AppConfig.String("url") is used to obtain information from a configuration file (typically app.conf), which might look something like the following:

appname = hs
url ="http://www.api.com/api.html"

Links

• Previous section: Designing controllers
• Next section: Adding, deleting and updating blogs