Logger

We can use logs to observe program behavior, diagnose problems, or configure corresponding alarms. And defining a well structured log can improve search efficiency and facilitate handling of problems.

Design concept

For convenience, Kratos defines two levels of abstraction. Logger unifies the access mode of logs and helper interface unifies the call mode of logstore.

Different companies and infrastructures may have different requirements for the printing method, format and output location of logs. Kratos abstracts the log component in order to adapt and migrate to various environments more flexibly, so that the use of logs in business code can be isolated from the specific implementation of the underlying log, and the overall maintainability can be improved.

Log of kratos has the following characteristics:

  • Logger is used to connect various log libraries or log platforms, which can be implemented by off the shelf or by yourself.
  • Helper is actually called in your project code, it is used to print logs in business code
  • Filter is used to filter or modify the output log (usually used for log desensitization)
  • Valuer is used to bind some global fixed or dynamic values (such as timestamp, traceID or instance ID) to the output log.

Helper - log in project code

Helper:Advanced log interface, which provides a series of help functions with log levels and formatting methods. This is usually recommended in business logic, which can simplify log code. You can think of it as a wrapper for the logger, which simplifies the parameters that need to be passed in when printing. Its usage is basically the following, and the specific usage will be introduced later

  1. helper.Info("hello")
  2. helper.Errorf("hello %s", "eric")

Logger - adapts to various log output methods

Logger:This is the underlying log interface, which is used to quickly adapt various log libraries to the framework. Only the simplest Log method needs to be implemented.

Interface

Kratos contains only the simplest Log interface for business-adapted log access. When your business logic needs to use custom logs inside the kratos framework, you only need to implement the Log method simply. Kratos logs also provide some log helpful features such as valuer, helper, filter, and so on, which can be implemented directly using the framework’s built-in implementations when we need them, or by ourselves.

  1. type Logger interface {
  2.     Log(level Level, keyvals ...interface{}) error
  3. }

Level parameter is used to identify the level of the log, which can be found in level.go.

keyvals is a tiled array of key-values. Its length needs to be an even number, with keys on odd bits and values on even bits. The use of this Logger interface after implementation, as follows:

  1. logger.Log(log.LevelInfo, "msg", "hello", "instance_id", 123)

Its significance is that by simply using the Logger interface, you can quickly adaptation your log library and use Helper to unify the printing behavior.

Log level

Log levels are defined in level.go, you can pass them in when using the underlying Log method, and they will be output to the level field of the log. Using a specific method with log level such as .Infof in the advanced interface Helper will automatically apply the level without binding the level yourself.

  1. log.LevelDebug
  2. log.LevelInfo
  3. log.LevelWarn
  4. log.LevelError
  5. log.LevelFatal

Adaptation and implementation

We have implemented some plug-ins in contrib/log to adapt to the currently commonly used log libraries. You can also refer to their codes to implement the adaptation of the log library you need:

  • std stdout, built into Kratos
  • fluent output to fluentd
  • zap Adapted to uber’s zap log library
  • aliyun output to aliyun log service

Use

The Kratos log library is very simple to use, similar to most logging libraries.

DefaultLogger: default logger

If you feel that creating a logger is troublesome, You can directly use the default-initialized log.DefaultLogger instance of the framework, which directly calls the log of the go standard library at the bottom, and can print to standard output.

stdLogger

This framework has a built-in implementation of stdLogger, capable of printing to standard output. Use the NewStdLogger method to pass in a io.Writer.

  1. //Output to console
  2. l := log.DefaultLogged
  3. l.Log(log.LevelInfo, "stdout_key", "stdout_value")
  5. // Output to ./test.log file
  6. f, err := os.OpenFile("test.log", os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0666)
  7. if err != nil {
  8.     return
  9. }
  10. l = log.NewStdLogger(f)
  11. l.Log(log.LevelInfo, "file_key", "file_value")

initialize

First, you need to create a Logger, here you can choose: the built-in std print to standard output, or find an already implemented adaptation under contrib, or use your own implemented Logger.

  1. import "github.com/go-kratos/kratos/v2/log"
  3. h := NewHelper(yourlogger)
  5. // You can use the default logger directly
  6. h := NewHelper(log.DefaultLogger)

Or find a plugin in contrib/log里 to use, for example, here we want to use fluentd:

  1. import "github.com/go-kratos/kratos/contrib/log/fluent/v2"
  3. logger, err := fluent.NewLogger("unix:///var/run/fluent/fluent.sock")
  4. if err != nil {
  5.     return 
  6. }
  7. h := log.NewHelper(logger)

You can specify the default log to print to the field, if not set, the default is msg.

  1. NewHelper(logger, WithMessageKey("message"))

Print log

Note: The method of calling the Fatal level will interrupt the program running after printing the log, please use it with caution. Directly print logs of different levels, which will be entered into messageKey, the default is msg.

  1. h.Debug("Are you OK?")
  2. h.Info("42 is the answer to life, the universe, and everything")
  3. h.Warn("We are under attack!")
  4. h.Error("Houston, we have a problem.")
  5. h.Fatal("So Long, and Thanks for All the Fish.")

Format and print logs of different levels, all methods end with f

  1. h.Debugf("Hello %s", "boy")
  2. h.Infof("%d is the answer to life, the universe, and everything", 233)
  3. h.Warnf("We are under attack %s!", "boss")
  4. h.Errorf("%s, we have a problem.", "Master Shifu")
  5. h.Fatalf("So Long, and Thanks for All the %s.", "banana")

Format and print logs of different levels. The methods all end with w. The parameter is a key value pair, and multiple groups can be entered.

  1. h.Debugw("custom_key", "Are you OK?")
  2. h.Infow("custom_key", "42 is the answer to life, the universe, and everything")
  3. h.Warnw("custom_key", "We are under attack!")
  4. h.Errorw("custom_key", "Houston, we have a problem.")
  5. h.Fatalw("custom_key", "So Long, and Thanks for All the Fish.")

Use the underlying Log interface to print key and value.

  1. h.Log(log.LevelInfo, "key1", "value1")

Valuer: sets global fields

In business logs, we usually output some global fields in each log, such as timestamp, instance id, tracking id, user id, calling function name, etc. Obviously, it is very troublesome to manually write these values in each log . To solve this problem, Valuer can be used. You can think of it as the “middleware” of the logger, and use it to type some global information into the log.

The log.With method will return a new Logger and bind the Valuer of the parameter to it. Note that the parameters should be written in the order of key and value. The method of use is as follows:

  1. logger = log.With(logger, "ts", log.DefaultTimestamp, "caller", log.DefaultCaller)

By default, the framework provides the following Valuers for use. You can also refer to their codes to implement custom Valuers.

Filter: Log filtering

Sometimes there may be sensitive information in the log, which needs to be desensitized, or only high-level logs can be printed. At this time, Filter can be used to filter the output of the log. The usual usage is to use Filter to wrap the original Logger and use To create Helper use.

It provides the following parameters:

  • FilterLevel filters according to the log level, logs below this level will not be output. For example, if FilterLevel(log.LevelError) is passed in here, the debug/info/warn log will be filtered out and will not be output, and error and fatal will be output normally.
  • FilterKey(key ...string) FilterOption Filter by key, the value of these keys will be masked by ***
  • FilterValue(value ...string) FilterOption Filter by value, matching values will be masked by ***
  • FilterFunc(f func(level Level, keyvals ...interface{}) bool) Use a custom function to process the log. Keyvals are the key and the corresponding value, which can be read according to the parity.
  1. h := NewHelper(
  2.     NewFilter(logger,
  3.         // Level filtration
  4.         FilterLevel(log.LevelError),
  6.         // Press key to mask
  7.         FilterKey("username"),
  9.         // Press value to mask
  10.         FilterValue("hello"),
  12.         // Custom filter function
  13.         FilterFunc(
  14.             func (level Level, keyvals ...interface{}) bool {
  15.                 if level == LevelWarn {
  16.                     return true
  17.                 }
  18.                 for i := 0; i < len(keyvals); i++ {
  19.                     if keyvals[i] == "password" {
  20.                         keyvals[i+1] = fuzzyStr
  21.                     }
  22.                 }
  23.                 return false
  24.             }
  25.         ),
  26.     ),
  27. )
  29. h.Log(log.LevelDebug, "msg", "test debug")
  30. h.Info("hello")
  31. h.Infow("password", "123456")
  32. h.Infow("username", "kratos")
  33. h.Warn("warn log")

Bind context

Set the context, using the following method will return a helper instance bound to the specified context

  1. newHelper := h.WithContext(ctx)

Request log middleware

We provide a log middleware in middleware/logging, which can record the routing, parameters, time-consuming and other information of each request on the server or client side. When using it, it is recommended to desensitize the request parameter log with Filter to avoid leakage of sensitive information. The code of this middleware also clearly shows how to obtain and process request and return information in the middleware, which has great reference value. You can implement your own log middleware based on its code.

Global log

If you are in a project and just want to use a simple log function that can be printed at any time in the global, we provide a global log.

  1. import "github.com/go-kratos/kratos/v2/log"
  3. log.Info("info")
  4. log.Warn("warn")

The above is using the default log.DefaultLogger standard output. You can also find an already implemented adaptation under contrib, or use your own implemented Logger and use log.SetLogger to set the global log logger.

  1. // Setting up a global logger with zap log
  3. import (
  4.     "os"
  6.     "go.uber.org/zap"
  7.     "go.uber.org/zap/zapcore"
  9.     kratoszap "github.com/go-kratos/kratos/contrib/log/zap/v2"
  10.     "github.com/go-kratos/kratos/v2/log"
  11. )
  13. f, err := os.OpenFile("test.log", os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0666)
  14. if err != nil {
  15.     return
  16. }
  17. writeSyncer := zapcore.AddSync(f)
  19. encoder := zapcore.NewJSONEncoder(zap.NewProductionEncoderConfig())
  20. core := zapcore.NewCore(encoder, writeSyncer, zapcore.DebugLevel)
  21. z := zap.New(core)
  23. logger := kratoszap.NewLogger(z)
  24. log.SetLogger(logger)
  26. // Print log
  27. log.Info("info")
  28. log.Debug("debug")

kratos-layout

In our default project template, we initialized the logger instance at the program entry in the main() function of [cmd/server/main.go](https://github.com/go-kratos/kratos-layout/blob/cf30efc32d78338e8e4739d3288feeba426388a5/cmd/server/main.go#L49), and injected some global log values, which will be typed into all output logs.

You can modify the logger used here to customize the printed value, or replace it with the logger implementation you need.

  1. logger := log.With(log.NewStdLogger(os.Stdout),
  2.     "ts", log.DefaultTimestamp,
  3.     "caller", log.DefaultCaller,
  4.     "service.id", id,
  5.     "service.name", Name,
  6.     "service.version", Version,
  7.     "trace_id", tracing.TraceID(),
  8.     "span_id", tracing.SpanID(),
  9. )

This logger will be generated by the dependency injection tool wire and injected into each layer of the project for its internal use. A specific internal use example can refer to internal/service/greeter.go. We will inject the logger instance here, wrap it as a Helper with log.NewHelper and bind it to the service, so that the bound helper object can be called at this layer to log.

  1. func NewGreeterService(uc *biz.GreeterUsecase, logger log.Logger) *GreeterService {
  2.     return &GreeterService{uc: uc, log: log.NewHelper(logger)} // 初始化和绑定helper
  3. }
  5. func (s *GreeterService) SayHello(ctx context.Context, in *v1.HelloRequest) (*v1.HelloReply, error) {
  6.     // Print log
  7.     s.log.WithContext(ctx).Infof("SayHello Received: %v", in.GetName())
  9.     return &v1.HelloReply{Message: "Hello " + in.GetName()}, nil
  10. }

The initialization and usage methods of other layers are also the same. In the biz layer and the data layer, we also give a sample of logger injection, you can refer to.