Customization and encapsulation of go language zap library logger

Catalog

Preface

Go Language native Logger

Go Language native Logger The shortcomings of

Zap Log Library

Zap How to use

install zap

Set up Logger

customized Zap Of Logger

Log cutting

encapsulation Logger

summary

Preface

Logs are very important to both programs and programmers , How important , If you want to work healthily in the company for a long time, you must learn to paddle at different stages , One of the keys to a staged stroke is to work faster than expected but pretend ... incorrect , There is something wrong with the beginning , Let's start again .

Logs are very important to both programs and programmers , Programmers can solve problems faster or slower than experience , It depends on whether the log can effectively record the scene and context of the problem .

Then let the program record effective logs , In addition to the accuracy of the points recorded in the program log , You also need a person to weigh your hands Logger . A good Logger ( Loggers ) Be able to provide the following capabilities ：

It supports writing logs to multiple output streams , For example, you can selectively test 、 The development environment outputs logs to both the console and the log file , The production environment is only exported to a file .

Support multi-level log levels , For example, common ones are ：TRACE,DEBUG,INFO,WARN,ERROR etc. .

Support structured output , Structured output is now commonly used JSON Formal , In this way, the unified logging platform , adopt logstash Such components directly aggregate logs onto the logging platform .

Log cutting is required -- log rotation, By date 、 The time interval or file size cuts the log .

stay Log Entry in （ Each line of records ） In addition to actively recorded information , Also include functions such as printing logs 、 The document where it is 、 Line number 、 Record time, etc .

Today I will show you how to use Go In the language development project, we should build a "hands-on" Logger, Before that, let's go back to 2009 year , have a look Go Language has provided us with built-in since its birth Logger.

Go Language native Logger

Go Language with itself log Built in bag , Provides us with a default Logger, You can use it directly . The detailed usage of this library can be found in the official documentation ：pkg.go.dev/log

Use log Log , It will be output to the console by default . Here's an example ：

package mainimport ("log""net/http")func main() {simpleHttpGet("www.google.com")simpleHttpGet("https://www.baidu.com")}func simpleHttpGet(url string) {resp, err := http.Get(url)if err != nil {log.Printf("Error fetching url %s : %s", url, err.Error())} else {log.Printf("Status Code for %s : %s", url, resp.Status)resp.Body.Close()}return}

In this routine , To two web sites respectively GET request , Then the return status code is recorded / Request error . There will be similar output after executing the program ：

2022/05/15 15:15:26 Error fetching url www.baidu.com : Get "www.baidu.com": unsupported protocol scheme "" 2022/05/15 15:15:26 Status Code for https://www.baidu.com : 200 OK

Because the first request URL The head of China Association is missing , Therefore, the request cannot be successfully initiated , The log also records the error information well .

Go Built in log Of course, the package also supports the output of logs to files , adopt log.SetOutput You can put any io.Writer The implementation of is set as the output of the log . Let's change the above routine to output logs to files .

You can try the running effect by yourself , There will be no more demonstrations here .

Go Language native Logger The shortcomings of

Native Logger The advantages of , Obvious , Simple 、 Open the box , No reference to external third-party libraries . We can do the same for a Logger Let's take a look at the default Logger Whether it can be used in the project .

Only one basic log level Print Options . I won't support it INFO/DEBUG And so on .

For error logs , It has Fatal and PanicFatal Log by calling os.Exit(1) To end the program

Panic The log throws a panic

But it's missing one ERROR The level of logging , This level can be used without throwing panic Or log out of the program

Lack of ability to structure log formats —— Only simple text output is supported , You cannot format a log record as JSON Format .

Does not provide the ability to cut logs .

Zap Log Library

stay Go In the ecology of , There are many log libraries to choose from , We briefly introduced before logrus The use of this library : Check me out. , It is associated with Go The built-in log Library in api Layer compatibility , Directly log.Logger Interface , Support the system level of the program Logger Switch to it .

however logrus In performance sensitive scenarios, it doesn't smell good , Use more of Uber Open source zap Log Library . because Uber In today's world Go The contribution of ecology is very high , Plus its own business — Performance sensitive scenarios of online car Hailing , therefore Uber Open source libraries are very popular . Now do the project , Use Zap Do the log Logger There are so many . Programmer's heart OS Should be , No matter how high my concurrence is , It's over in the morning , In case one day we can 2 A concurrent work suddenly became 2W Concurrency .

Zap A big reason for high performance is ： No reflection , Each field to be written in the log must carry a type

logger.Info( "Success..", zap.String("statusCode", resp.Status), zap.String("url", url))

It writes a record to the log ,Message yes "Success.." In addition, two string key value pairs are written . Zap For the fields to be written in the log , Each type has a corresponding method to convert the field to zap.Field type . such as ：

zap.Int('key', 123)zap.Bool('key', true)zap.Error('err', err)zap.Any('arbitraryType', &User{})

There are many other methods of this type , Don't list them one by one . This way of recording logs makes the user experience slightly worse , However, considering the performance gains, the loss in the use experience is acceptable .

Now let's learn Zap How to use , Then use in the project Zap Do some custom configuration and encapsulation , Make it better to use , The most important thing is to match what we said at the beginning about good Logger Five criteria of .

Zap How to use install zap

First of all ,zap Installation method of , Directly run the following command to download zap To the local dependency library .

go get -u go.uber.org/zap Set up Logger

Let's start with zap The configuration provided is good Logger , It will be customized later .

By calling zap.NewProduction()、zap.NewDevelopment()、zap.Example() These three methods , Can be created Logger.

The above three methods can create Logger, They are all right Logger Different configurations are made , such as zap.NewProduction() Created Logger When logging, the calling function information will be automatically recorded 、 Time to log, etc , These three need not be tangled , Use both directly zap.NewProduction(), And when used in the project , We won't just use zap Configured Logger , More detailed customization is needed .

zap Of Logger Provides a way to record logs of different levels , Like the log level from low to high, there are ：Debug、Info、Warn、Error These levels have corresponding methods . They are all used in the same way , Here is Info Method signature of method .

func (log *Logger) Info(msg string, fields ...Field) {if ce := log.check(InfoLevel, msg); ce != nil {ce.Write(fields...)}}

The first parameter of the method is in the log msg Field information to be recorded ,msg It is a fixed field in the log line record , To add another field to the log , Direct delivery zap.Field Type , We've already said that zap.Field Type field , Is the zap.String("key", "value") This kind of method creates . because Info Method signature fileds Parameter declarations are variable parameters , Therefore, you can add any number of fields to the log line records , For example, in the routine ：

logger.Info("Success..", zap.String("statusCode", resp.Status), zap.String("url", url))

In the records of today's ambition , except msg Field , And added statusCode,url Two custom fields . Used in the above routine zap.NewProduction() Created Logger Will output to the console JSON Log lines in format , For example, we use Info After the method , The console will have output similar to the following .

{"level":"info","ts":1558882294.665447,"caller":"basiclogger/UberGoLogger.go:31","msg":"Success..","statusCode":"200 OK","url":"https://www.baidu.com"}

customized Zap Of Logger

Now let's put zap Do further custom configuration , The log can not only be output to the console , It can also be output to a file , Then the log time is in the format of timestamp , Replace it with something that is easier for human beings to understand DateTime Time format .

Let's talk less , Go straight to the code , The necessary explanations are put in the notes .

var logger *zap.Loggerfunc init() {encoderConfig := zap.NewProductionEncoderConfig() // Format the time in the log record encoderConfig.EncodeTime = zapcore.ISO8601TimeEncoder // journal Encoder still JSONEncoder, Format the log lines as JSON Format encoder := zapcore.NewJSONEncoder(encoderConfig)file, _ := os.OpenFile("/tmp/test.log", os.O_CREATE|os.O_APPEND|os.O_WRONLY, 644)fileWriteSyncer = zapcore.AddSync(file)core := zapcore.NewTee(// Write logs to the console and files at the same time , Remember to remove the console write in the production environment , The basic of logging is Debug And above , Remember to change the production environment to Infozapcore.NewCore(encoder, zapcore.AddSync(os.Stdout), zapcore.DebugLevel),zapcore.NewCore(encoder, fileWriteSyncer, zapcore.DebugLevel),)logger = zap.New(core)} Log cutting

Zap Log cutting is not supported by itself , You can use another library lumberjack Assist in cutting .

func getFileLogWriter() (writeSyncer zapcore.WriteSyncer) {// Use lumberjack Realization log rotatelumberJackLogger := &lumberjack.Logger{Filename: "/tmp/test.log",MaxSize: 100, // Single file is the largest 100MMaxBackups: 60, // More than 60 Log files , Clean up older logs MaxAge: 1, // One cut a day Compress: false,}return zapcore.AddSync(lumberJackLogger)} encapsulation Logger

We can't use logs every time , It's all set up like this , So the best thing is to put these configuration initializations in a separate package , This initializes once in the project .

In addition to the above configurations , Our configuration also lacks some log caller information , Like the function name 、 file location 、 Line number, etc , In this way, when checking problems and reading logs , The timeliness of positioning problems will be improved a lot .

We are right. Logger Do the packaging again .

// Send a private message go-logger Official account 「 Network management bi Talk about 」// Complete code and usage are available Demopackage zlog// Simply encapsulate it zap The use of log base // Usage mode ：// zlog.Debug("hello", zap.String("name", "Kevin"), zap.Any("arbitraryObj", dummyObject))// zlog.Info("hello", zap.String("name", "Kevin"), zap.Any("arbitraryObj", dummyObject))// zlog.Warn("hello", zap.String("name", "Kevin"), zap.Any("arbitraryObj", dummyObject))var logger *zap.Loggerfunc init() {......}func getFileLogWriter() (writeSyncer zapcore.WriteSyncer) {......}func Info(message string, fields ...zap.Field) {callerFields := getCallerInfoForLog()fields = append(fields, callerFields...)logger.Info(message, fields...)}func Debug(message string, fields ...zap.Field) {callerFields := getCallerInfoForLog()fields = append(fields, callerFields...)logger.Debug(message, fields...)}func Error(message string, fields ...zap.Field) {callerFields := getCallerInfoForLog()fields = append(fields, callerFields...)logger.Error(message, fields...)}func Warn(message string, fields ...zap.Field) {callerFields := getCallerInfoForLog()fields = append(fields, callerFields...)logger.Warn(message, fields...)}func getCallerInfoForLog() (callerFields []zap.Field) {pc, file, line, ok := runtime.Caller(2) // Two layers of backtracking , Get the function information of the caller who writes the log if !ok {return}funcName := runtime.FuncForPC(pc).Name()funcName = path.Base(funcName) //Base Function returns the last element of the path , Keep only function names callerFields = append(callerFields, zap.String("func", funcName), zap.String("file", file), zap.Int("line", line))return}

Why don't zap.New(core, zap.AddCaller()) This way, , Add the caller information in the log line ？ I want to be more flexible , You can set the corresponding log fields by yourself , So the Caller Put several pieces of information in separate fields , After the logs are collected on the log platform , It is also more convenient to search logs .

Try using our encapsulated log in the following routine Logger Do a simple test .

package mainimport ("example.com/utils/zlog")type User strunct { Name stirng}func main() { user := &User{ "Name": "Kevin" } zlog.Info("test log", zap.Any("user", user))}

The output is similar to the following output .

{"level":"info","ts":"2022-05-15T21:22:22.687+0800","msg":"test log","res":{"Name":"Kevin"},"func":"main.Main","file":"/Users/Kevin/go/src/example.com/demo/zap.go","line":84}

summary

About Zap Logger Customization and packaging of , Here are just some basic and necessary entry-level customization , When you know , You can refer to the interface provided by the official document for more customization .

Source link  https://github.com/go-study-lab/go-http-server/blob/master/utils/zlog/log.go

More about Go Language Zap library Logger For customized packaging information, please pay attention to other relevant articles on the software development network ！