Deploy loglistener in tke container to collect logs to CLS

tke The business log collection of the cluster container usually uses the log collection component to collect logs , But there is one problem , The log collection cannot be delivered to cls, That is to say, the logs of the cluster can only be delivered to those in the same region cls in , But in some areas there is no cls Service , Then you cannot use the log collection service to collect tke Cluster container log to cls 了 . There is another way to deploy in the container LogListener, Post the container log to via the public network cls in . Today, let's talk about the specific implementation of this log collection method .

We're here docker Build a mirror image , take LogListener Install it into our image , Our image build directory is as follows

Be careful ： The application in my example is demo, We will replace it with our own business procedures according to the actual situation , Corresponding dockerfile It also needs to be modified

[[email protected] log-collect]# ll
total 11824
-rwxrwxrwx 1 root root 12098786 Jul  3 18:09 demo
-rw-r--r-- 1 root root      566 Jul  3 18:19 Dockerfile
-rwxr-xr-x 1 root root       56 Jul  3 18:19 start.sh

• demo： One go Program demo The executable of , The listening port is 3000, The log is in /data Under the table of contents
• Dockerfile： Used for image building
• start.sh： The startup script of the image

Let's take a look at the corresponding file contents

demo Files can be obtained directly from the connection https://github.com/nieweixing/go-test/blob/main/main/demo

Dockerfile Is as follows ,secretid and secretkey Replace with your own , If it is a public network, you need to add parameters -network internet

FROM centos:7
RUN yum install wget -y && mkdir -p /data
RUN wget https://mirrors.tencent.com/install/cls/loglistener-linux-x64-2.6.1.tar.gz  && tar -zxvf loglistener-linux-x64-2.6.1.tar.gz -C /usr/local && cd /usr/local/loglistener-2.6.1/tools && ./loglistener.sh install
RUN /usr/local/loglistener-2.6.1/tools/loglistener.sh init -secretid AKIDJNM71vniy85mCNxxxxxx -secretkey 608zbx5wmY4YyF9uRitsxxxxx -region ap-guangzhou -network internet
COPY demo /root
COPY start.sh /root/start.sh
CMD ["/root/start.sh"]

LogListener Installation reference documentation for https://cloud.tencent.com/document/product/614/17414

start.sh Is as follows

#!/bin/bash

/etc/init.d/loglistenerd start
./root/demo

Create a directory , Then access the file to the directory , Execute the image build command to build an image

docker build -t xxxx .

You can also use the good image I built to test ccr.ccs.tencentyun.com/nwx_registry/log-to-cls:latest

After the image is built , Here we are cls It is in Guangzhou , We deployed the image to Beijing tke colony , See if it can be delivered to across regional public networks cls

We exposed a public network clb Type of service To provide access to

And then we have cls There are machine groups and topics configured to save logs , To configure the machine group, you need to go to the container to get the following group ip, stay cls Configuring machine groups ip Is to fill in group ip, Not a container ip.

[[email protected] /]# /etc/init.d/loglistenerd check | grep "group ip"
group ip:172.18.0.15

We went to the cls Console create machine group

Then create a log topic to associate with the machine group created in the previous step

The acquisition path configured here is /data/ All under log Log file at the end

cls After the theme is configured , Let's visit the corresponding service, Then the program prints some logs , And then to cls Search

for i in {1..100}; do curl 49.xx.xx.xx:3000/health_check; done

First, let's go to the container to see if the log is generated

[[email protected] data]# tail /data/go.log
2021/07/03 11:02:33  I visited Health check This path 
2021/07/03 11:02:33  I visited Health check This path 
2021/07/03 11:02:33  I visited Health check This path 
2021/07/03 11:02:34  I visited Health check This path 
2021/07/03 11:02:34  I visited Health check This path 
2021/07/03 11:02:34  I visited Health check This path 
2021/07/03 11:02:34  I visited Health check This path 
2021/07/03 11:02:34  I visited Health check This path 
2021/07/03 11:02:34  I visited Health check This path 
2021/07/03 11:02:34  I visited Health check This path 

Enter the container to view the log file. The access log is generated , Then we go to cls Search below

cls Only logs can be retrieved , Here we use LogListener The collection container log is posted to cls Was a success