Tencent cloud layer 7 load balancing log analysis and monitoring

LB common problem

Abnormal location

• A lot of QPS Scene , A few client requests are abnormal ,RS The client did not receive the request ,LB Whether to receive , No way to judge .
• The end customer feedback part of the request is abnormal , For example, slow ,RS logging response_time normal , Where does the time come from ？
• Some time , Intranet 7 Layer request exception , What's the problem

Statistical analysis

• Expect a full link time-consuming Topology ,request_time,connect,response_time.
• Open the http2, Whether to take effect , What is the proportion of the entire agreement ？
• The core domain names are distributed on different instances , Request proportion analysis .

LB7 Layer access CLS The way

Single instance access

• Select the corresponding 7 Layer instance , Click the small round pen to edit .

• open " log enable " switch

• Select the corresponding log set and log topic , If there is no appropriate log set or log topic , You can go to " Access log " New page , Then click submit to complete .

• To log set management , Select the corresponding log topic (topic) Edit index .

• There are logs coming in , You can choose to automatically configure , It is recommended to open all " Turn on Statistics ", For the subsequent statistical analysis function .

Batch access （ establish CLB Proprietary log sets ）

remarks ： At present, batch access , Need to find CLB Product launch white list , To see the entrance .

Suggest ： Different log topics are distinguished according to the actual business situation , such as http layer , Cache layer , Data layer or according to business dimension , Financial business , Main station service , Order business, etc , because CLS It is also a pipe , Different paths can be selected for different log topics , To COS,CKAFKA,SCF And so on , Archive data .

• After opening the white list , Click on " Access log " Button , Get into clblog Log set configuration page .

• CLB The log set name is fixed , No need to fill in , You can select a save time .

• New blog topic , It is recommended to fill in according to the actual situation , Select add corresponding LB example , Save it .

• Save complete , wait a moment 5·10 Minutes .

• Index on and statistics on , Refer to single instance access .

LB Description of log field meaning

Reference link ：https://cloud.tencent.com/document/product/214/41379

Grammar and rules

For specific syntax, please refer to ：https://cloud.tencent.com/document/product/614/47044

Rule of grammar

Examples of retrieval statements

The actual scene uses

• Retrieve one URL, An example of a request time greater than a certain value

request:"HEAD /aaa/ HTTP/1.1" AND request_time:>0.005

• View a certain rs 4xx Requested log

status:[400 TO 500} AND upstream_addr:"10.0.1.12:80" 

Analysis statistics add Dashboard

For an introduction to the analysis, please refer to ：https://cloud.tencent.com/document/product/614/44061

Time market

• For example, the average request duration , Real time observation of website response ：

* | SELECT HISTOGRAM(CAST(__TIMESTAMP__ AS TIMESTAMP), INTERVAL 1 MINUTE) AS dt,AVG(request_time) AS " Average request processing time /clb example ", server_addr GROUP BY dt, server_addr order by dt

The processing time of each stage can be clearly judged in detail , Fastest location problem .

Large capacity

• For example, real-time to all rs Number of requests for , Judge each rs Load capacity ：

* | SELECT HISTOGRAM(CAST(__TIMESTAMP__ AS TIMESTAMP), INTERVAL 1 MINUTE) AS dt, COUNT(1) AS " Every minute to rs Number of requests for ", upstream_addr GROUP BY dt, upstream_addr order by dt

Status code market

• For example, the status code of each dimension is counted in real time , To understand the health level ：

* | SELECT HISTOGRAM(CAST(__TIMESTAMP__ AS TIMESTAMP), INTERVAL 1 MINUTE) AS dt, count(1), status GROUP BY dt, status order by dt

Add statistical analysis charts to Dashboard

Select the corresponding icon , Click on " Add to dashboard " that will do ：

Real time alarm

Alarm introduction reference ：https://cloud.tencent.com/document/product/614/51741

Add alarm rules flexibly based on retrieval analysis , Add an alarm strategy according to the alarm rules , Alert notification to wechat , Enterprise WeChat ,webhook etc. , Real time receiving line LB The health of .