CLB health check principle and abnormal troubleshooting

One 、 Health check on the seventh floor

Health check on the seventh floor , Use HTTP agreement , Support GET、HEAD Two request methods ,HEAD Get only header information , Don't get the actual content , Lighter detection , Two ways , It's all dependent on RS Back to HTTP CODE Compare with the set health status code ( The default is 1xx、2xx、3xx、4xx), If it is not within the range of health status code or no status code is returned within the response timeout time and the number of times the unhealthy threshold is reached , Is judged to be unhealthy .

You can see , After three handshakes and establishment of Association , The first 12 Package No HEAD request , The first 19 Bag No RS The normal response is accompanied by HTTP Status code 200 OK,CLB Get 200 After the status code , Compare with the checked range of status code detection , The comparison is within the range , Deemed healthy .

And under the seven layer monitor ,CLB To RS It is equivalent to a reverse proxy process , This link uses a short connection , Each request RS Create a new connection ;

At the same time, it can be seen from the above figure , It will not be normal and RS wave , It is RST,ACK Disconnect , This can save some of the extra traffic from the normal handshake .

Two 、 Health check on the fourth floor

TCP/HTTP

Health check support for the four layer monitor TCP、HTTP、 There are three custom protocols , The first two are the mainstream usage ：

Four layer listener , As the name implies, the transport layer protocol , by IP:PORT It's a new way of detecting , If the check port is not filled in , The default is back end RS Listening port of , The checking logic is simple , Probe whether the port is normally connected ：

CLB issue SYN Packet request handshake , Get the opposite SYN,ACK Then it is regarded as healthy , You don't need to shake hands , next RST,ACK Disconnect ;

four layers TCP Listeners also support HTTP It's a new way of detecting , The detection principle is the same as that of the seven layer monitor .

UDP

udp Detection is divided into Check port and ping Probe ：

1. Check the probe logic of the port

Inspection request 、 Check if the returned result is not filled in , When the following two conditions are met at the same time , The health examination is considered normal , Otherwise abnormal ：

• Ping Detection is normal
• UDP Detection port ,RS No echo xxx Port Unreachable

2. Specifies the detection logic for echoing text

When checking requests 、 Check that the returned result is filled in text or hexadecimal ,CLB When detecting, you will carry the filled content to detect RS port , When RS When returning to a package with specific content , Is considered healthy , Otherwise it's not healthy ：

Use socat test ,LB Ask for something to carry ,RS Just echo something ：

socat -v udp-l:2115,fork exec:'/bin/cat'

RS Upper grab bag ：

The health check is set to normal ：

3、 ... and 、 Troubleshooting steps for abnormal health check

1. Make sure the security team 、iptables Waiting will not be an obstacle

CLB The probe will carry its own by default VIP To request RS, If RS Failed to release VIP Or health check port ,CLB The expected response is not received within the timeout range , It will be judged as timeout , The correct approach ：

• Make sure RS Whether the security group is released VIP, If the security group pass through function is enabled , From LB The request to come in will not be seen again RS Our security group ;
• Make sure iptables Wait for the software to release the source normally , Even if the security group pass through function is enabled ,OS Layer software can still intercept requests , If not, use the following command to release ： iptables -I INPUT -s [clb vip] -d 0.0.0.0/0 -j ACCEPT , Let go CLB Request , This can only ensure VIP The probe request of can reach RS,client visit CLB when ,CLB Can carry client Of IP to RS, If you don't want to limit client visit , Then a certain network segment or all network segments are put through : iptables -I INPUT -s 0.0.0.0/0 -d 0.0.0.0/0 -j ACCEPT, Or empty it directly iptables The rules ：

iptables-sve > iptables-rules.bak  # A copy can be saved before emptying 
iptables -F
iptables -X  

2. If it's a traditional account , Public network LB, Make sure RS There is public network bandwidth

About account attributes , Refer to official documents :

• Standard account type billing instructions
• Traditional account type billing instructions
• How to judge account type

The bandwidth billing of traditional accounts is managed on specific instances , So if it is a public network CLB, The health check detection uses the public network VIP, Then you need to ensure the binding back end RS Also have the public network capability , Even if there is no public network IP You can also purchase public network bandwidth ,cvm Console --> Resource adjustment --> Adjust network ：

3. Make sure to check whether the port listens normally

For example, judge the health examination 80 Whether the port is listening normally ：

netstat -lntup|grep 80
lsof -i :80
ss -lntup|awk '$5~/80/{print $5}' 

If there is no normal monitoring , Check whether the corresponding service is running normally .