System design: load balancing

background

Load balancing begins with 20 century 90 years , At that time, hardware devices distributed traffic through the network . Organizations want to improve the accessibility of applications running on servers . Final , As the application delivers the controller （ADC） Appearance , Load balancing takes on more responsibility . They provide security and seamless access to applications during peak hours .

adc Divided into three categories ： Hardware device 、 Virtual device （ Basically software extracted from legacy hardware ） And software local load balancer . As computing moves to the cloud , Software adc Perform tasks similar to hardware . They also have additional functionality and flexibility . They enable organizations to quickly respond to needs in the cloud 、 Safely extend its application services . modern adc Allow organizations to integrate web-based services . These services include SSL/TLS uninstall 、 cache 、 Compress 、 Intrusion detection and detection web Application Firewall . This creates shorter delivery times and greater scalability .

Load balancing definitions

Load balancing is the process of distributing network traffic among multiple servers . This ensures that no server is burdened with excessive demand . By evenly distributing the work , Load balancing improves the responsiveness of applications . It also improves the usability of applications and websites to users . No load balancer , Modern applications cannot run . as time goes on , Software load balancer adds additional features including application security .

Load balancing

Load Balancer （LB） Is another key component of any distributed system . It helps distribute traffic across server clusters , To improve the application 、 Responsiveness and availability of websites or databases .LB It also tracks the status of all resources when distributing requests . If the server cannot receive new requests or does not respond or the error rate increases ,LB Will stop sending traffic to such servers .

Usually , The load balancer is located between the client and the server , Accept incoming network and application traffic , And use various algorithms to distribute the traffic on multiple back-end servers . By balancing application requests across multiple servers , Load balancers can reduce the load on a single server , And prevent any one application server from becoming a single point of failure , This improves the overall availability and responsiveness of the application .

Load balancing

To take full advantage of scalability and redundancy , We can try to balance the load of each layer of the system . We can add load balancing in three places ：

• In users and web Between servers （ Usually, the gateway layer in our sense ）

• stay web Server and internal platform layer （ Like an application ） Such as server or cache server （ Usually the application layer between applications in our sense ）

• Between the internal platform layer and the database .（ Usually, the data storage layer in our sense ）

If it corresponds to MVC Layer architecture , The above correspond to V The view is directly connected to the user's ,C Control the... Of direct docking applications ,M The model is directly connected to the data

Load balancing location

Load balancing OSI The role in the model

Load balancing can do more than just act as a network traffic policeman . Software load balancers provide benefits such as predictive analysis , Traffic bottlenecks can be identified before they occur . therefore , Software load balancers provide organizations with actionable insights . These are the keys to automation , Helps drive business decisions .

In the seven layer open systems interconnection （OSI） In the model , The network firewall is at level one to three （L1 Physical cabling 、L2 Data link and L3 The Internet ）. meanwhile , Load balancing occurs at the fourth to seventh layers （L4 Transport layer 、L5 The session layer 、L6 Presentation layer and L7 application layer ） Between .

The load balancer has different functions , Include ：

L4- According to the protocol from the network and transport layer （ Such as IP Address and TCP port ） Data guided communication .

L7- Add content switching to load balancing . This allows based on, for example HTTP head 、 Uniform resource identifiers 、SSL conversation ID and HTML Routing decision of form data and other attributes .

GSLB- Global server load balancing will L4 and L7 Extend functionality to servers in different geographic locations .

Load balancing and SSL

Secure socket layer （SSL） Is in web Standard security technology for establishing encrypted links between servers and browsers .SSL Traffic is usually decrypted on the load balancer . When the load balancer decrypts the traffic before delivering the request , be called SSL End . Load balancer makes web The server does not have to spend the extra... Required for decryption CPU cycle . This will improve application performance .

however ,SSL Termination can cause security problems . Load balancers and web Communication between servers is no longer encrypted . This can expose the application to possible attacks . however , When Load balancer and web Servers are located in the same data center when , The risk is reduced .

Another solution is SSL Pass on . The load balancer simply passes the encrypted request to web The server . then web The server decrypts . This will be web Use more on the server CPU Ability . But organizations that need additional security may find the extra overhead worthwhile .

Load balancing and security

As computing continues to move to the cloud , Load balancing plays an important role in security . The offload feature of the load balancer protects organizations from distributed denial of service （DDoS） attack . It does this by shifting attack traffic from corporate servers to public cloud providers . With DDoS The number and scale of attacks are increasing ,DDoS Attacks account for a large proportion of cybercrime . Hardware defense , Such as perimeter firewall , It could be expensive , Requires a lot of maintenance . Software load balancer with cloud offload provides efficient and cost-effective protection .

The benefits of load balancing

Faster user experience 、 Uninterrupted service . Users do not have to wait for a struggling server to complete the previous task . contrary , Their requests are immediately passed on to more accessible resources .

• Service provider experience Less downtime and higher throughput . Even a complete server failure will not affect the end-user experience , Because the load balancer just routes it to a healthy server .

• Load balancing makes it easier for system administrators to handle incoming requests , meanwhile Reduce user waiting time .

• Intelligent load balancer Provides benefits such as predictive analysis , Traffic bottlenecks can be identified before they occur . therefore , Intelligent load balancers provide organizations with actionable insights . These are the keys to automation , Helps drive business decisions .

• System administrator Fewer failures or pressure components are encountered . Load balancing is not about letting a single device do a lot of work , Instead, let multiple devices perform a small amount of work .

Load balancing algorithm

How the load balancer selects the back-end server ？

The load balancer considers two factors before forwarding requests to the back-end server . They will first ensure that the server of their choice actually responds appropriately to the request , Then a preconfigured algorithm is used to select one from a group of normal servers . We will soon discuss these algorithms .

Health check - The load balancer should only forward traffic to “ normal ” The back-end server . To monitor the health of the back-end server ,“ Health check ” Periodically try to connect to the back-end server , To ensure that the server is listening . If the server fails the health check , It will be automatically removed from the pool , And before it responds to the health check again , Will not forward traffic to it .

There are a variety of load balancing methods , They use different algorithms according to different needs .

• Minimum connection method - This method directs traffic to the server with the least active connections . When there are a large number of persistent client connections that are unevenly distributed between servers , This method is very useful .

• Minimum response time method - This algorithm directs traffic to the server with the least active connections and the lowest average response time .

• Minimum bandwidth method - This method selects the server currently serving the minimum traffic , In megabits per second （Mbps） Measured in units .

• Rotation training method - This method iterates through the server list , And send each new request to the next server . When it reaches the end of the list , It will start from scratch . When the server has the same specifications and there are not many persistent connections , It's the most useful .

• Weighted cycle method - Weighted round robin scheduling is designed to better handle servers with different processing capabilities . Assign a weight to each server （ Integer value indicating processing capacity ）. The server with higher weight receives new connections before the server with lower weight , A server with a higher weight gets more connections than a server with a lower weight .

•IP Hash - In this way , Computing client IP Hash of address , Redirect the request to the server .

As applications become more complex 、 Increasing user demand and traffic , Load balancing has become a necessity . Load balancers allow organizations to build flexible Networks , Without compromising safety 、 Service or performance , Able to meet new challenges .

Redundant load balancer

The load balancer may be a single point of failure ; To overcome this problem , A second load balancer can be connected to the first load balancer to form a cluster . Each load balancer monitors the health of the other load balancer , Because the two load balancers have the same service traffic and fault detection capability , So when the main load balancer fails , The second load balancer will take over

Active and standby load balancers

Reference material

What is load balancing ：https://avinetworks.com/what-is-load-balancing/

wiki Wikipedia ：https://en.wikipedia.org/wiki/Load_balancing_(computing)

Flexible and extensible design ：https://lethain.com/introduction-to-architecting-systems-for-scale/