Glossary: load balancing load balance

Load balancing is a computer networking technique used to distribute load among multiple computers (computer clusters), network connections, CPUs, disk drives, or other resources to optimize resource usage and maximize throughput , minimize the response time while avoiding overload.

Using multiple server components with load balancing instead of a single component can improve reliability (HA) through redundancy. Load balancing services are usually done by dedicated software and hardware.

One of the most important applications of Internet-based service
load balancing is the use of multiple servers to provide a single service, which is sometimes referred to as a server farm. Generally, load balancing is mainly applied to Web sites, large Internet Relay Chat networks, high-traffic file download sites, NNTP (Network News Transfer Protocol) services and DNS services. Now load balancers also start to support database services, called database load balancers.

For Internet services, a load balancer is usually a software program that listens to an external port, through which Internet users can access the service, and the software serving as a load balancer will forward the user's request to the background intranet server, The intranet server returns the requested response to the load balancer, and the load balancer sends the response to the user, thus hiding the intranet structure from Internet users, preventing users from directly accessing the background (intranet) server, making the server more secure , which can block attacks on the core network stack and services running on other ports.

Some load balancers provide some special features to handle the situation when all the backend servers fail. For example forwarding requests to an alternate load balancer, displaying a message about service outages, etc. Load balancers allow IT teams to significantly improve fault tolerance. It can automatically provide substantial capacity to handle any increase or decrease in application traffic.

Architecture
High-performance systems typically use multiple layers of load balancing. Additional dedicated hardware load balancers as well as pure software load balancers, including open source ones such as the mod proxy balancer extension for the Apache web server, nginx, Varnish and Pound reverse proxies and load balancers. Use Gearman to distribute suitable computing tasks to multiple computers, so that such a large number of tasks can be completed faster.
For a multi-tier architecture, there are two designs behind the load balancer or network distributor, termed Bowties and Stovepipes. In the Stovepipe design, transactions are distributed from the top, and then travel from a fixed channel through a series of hardware and software devices to their final destination. With Bowties design, there are multiple paths for transaction processing in each layer to choose from. In the network structure of transaction processing, it may be Stovepipes, or Bowties, or a grocery architecture according to the actual needs of each layer.

Persistence [Session Sharing]
An important issue that the load balancer needs to deal with is: how to save user sessions? If the session information is stored in the background server, the user's next request may be assigned to a different background server, and the user session cannot continue at this time. A load balancer can cache user sessions and then distribute user requests to different backend servers. But this will bring the load problem of the load balancer.

One solution is to send all requests in a user session to the same background server. Namely persistence or stickiness. The disadvantage of this method is that it cannot tolerate failover. If the background server fails, the session provided by it will not be available, and any sessions that depend on it will be lost. This problem is usually related to the data center. Although the Web Service is not connection-oriented, the back-end database is inherently connection-oriented.

The second scheme is to allocate the server providing the service according to the user name and the client IP, or it can be randomly allocated. Because the client may connect to the Internet through DHCP, NAT or Web proxy, its IP address may change frequently, which makes the service quality of this solution unguaranteed. Random allocation is stored by the load balancer in the session information store. If the load balancer is replaced or fails, this information may be lost; in addition, when the load (load balancer) is high, in order to ensure that the allocation table space will not be exhausted, the time-out allocation information must be deleted. The random assignment method also requires the client to maintain session state, which can cause problems if cookies are disabled in the client's browser. Good load balancers use a variety of persistence (session persistence) techniques to avoid failures when some of them are unavailable.

Another solution is to store each session information in a database. Since this scheme will increase the load of the database, this scheme is not good for performance improvement. The database is best used to store session data with long sessions. In order to avoid a single point of failure of the database and improve its scalability, the database is usually replicated on multiple servers, and requests are distributed to the database server through a load balancer. The state server technology in Microsoft ASP.net is a typical example of a session database. All servers in the cluster save their session information to the state server, and they can query the state server for session data.

Fortunately there is a more efficient way, usually the client browser can save the user's per-session information. For example, using browser cookies, encrypting the data and adding a timestamp can ensure security; URL rewriting. Storing session information on the client is usually the preferred solution, as this gives the load balancer the flexibility to choose a backend server to handle user requests. However, this method is not suitable for some more complex e-commerce, because the session data in e-commerce is large, and the server needs to reprocess the session information frequently; at the same time, URL rewriting has security problems because users can change the session flow data ; Encrypting client-side cookies has also been controversial for security, unless all sessions go over HTTPS, which is vulnerable to man-in-the-middle attacks.

Relationship to Failover
Load balancing is very useful for redundancy of communication links. For example, a company may have multiple Internet access lines to ensure that one can still connect to the Internet in the event of a failure.
A failover architecture means that one connection is in normal use, and the other is a backup, only enabled when the first fails.

With a load balancer, both (multiple) connections can be used. There is a device or program that monitors the connectivity of all connections in real time and routes the packets being sent. Using multiple connections simultaneously can increase bandwidth.
Many telecommunications companies have multiple lines available on their internal network or connected to external networks (other telecommunications networks). To avoid network congestion on one link, minimize the cost of connecting to other networks, or improve network reliability, they use load balancing to shift traffic from one link to another.
Load balancing is often used for failover - continuous service when one or more components fail If the component is not responding, the load balancer will notice and stop sending data to it. Also when a component comes back online, the load balancer starts sending data to it again. In order to work properly as described above, there must be at least one redundant service in the load balancing system. It is more economical and flexible to adopt the one-use-one-standby scheme (one component provides services, one standby, when the main component fails, the standby component will take over and continue to provide services) than the failover scheme. Some types of RAID systems use a hot-spare feature similar to this.