A great idea leads to a great product, what if it goes viral and you find the next facebook or twitter in your hands, and it gets slower and slower as you get more users ? For the full stack, an important skill to solve such problems is load balancing .
What is load balancing
The term load originated from a typical system and refers to a device connected to a circuit that consumes electrical energy. The function of the load (electrical appliance) is to convert electrical energy into other forms of energy. By extension, one is an entity and the other is a transformation.
Therefore, for the entity, the content of the data field in the communication frame or message is called the payload, and the network load refers to the traffic carried by the network relay and the number of users carried by the network device.
The conversion is further explained as the usage of resources. The average system load is the load of the CPU, that is, the workload. The information it contains is not the usage of the CPU, but the sum of the number of processes that the CPU is processing and waiting for the CPU to process in a period of time. statistics.
Understand the load, then the load balancing is easy to understand. The definition given by wikipedia is as follows:
Load balancing is a computer networking technique used to distribute load among multiple computers (computer clusters), network connections, CPUs, disk drives, or other resources to optimize resource usage and maximize throughput , minimize the response time while avoiding overload. Using multiple server components with load balancing instead of a single component can increase reliability through redundancy. Load balancing services are usually done by dedicated software and hardware.
Moreover, Wikipedia's own system uses load balancing.
wikipedia
Each technology has its application scenarios and fields, and load balancing mainly solves system performance problems. However, if you understand the root cause, you can know that load balancing cannot be mentioned as a performance problem. If the load is reduced, a little less balancing may also solve the problem, such as caching.
DNS-based load balancing
DNS-based load balancing is the simplest method of load balancing, which can be said to be poor man's load balancing.
DNS maps domain names to IP addresses and vice versa. All core DNS servers are clustered, and the most used DNS server is probably BIND. When querying the DNS server, it is recommended to use dig; when querying DNS resolution, it is recommended to use nslookup. Using DNS caching can improve the performance of DNS resolution. An example of using Dig on mac is as follows:
dig usage
The load balancing implemented by DNS is very simple. The rotation method is adopted, as long as multiple A records are added for the domain name to be served.
E.g:
abel.com . _ IN A 168 .168 .168 .168
abel.com . _ IN A 168 .168 .168 .168
abel.com . _ IN A 168 .168 .168 .168
abel.com . _ IN A 168 .168 .168 .168
DNS-based load balancing is simple, easy to debug, and easy to scale. The flaw is that it has chronic amnesia and cannot preserve session information from one request to the next. Moreover, only the target service addresses are balanced, and the load intensity of request processing cannot be considered for balancing, and the fault tolerance is poor.
Service providers that support DNS load balancing include AWS Route 53 and dnspod.
HTTP load balancing
Load balancing solves performance problems, and you must first understand the status of a single server. In general, nginx has a higher response rate than Apache, so sometimes changing the web server can improve performance.
Ways to improve Apache Http are disabling the empty module, disabling DNS queries, using the compression module, not using the SymLinksIfOwnerMatch option, and enabling FollowSymLinks in the Directory option, etc.
Nginx itself is high-performance, but it can match the hardware platform of the server by adjusting worker_processes and worker_cpu_affinity, and can also treat compression differently and use its caching capabilities. E.g
Http{
gzip on;
gzip_static on;
gzip_comp_level 2;
gzip_types application/javascript;
}
HTTP的负载均衡相当于7层负载均衡，不论Apache 还是 Nginx 都可以充当HTTP的负载均衡器。
以基于权重的负载均衡为例，可以配置Nginx把请求更多地分发到高配置的后端服务器上，把相对较少的请求分发到低配服务器。配置的示例如下：
http{
upstream sampleapp {
server 192.168.1.23 weight=2;
server 192.168.1.24;
}
....
server{
listen 80;
...
location / {
proxy_pass http://myapp;
}
}
Nginx 作为负载均衡工作在7层，可以对做正则规则处理（如针对域名、目录进行分流等） ，配置简单，能ping通就能进行负载功能，可以通过端口检测后端服务器状态，不支持url检测。Nginx 负载均衡抗高并发，采用epoll网络模型处理客户请求，但应用范围受限。
数据库负载均衡
数据库负载均衡的一般用法从读写分离开始的，因为一般的应用都是读多写少的缘故吧。将数据库做成主从，主数据用于写操作，从数据库用于读操作，事务一般在主库完成。
数据库集群是数据库负载均衡的典型方式，集群管理服务器作为负载均衡器，例如mysql cluster。
更简单的方式是通过Haproxy 来完成负载均衡的调度。
Haproxy 均衡数据库
HAProxy能够补充Nginx的一些缺点比如Session的保持，Cookie的引导等工作，支持url检测后端的服务器出问题的检测会有很好的帮助。
HAProxy拥有更多的负载均衡策略比如：动态加权轮循(Dynamic Round Robin)，加权源地址哈希(Weighted Source Hash)，加权URL哈希和加权参数哈希(Weighted Parameter Hash）等，单纯从效率上来讲HAProxy更会比Nginx有更出色的负载均衡速度。
网络连接的负载均衡
LVS(IPVS，IP虚拟服务器)是在四层交换上设置Web服务的虚拟IP地址，对客户端是可见的。当客户访问此Web应用时，客户端的Http请求会先被第四层交换机接收到，它将基于第四层交换技术实时检测后台Web服务器的负载，根据设定的算法进行快速交换。常见的算法有轮询、加权、最少连接、随机和响应时间等。
LVS抗负载能力强，使用IP负载均衡技术，只做分发，所以LVS本身并没有多少流量产生。 LVS的稳定性和可靠性都很好应用范围比较广，可以对所有应用做负载均衡，缺陷是不支持正则处理，不能做动静分离。
通过LVS+Keepalived构建的LVS集群，LVS负载均衡用户请求到后端服务器，Keepalived的作用是检测web服务器的状态，如果有一台web服务器死机，或工作出现故障，Keepalived将检测到，并将有故障的web服务器从系统中剔除，当web服务器工作正常后Keepalived自动将web服务器加入到服务器群中，这些工作全部自动完成，不需要人工干涉，需要人工做的只是修复故障的web服务器。
下图是Keepalived的原理图：
KeepLived 的原理图
SSL负载均衡
信任是互联网的基石，出于安全性的考量，服务中往往需要SSL的连接。SSL 有两种认证方式：双向认证 SSL 协议要求服务器和用户双方都有证书；单向认证 SSL 协议不需要客户拥有CA证书。一般Web应用，配置SSL单向认证即可。但部分金融行业用户的应用对接，可能会要求对客户端(相对而言)做身份验证。这时就需要做SSL双向认证。
SSL 属于应用层的协议，所以只能在 7 层上来做，而 HAProxy 也是支持 SSL 协议的，所以一种方式是只需简单的让 HAProxy 开启 SSL 支持完成对内解密对外加密的处理, 但引入 SSL 处理是有额外的性能开销的（如上面谈到的认证）, 所以 一般采用SSL proxy farm， 典型的架构如下：
SSL 负载均衡
压力和负载测试
测试负载的状况，一般要涉及负载或压力测试。
负载测试是模拟实际软件系统所承受的负载条件的系统负荷，通过不断增加负载载（如逐渐增加模拟用户的数量）或其它加载方式来观察不同负载下系统的响应时间和数据吞吐量、系统占用的资源等，以检验系统的行为和特性，并发现系统可能存在的性能瓶颈、内存泄漏、不能实时同步等问题。
负载测试更多地体现了一种方法或一种技术。压力测试是在强负载（大数据量、大量并发用户等）下的测试，查看应用系统在峰值使用情况下操作行为，从而有效地发现系统的某项功能隐患、系统是否具有良好的容错能力和可恢复能力。压力测试分为高负载下的长时间（如24小时以上）的稳定性压力测试和极限负载情况下导致系统崩溃的破坏性压力测试。
压力测试可以被看作是负载测试的一种，即高负载下的负载测试，或者说压力测试采用负载测试技术。
简单地，httperf 或者Apache AB 就可以测量HTTP 服务器的负载性能。
云服务的负载均衡
云时代的到来，使负载均衡成了平台级的服务，几乎所有的云服务提供商都提供了负载均衡服务。下面是阿里云的负载均衡基础框架图：
阿里云的slb
In particular, the vpc of qingcloud is also quite unique. The private network is used for interconnection between hosts, which is similar to the use of a switch (L2 Switch) ad hoc LAN. The elastic IP is good, and managing the router is very considerate.
AWS's load balancing is still a model in the industry. The official schematic diagram is as follows:
AWS ELB
high availability
High availability is another value brought by load balancing, which is often used for failover. When one or more components fail, these components that can continue to provide services are continuously monitored. When a component does not respond, the load balancer will find it and stop sending data to it. Also when a component comes back online, the load balancer starts sending data to it again.
As an indicator of high availability, SLA generally has three time standards: 99.9%, 99.99%, 99.999%. It means that the offline time of uninterrupted operation does not exceed:
- 3 9s: 8.76 hours
- 4 9s: 52.26 hours
- 5 9s: 5.26 minutes
The disaster recovery plan between three and two places is not something that everyone can do. With cloud services, it becomes less difficult. The following is the disaster recovery diagram provided by Alibaba Cloud. It is deployed in multiple availability zones and can still work normally after the computer room is down.
The monitoring of the system plays a big role in the high availability of the system. I personally recommend zabbix.
Overall, load balancing is an important technology in system architecture and DevOps, and has a huge impact on system performance. Of course, if there is a higher demand, you need to consider a hardware load balancing solution, such as F5.