All systems first experienced an era when a single machine did all the business. One program + one MySQL database can meet the requirements for development and the first version to go online. With the increase of data and the growth of business, these applications face a problem of expansion and expansion of access. The simplest expansion is horizontal expansion, which originally increased from one MySQL to two or more to form a cluster, so the simplest capability is to provide stronger service capabilities. If the original traffic supported 1000 per second, it can now support 2000 (ideal value), which is equivalent to distributing the service capacity to multiple nodes. There are many issues involved, the first is the mutual backup of data, and then how to allocate computing power and how to access it from the outside. This article introduces HaProxy and KeepAlived mainly to deal with an external access problem.
In the following introduction, it is assumed that there are 2 mysql, namely mysqlA and mysqlB.
haproxy request distribution
It can be understood that nginx is used for back-end load balancing. HaProxy can provide external capabilities by listening to a unified port, and then distribute them internally. In addition to supporting http7-layer processing, it also supports 4-layer forwarding for mysql by the way. (You can consider using lvs for more advanced ones) Here, configure the two mysqls in the backend respectively, and check the corresponding services through option mysql-check.
When the program accesses, it no longer accesses the specific mysql machine, but accesses the machine where the HaProxy is located. It is mentioned here that an additional machine is required to provide services, but since it is only used by HaProxy, its basis is very low, and a minimum machine is sufficient, and the cost is not large. At the same time, the corresponding port can also fill in the port exposed by HaProxy. It is considered that there is only one mysql to the outside world, that is, it is transparent to the outside world.
When HaProxy is processing, it forwards itself according to the corresponding strategy. The simplest strategy is round robin (ribbon). Of course, other weighted or random, etc., need to be configured in detail. At the same time, according to the specific time interval set, the validity of the back-end service is checked. When mysqlA or B cannot work, it will be automatically removed from the available list.
After adding HaProxy, the load problem is solved, but another problem comes again, that is, the problem of single point of service. If once this HaProxy machine hangs (or network reasons). Although the mysql server is not down, the entire service is also unavailable. The front-end program doesn't automatically fall back to accessing the original mysql (it can't even access it due to firewall issues). At this time, another thing is used to ensure that HaProxy will not become a single point, namely KeepAlived.
KeepAlived High Availability
The way to ensure that the service will not be a single point is to add a machine, but if you add a machine, multiple IPs will appear. How to ensure that the front-end program uses a single IP and can ensure that the actual processing machine of the back-end is multiple, this is the role of KeepAlived.
In order to ensure the high availability of HaProxy, we have added another machine, namely HaProxyA and HaProxyB (equivalent to the original 2 mysql machines, plus 2 HaProxy machines).
Through KeepAlived, we can create a third IP, which is provided by ip3 to provide services to the outside world. However, unlike the forwarding nature of HaProxy, ip3 here is actually a mapping of the same name between HaProxyA and HaProxyB. It can be understood that both HaProxyA and HaProxyB are vying for this ip, and whichever is vying for it will provide the service. It can be understood that within a certain period of time, the following relationship is guaranteed
IP3<=>IP1 IP2 wait IP3
Because KeepAlived does not provide any processing power, in fact the final processing can fall on the program that can process the information. Therefore, we need to deploy KeepAlived and HaProxy together, that is, KeepAlived is responsible for grabbing ip and receiving front-end requests. After receiving the request, the system automatically distributes the request to HaProxy on the same machine for processing. That is, a machine has 2 IPs, ip1 is responsible for receiving requests, and ip2 is responsible for actual information processing (metaphorically, how to monitor requests and port handlers)
In the previous processing model, because KeepAlived does not process requests, this model is actually problematic if HaProxy on its machine is not available. Therefore, KeepAlived has to monitor whether HaProxy on its own machine is valid. In the configuration, this effect can be achieved through the track_script directive, which is similar to the working mode of HaProxy. It can periodically execute the monitoring script to check whether HaProxy is available. If it is not available, there are two ways to deal with it, one is to forcibly restart HaProxy, and the other is to cancel the preempted ip position (such as killing yourself) and give up the corresponding ip3 position. In this way, IP2 can be automatically bound (metaphorically) with IP3, that is, IP2 provides processing power.
Based on the configuration of KeepAlived, only one machine is working at a single moment, and another machine is preparing. It can be understood that the service capacity here is only half. Fortunately, the resources occupied by KeepAlived and HaProxy are small and the cost is not high.
final deployment model
- MysqlA (ip5) and MysqlB (ip6) expand horizontally, communicate data through dual masters, and provide service capabilities at the same time
- Provide the load capacity of mysql through HaProxyA (ip3) and HaProxyB (ip4), route requests to the specified mysql server, and monitor the availability of the mysql database on the backend
- Deploy KeepAlivedA and KeepAlivedB with HaProxyA and HaProxyB respectively, bind VIP ip1 at the same time, provide external access ip, and monitor the availability of the local HaProxy
Through the above deployment, a front-end data access may be the following access path
IP1=>IP3->IP5
IP1=>IP3->IP6
IP1=>IP4->IP5
IP1=>IP4->IP6
Note: This article assumes that mysql is a dual-active deployment, not active-standby. The main consideration is not to waste resources, and there is no additional backup, mainly to simplify the model