I. Introduction
Mysql-cluster itself achieves high availability, but its SQL nodes do not implement load balancing. How can the program access the SQL nodes in a balanced manner? Which database server IP should be written in the program?
LVS implements load balancing, that is, the client's needs are distributed to the back-end Web application server using a specific load balancing algorithm.
Keepalived is used to achieve high availability, providing health checks and failover, that is, ensuring that after the main LVS goes down, the slave LVS can take over in a short time, thereby ensuring the stability of the system or website. After Keepalived cannot receive the information from the master node in the back-end realserver, keepalived can call the ipvsadm command to generate rules by itself. It can be automatically implemented and "take over" the VIP and ipvs rules of the master node, apply them to the slave node, and continue to serve users. .
Just write the virtual IP in the program.
Topology:
2: Experimental environment
Mysql cluster has been installed
Mysql cluster installation, please refer to: https://blog.csdn.net/hualinger/article/details/131092300
Three: Experimental steps
3.1: Install ipvsadm
Installed on lvs-master (master scheduler) and lvs-backup (slave scheduler) respectively.
#downloadipvsadm
Click here to download ipvsadm:
http://www.linuxvirtualserver.org/software/kernel-2.6/ipvsadm-1.24.tar.gz
#View kernel version
uname -r
2.6.32-431.el6.x86_64
First cd /usr/src/kernels/ to see if the file exists:
2.6.32-431.el6.x86_64
If not, you need: yum -y install kernel-devel
Otherwise, an error will be reported during subsequent compilation: such as libipvs.h:14:23: error: net/ip_vs.h: No such file or directory
#Create soft connection
ln -s /usr/src/kernels/2.6.32-573.3.1.el6.x86_64/ /usr/src/linux
Note: When making a soft connection, you need to provide the absolute path of 2.6.32-573.3.1.el6.x86_64. If you directly enter the /usr/src/kernels directory, this soft connection will not work: ln -s 2.6.32-573.3. 1.el6.x86_64/ /usr/src/linux
#compile
cd /download
tar -xvf ipvsadm-1.24.tar.gz
cd ipvsadm-1.24
[root@ser6-70 ipvsadm-1.24]# make
make -C libipvs
make[1]: Entering directory `/download/ipvsadm-1.24/libipvs'
gcc -Wall -Wunused -Wstrict-prototypes -g -O2 -I/usr/src/linux/include -DHAVE_NET_IP_VS_H -c -o libipvs.o libipvs.c
ar rv libipvs.a libipvs.o
ar: creating libipvs.a
a - libipvs.o
make[1]: Leaving directory `/download/ipvsadm-1.24/libipvs'
gcc -Wall -Wunused -Wstrict-prototypes -g -O2 -I/usr/src/linux/include -I.. -I. -DVERSION=\"1.24\" -DSCHEDULERS=\""rr|wrr|lc|wlc|lblc|lblcr|dh|sh|sed|nq"\" -DHAVE_NET_IP_VS_H -c -o ipvsadm.o ipvsadm.c
ipvsadm.c: In function ‘print_largenum’:
ipvsadm.c:1564: warning: field width should have type ‘int’, but argument 2 has type ‘size_t’
gcc -Wall -Wunused -Wstrict-prototypes -g -O2 -I/usr/src/linux/include -I.. -I. -DVERSION=\"1.24\" -DSCHEDULERS=\""rr|wrr|lc|wlc|lblc|lblcr|dh|sh|sed|nq"\" -DHAVE_NET_IP_VS_H -c -o config_stream.o config_stream.c
gcc -Wall -Wunused -Wstrict-prototypes -g -O2 -I/usr/src/linux/include -I.. -I. -DVERSION=\"1.24\" -DSCHEDULERS=\""rr|wrr|lc|wlc|lblc|lblcr|dh|sh|sed|nq"\" -DHAVE_NET_IP_VS_H -c -o dynamic_array.o dynamic_array.c
gcc -Wall -Wunused -Wstrict-prototypes -g -O2 -o ipvsadm ipvsadm.o config_stream.o dynamic_array.o libipvs/libipvs.a
[root@ser6-70 ipvsadm-1.24]# make install
make -C libipvs
make[1]: Entering directory `/download/ipvsadm-1.24/libipvs'
make[1]: Nothing to be done for `all'.
make[1]: Leaving directory `/download/ipvsadm-1.24/libipvs'
if [ ! -d /sbin ]; then mkdir -p /sbin; fi
install -m 0755 -s ipvsadm /sbin
install -m 0755 ipvsadm-save /sbin
install -m 0755 ipvsadm-restore /sbin
[ -d /usr/man/man8 ] || mkdir -p /usr/man/man8
install -m 0644 ipvsadm.8 /usr/man/man8
install -m 0644 ipvsadm-save.8 /usr/man/man8
install -m 0644 ipvsadm-restore.8 /usr/man/man8
if [ -d /etc/rc.d/init.d ]; then \
install -m 0755 ipvsadm.sh /etc/rc.d/init.d/ipvsadm; \
fi
#Verify whether the installation is successful
[root@ser6-70 ipvsadm-1.24]# /sbin/ipvsadm -v
ipvsadm v1.24 2005/12/10 (compiled with getopt_long and IPVS v1.2.1)
The installation is successful.
3.2: Install keepalived
Installed on lvs_master (master scheduler) and lvs-backup (slave scheduler) respectively.
Before installing keepalived, you must first install popt-devel and openssl:
yum install popt-devel -y
yum install -y openssl openssl-devel
#Install keepalived
wget http://www.keepalived.org/software/keepalived-1.1.19.tar.gz
#decompression
tar zxvf keepalived-1.1.19.tar.gz
#initialization
[root@ser6-70 linux]# cd keepalived-1.1.19
[root@ser6-70 keepalived-1.1.19]# ./configure --prefix=/usr/local/keepalived
checking for gcc... gcc
checking for C compiler default output file name... a.out
checking whether the C compiler works... yes
checking whether we are cross compiling... no
checking for suffix of executables...
checking for suffix of object files... o
checking whether we are using the GNU C compiler... yes
checking whether gcc accepts -g... yes
checking for gcc option to accept ISO C89... none needed
checking for a BSD-compatible install... /usr/bin/install -c
checking for strip... strip
checking how to run the C preprocessor... gcc -E
checking for grep that handles long lines and -e... /bin/grep
checking for egrep... /bin/grep -E
checking for ANSI C header files... yes
checking for sys/wait.h that is POSIX.1 compatible... yes
checking for sys/types.h... yes
checking for sys/stat.h... yes
checking for stdlib.h... yes
checking for string.h... yes
checking for memory.h... yes
checking for strings.h... yes
checking for inttypes.h... yes
checking for stdint.h... yes
checking for unistd.h... yes
checking fcntl.h usability... yes
checking fcntl.h presence... yes
checking for fcntl.h... yes
checking syslog.h usability... yes
checking syslog.h presence... yes
checking for syslog.h... yes
checking for unistd.h... (cached) yes
checking sys/ioctl.h usability... yes
checking sys/ioctl.h presence... yes
checking for sys/ioctl.h... yes
checking sys/time.h usability... yes
checking sys/time.h presence... yes
checking for sys/time.h... yes
checking openssl/ssl.h usability... yes
checking openssl/ssl.h presence... yes
checking for openssl/ssl.h... yes
checking openssl/md5.h usability... yes
checking openssl/md5.h presence... yes
checking for openssl/md5.h... yes
checking openssl/err.h usability... yes
checking openssl/err.h presence... yes
checking for openssl/err.h... yes
checking for MD5_Init in -lcrypto... yes
checking for SSL_CTX_new in -lssl... yes
checking for poptGetContext in -lpopt... yes
checking for kernel version... 2.6.32
checking net/ip_vs.h usability... yes
checking net/ip_vs.h presence... yes
checking for net/ip_vs.h... yes
checking for IPVS syncd support... yes
checking for an ANSI C-conforming const... yes
checking for pid_t... yes
checking whether time.h and sys/time.h may both be included... yes
checking whether gcc needs -traditional... no
checking for working memcmp... yes
checking return type of signal handlers... void
checking for gettimeofday... yes
checking for select... yes
checking for socket... yes
checking for strerror... yes
checking for strtol... yes
checking for uname... yes
configure: creating ./config.status
config.status: creating Makefile
config.status: creating genhash/Makefile
config.status: WARNING: 'genhash/Makefile.in' seems to ignore the --datarootdir setting
config.status: creating keepalived/core/Makefile
config.status: creating keepalived/include/config.h
config.status: creating keepalived.spec
config.status: creating keepalived/Makefile
config.status: WARNING: 'keepalived/Makefile.in' seems to ignore the --datarootdir setting
config.status: creating lib/Makefile
config.status: creating keepalived/vrrp/Makefile
config.status: creating keepalived/check/Makefile
config.status: creating keepalived/libipvs-2.6/Makefile
Keepalived configuration
------------------------
Keepalived version : 1.1.19
Compiler : gcc
Compiler flags : -g -O2
Extra Lib : -lpopt -lssl -lcrypto
Use IPVS Framework : Yes
IPVS sync daemon support : Yes
Use VRRP Framework : Yes
Use Debug flags : No
All of the following are YES, indicating that the configuration is successful.
Use IPVS Framework : Yes
IPVS sync daemon support : Yes
Use VRRP Framework : Yes
#compile
[root@ser6-70 keepalived-1.1.19]# make
make -C lib || exit 1;
make[1]: Entering directory `/download/keepalived-1.1.19/lib'
gcc -g -O2 -I. -Wall -Wunused -Wstrict-prototypes -c memory.c
gcc -g -O2 -I. -Wall -Wunused -Wstrict-prototypes -c utils.c
gcc -g -O2 -I. -Wall -Wunused -Wstrict-prototypes -c notify.c
gcc -g -O2 -I. -Wall -Wunused -Wstrict-prototypes -c timer.c
gcc -g -O2 -I. -Wall -Wunused -Wstrict-prototypes -c scheduler.c
gcc -g -O2 -I. -Wall -Wunused -Wstrict-prototypes -c vector.c
gcc -g -O2 -I. -Wall -Wunused -Wstrict-prototypes -c list.c
gcc -g -O2 -I. -Wall -Wunused -Wstrict-prototypes -c html.c
gcc -g -O2 -I. -Wall -Wunused -Wstrict-prototypes -c parser.c
gcc -g -O2 -I. -Wall -Wunused -Wstrict-prototypes -c signals.c
gcc -g -O2 -I. -Wall -Wunused -Wstrict-prototypes -c logger.c
make[1]: Leaving directory `/download/keepalived-1.1.19/lib'
make -C keepalived
make[1]: Entering directory `/download/keepalived-1.1.19/keepalived'
make[2]: Entering directory `/download/keepalived-1.1.19/keepalived/core'
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_WITH_VRRP_ -c main.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_WITH_VRRP_ -c daemon.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_WITH_VRRP_ -c pidfile.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_WITH_VRRP_ -c layer4.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_WITH_VRRP_ -c smtp.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_WITH_VRRP_ -c global_data.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_WITH_VRRP_ -c global_parser.c
make[2]: Leaving directory `/download/keepalived-1.1.19/keepalived/core'
make[2]: Entering directory `/download/keepalived-1.1.19/keepalived/check'
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -D_WITH_VRRP_ -c check_daemon.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -D_WITH_VRRP_ -c check_data.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -D_WITH_VRRP_ -c check_parser.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -D_WITH_VRRP_ -c check_api.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -D_WITH_VRRP_ -c check_tcp.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -D_WITH_VRRP_ -c check_http.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -D_WITH_VRRP_ -c check_ssl.c
check_ssl.c: In function ‘build_ssl_ctx’:
check_ssl.c:79: warning: assignment discards qualifiers from pointer target type
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -D_WITH_VRRP_ -c check_smtp.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -D_WITH_VRRP_ -c check_misc.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -D_WITH_VRRP_ -c ipwrapper.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -D_WITH_VRRP_ -c ipvswrapper.c
make[2]: Leaving directory `/download/keepalived-1.1.19/keepalived/check'
make[2]: Entering directory `/download/keepalived-1.1.19/keepalived/vrrp'
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -c vrrp_daemon.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -c vrrp_data.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -c vrrp_parser.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -c vrrp.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -c vrrp_notify.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -c vrrp_scheduler.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -c vrrp_sync.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -c vrrp_index.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -c vrrp_netlink.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -c vrrp_arp.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -c vrrp_if.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -c vrrp_track.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -c vrrp_ipaddress.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -c vrrp_iproute.c
gcc -g -O2 -I/usr/src/linux/include -I../include -I../../lib -Wall -Wunused -Wstrict-prototypes -D_KRNL_2_6_ -D_WITH_LVS_ -D_HAVE_IPVS_SYNCD_ -c vrrp_ipsecah.c
make[2]: Leaving directory `/download/keepalived-1.1.19/keepalived/vrrp'
make[2]: Entering directory `/download/keepalived-1.1.19/keepalived/libipvs-2.6'
gcc -g -O2 -I/usr/src/linux/include -Wall -Wunused -c -o libipvs.o libipvs.c
ar rv libipvs.a libipvs.o
ar: creating libipvs.a
a - libipvs.o
rm libipvs.o
make[2]: Leaving directory `/download/keepalived-1.1.19/keepalived/libipvs-2.6'
Building ../bin/keepalived
strip ../bin/keepalived
Make complete
make[1]: Leaving directory `/download/keepalived-1.1.19/keepalived'
make -C genhash
make[1]: Entering directory `/download/keepalived-1.1.19/genhash'
gcc -g -O2 -I/usr/src/linux/include -I../lib -Wall -Wunused -Wstrict-prototypes -c -o main.o main.c
gcc -g -O2 -I/usr/src/linux/include -I../lib -Wall -Wunused -Wstrict-prototypes -c -o sock.o sock.c
gcc -g -O2 -I/usr/src/linux/include -I../lib -Wall -Wunused -Wstrict-prototypes -c -o layer4.o layer4.c
gcc -g -O2 -I/usr/src/linux/include -I../lib -Wall -Wunused -Wstrict-prototypes -c -o http.o http.c
gcc -g -O2 -I/usr/src/linux/include -I../lib -Wall -Wunused -Wstrict-prototypes -c -o ssl.o ssl.c
ssl.c: In function ‘init_ssl’:
ssl.c:50: warning: assignment discards qualifiers from pointer target type
Building ../bin/genhash
strip ../bin/genhash
Make complete
make[1]: Leaving directory `/download/keepalived-1.1.19/genhash'
Make complete
[root@ser6-70 keepalived-1.1.19]# make install
make -C keepalived install
make[1]: Entering directory `/download/keepalived-1.1.19/keepalived'
install -d /usr/local/keepalived/sbin
install -m 700 ../bin/keepalived /usr/local/keepalived/sbin/
install -d /usr/local/keepalived/etc/rc.d/init.d
install -m 755 etc/init.d/keepalived.init /usr/local/keepalived/etc/rc.d/init.d/keepalived
install -d /usr/local/keepalived/etc/sysconfig
install -m 755 etc/init.d/keepalived.sysconfig /usr/local/keepalived/etc/sysconfig/keepalived
install -d /usr/local/keepalived/etc/keepalived/samples
install -m 644 etc/keepalived/keepalived.conf /usr/local/keepalived/etc/keepalived/
install -m 644 ../doc/samples/* /usr/local/keepalived/etc/keepalived/samples/
install -d /usr/local/keepalived/share/man/man5
install -d /usr/local/keepalived/share/man/man8
install -m 644 ../doc/man/man5/keepalived.conf.5 /usr/local/keepalived/share/man/man5
install -m 644 ../doc/man/man8/keepalived.8 /usr/local/keepalived/share/man/man8
make[1]: Leaving directory `/download/keepalived-1.1.19/keepalived'
make -C genhash install
make[1]: Entering directory `/download/keepalived-1.1.19/genhash'
install -d /usr/local/keepalived/bin
install -m 755 ../bin/genhash /usr/local/keepalived/bin/
install -d /usr/local/keepalived/share/man/man1
install -m 644 ../doc/man/man1/genhash.1 /usr/local/keepalived/share/man/man1
make[1]: Leaving directory `/download/keepalived-1.1.19/genhash'
[root@ser6-70 keepalived-1.1.19]#
cp /usr/local/keepalived/sbin/keepalived /usr/sbin/
cp /usr/local/keepalived/etc/sysconfig/keepalived /etc/sysconfig/
cp /usr/local/keepalived/etc/rc.d/init.d/keepalived /etc/init.d/
mkdir /etc/keepalived
3.3: Configure keepadlived
3.3.1 lvs-master configuration
vi /etc/keepalived/keepalived.conf
Add content:
global_defs {
notification_email {
baidandan@gooagoo.com #email 通知
}
notification_email_from baidandan@gooagoo.com
smtp_server smtp.mxhichina.com
smtp_connect_timeout 30
router_id LVS1 # 设置lvs的id，在一个网络内应该是唯一的
}
vrrp_sync_group test {
#设置vrrp组
group {
loadbalance
}
}
vrrp_instance loadbalance {
state MASTER #设置lvs的状态，报错MASTER和BACKUP两种，必须大写
interface eth0 #设置对外服务的接口
lvs_sync_daemon_inteface eth0 #设置lvs监听的接口
virtual_router_id 51 #设置虚拟路由表示
priority 180 #设置优先级，数值越大，优先级越高
advert_int 5 #设置同步时间间隔
authentication {
#设置验证类型和密码
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
#设置lvs vip
192.168.6.202
}
}
virtual_server 192.168.6.202 3306 {
delay_loop 6 #健康检查时间间隔
lb_algo rr #负载均衡调度算法
lb_kind DR #负载均衡转发规则
#persistence_timeout 20 #设置会话保持时间，对bbs等很有用
protocol TCP #协议
real_server 192.168.6.66 3306 {
weight 3 #设置权重
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
connect_port 3306
}
}
real_server 192.168.6.67 3306 {
weight 3
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
connect_port 3306
}
}
}
3.3.2 lvs-backup configuration
vi /etc/keepalived/keepalived.conf
Add to:
global_defs {
notification_email {
baidandan@gooagoo.com
}
notification_email_from baidandan@gooagoo.com
smtp_server smtp.mxhichina.com
smtp_connect_timeout 30
router_id LVS2
}
vrrp_sync_group test {
group {
loadbalance
}
}
vrrp_instance loadbalance {
state BACKUP
interface eth0
lvs_sync_daemon_inteface eth0
virtual_router_id 51
priority 150
advert_int 5
authentication {
auth_type PASS
auth_pass 1111
}
virtual_ipaddress {
192.168.6.202
}
}
virtual_server 192.168.6.202 3306 {
delay_loop 6
lb_algo rr
lb_kind DR
#persistence_timeout 20
protocol TCP
real_server 192.168.6.66 3306 {
weight 3
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
connect_port 3306
}
}
real_server 192.168.6.67 3306 {
weight 3
TCP_CHECK {
connect_timeout 3
nb_get_retry 3
delay_before_retry 3
connect_port 3306
}
}
}
3.3.3 Realserver configuration
Configure on all sql nodes:
vi /etc/rc.d/init.d/realserver.sh
Add to:
#!/bin/bash
# description: Config realserver lo and apply noarp
SNS_VIP=192.168.6.201
/etc/rc.d/init.d/functions
case "$1" in
start)
ifconfig lo:0 $SNS_VIP netmask 255.255.255.255 broadcast $SNS_VIP
/sbin/route add -host $SNS_VIP dev lo:0
echo "1" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "1" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "2" >/proc/sys/net/ipv4/conf/all/arp_announce
sysctl -p >/dev/null 2>&1
echo "RealServer Start OK"
;;
stop)
ifconfig lo:0 down
route del $SNS_VIP >/dev/null 2>&1
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/lo/arp_announce
echo "0" >/proc/sys/net/ipv4/conf/all/arp_ignore
echo "0" >/proc/sys/net/ipv4/conf/all/arp_announce
echo "RealServer Stoped"
;;
*)
echo "Usage: $0 {start|stop}"
exit 1
esac
exit 0
Grant execution permissions
chmod +x /etc/rc.d/init.d/functions
chmod +x /etc/rc.d/init.d/realserver.sh
#start up
/etc/rc.d/init.d/realserver.sh start
After startup, use the ifconfig command to see new additions similar to the following:
lo:0 Link encap:Local Loopback
inet addr:192.168.6.202 Mask:255.255.255.255
UP LOOPBACK RUNNING MTU:16436 Metric:1
Note: This must be enabled on the sql node, otherwise the mysql virtual ip cannot be connected later:
ERROR 2003 (HY000): Can't connect to MySQL server on '192.168.6.202' (110)
.3.4: Start keepalived
Start on lvs-master, lvs-backup:
service keepalived start
Starting keepalived: [ OK ]
After starting keepalived, the virtual IP can be pinged.
#View status
View at lvs-master:
[root@ser6-70 keepalived]# ip addr
1：lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2：eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 52:54:00:3d:fd:30 brd ff:ff:ff:ff:ff:ff
inet 192.168.6.70/24 brd 192.168.6.255 scope global eth0
inet 192.168.6.202/32 scope global eth0
inet6 fe80::5054:ff:fe3d:fd30/64 scope link
valid_lft forever preferred_lft forever
You can see the virtual IP 192.168.6.202.
[root@ser6-70 keepalived]# ipvsadm
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.6.202:mysql rr
-> 192.168.6.67:mysql Route 3 2 0
-> 192.168.6.66:mysql Route 3 0 0
– View status on lvs-backup
[root@ser6-71 keepalived]# ip addr
1：lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2：eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 52:54:00:3d:fd:72 brd ff:ff:ff:ff:ff:ff
inet 192.168.6.71/24 brd 192.168.6.255 scope global eth0
inet6 fe80::5054:ff:fe3d:fd72/64 scope link
valid_lft forever preferred_lft forever
I don’t see the virtual IP here, which means this is from lvs.
[root@ser6-71 keepalived]# ipvsadm
IP Virtual Server version 1.2.1 (size=4096)
Prot LocalAddress:Port Scheduler Flags
-> RemoteAddress:Port Forward Weight ActiveConn InActConn
TCP 192.168.6.202:mysql rr
-> 192.168.6.67:mysql Route 3 0 0
-> 192.168.6.66:mysql Route 3 0 0
3.4 Create users who allow remote access
Create a user on the sql node that allows remote access:
grant select,update,delete,insert on *.* to 'dandan' identified by 'dandan';
3.5 Testing
3.5.1 Test lvs load balancing
#Connect to the virtual ip on the client 192.168.6.51
[root@ser6-51 ~]# mysql -u dandan -pdandan -h 192.168.6.202
Warning: Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 3677
Server version: 5.5.35-ndb-7.2.15-cluster-gpl MySQL Cluster Community Server (GPL)
Copyright (c) 2000, 2014, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| dba |
| mysql |
| ndbinfo |
| performance_schema |
| test |
+--------------------+
6 rows in set (0.00 sec)
mysql> use dba;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> show tables;
+---------------+
| Tables_in_dba |
+---------------+
| t2 |
+---------------+
1 row in set (0.01 sec)
mysql> select * from t2;
+------+
| id |
+------+
| 1 |
| 4 |
| 2 |
| 3 |
| 5 |
| 1 |
+------+
6 rows in set (0.01 sec)
mysql> delete from t2 where id = 1;
Query OK, 2 rows affected (0.07 sec)
mysql> select * from t2;
+------+
| id |
+------+
| 3 |
| 5 |
| 2 |
| 4 |
+------+
4 rows in set (0.00 sec)
mysql>
View on lvs-master:
[root@ser6-70 keepalived]# ipvsadm -lcn
IPVS connection entries
pro expire state source virtual destination
TCP 14:29 ESTABLISHED 192.168.6.51:29569 192.168.6.202:3306 192.168.6.66:3306
I saw that 192.168.6.51 was connected to 192.168.6.66 through the virtual IP.
Enter the sql node 192.168.6.66 to see if the data is really deleted.
[root@ser6-66 ~]# mysql -u root -p
Enter password:
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 3731
Server version: 5.5.35-ndb-7.2.15-cluster-gpl MySQL Cluster Community Server (GPL)
Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> use dba;
Reading table information for completion of table and column names
You can turn off this feature to get a quicker startup with -A
Database changed
mysql> select * from t2;
+------+
| id |
+------+
| 3 |
| 5 |
| 2 |
| 4 |
+------+
4 rows in set (0.02 sec)
If it is really deleted, it means that the client successfully connected to the sql node through the virtual IP.
#Log in to vip on client 192.168.6.52
[root@ser6-52 ~]# mysql -u dandan -pdandan -h 192.168.6.202
Warning: Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 3739
Server version: 5.5.35-ndb-7.2.15-cluster-gpl MySQL Cluster Community Server (GPL)
Copyright (c) 2000, 2014, Oracle and/or its affiliates. All rights reserved.
Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql>
Check the status on lvs-master:
[root@ser6-70 keepalived]# ipvsadm -lcn
IPVS connection entries
pro expire state source virtual destination
TCP 12:36 ESTABLISHED 192.168.6.51:29569 192.168.6.202:3306 192.168.6.66:3306
TCP 14:54 ESTABLISHED 192.168.6.52:52348 192.168.6.202:3306 192.168.6.67:3306
It shows that 192.168.6.52 is connected to 192.168.6.67. This shows that lvs has achieved load balancing. The two clients are connected to different sql nodes.
Note: You can only enter ipvsadm -lcn on lvs-master to view, and the view on lvs-backup is empty:
[root@ser6-71 keepalived]# ipvsadm -lcn
IPVS connection entries
pro expire state source virtual destination
3.5.2 Test keepalived high availability
Now turn off keepalived of lvs-master to see if high availability can be achieved.
[root@ser6-70 keepalived]# service keepalived stop
Stopping keepalived: [ OK ]
After a few seconds, check on lvs-backup:
[root@ser6-71 keepalived]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 52:54:00:3d:fd:72 brd ff:ff:ff:ff:ff:ff
inet 192.168.6.71/24 brd 192.168.6.255 scope global eth0
inet 192.168.6.202/32 scope global eth0
inet6 fe80::5054:ff:fe3d:fd72/64 scope link
valid_lft forever preferred_lft forever
See that the virtual IP has drifted to lvs-backup.
[root@ser6-71 keepalived]# ipvsadm -lcn
IPVS connection entries
pro expire state source virtual destination
TCP 14:25 ESTABLISHED 192.168.6.52:54311 192.168.6.202:3306 192.168.6.67:3306
TCP 14:49 ESTABLISHED 192.168.6.51:29604 192.168.6.202:3306 192.168.6.66:3306
The client can also access the VIP node normally, so I won’t go into details here.
After starting keepalived on lvs-master, vip will automatically drift to the master node.
[root@ser6-70 keepalived]# service keepalived start
Starting keepalived: [ OK ]
[root@ser6-70 keepalived]# ip addr
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
inet 127.0.0.1/8 scope host lo
inet6 ::1/128 scope host
valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
link/ether 52:54:00:3d:fd:30 brd ff:ff:ff:ff:ff:ff
inet 192.168.6.70/24 brd 192.168.6.255 scope global eth0
inet 192.168.6.202/32 scope global eth0
inet6 fe80::5054:ff:fe3d:fd30/64 scope link
valid_lft forever preferred_lft forever