from：http://www.tuicool.com/articles/zeQ3ia
引言
Nginx是一个高性能的代理服务器，单台Nginx容易出现单点故障，使用keepalived可以实现Nginx的故障转移，保证了网站的高可用性
一、 使用Nginx+keepalived的两种方案
1、主从模式
使用一个VIP，前端有2台服务器，一主一从，正常情况下是主服务器提供服务只有当主服务器不能正常提供服务之后，从服务器才提供服务，此时总会有一台服务器是空闲状态。
2、双主模式
使用两个VIP，前段有2台服务器，互为主从，两台服务器同时工作，不存在资源浪费情况。同时在前段的DNS服务器对网站做多条A记录，实现了Nginx的负载均衡，当一台服务器故障时候，资源会转移到另一台服务器，继续提供服务，在大型的网站中多数都使用此种架构。在此使用主主模式配置Nginx+keepalived的高可用性。
二、准备实验环境
1、服务器IP地址规划
VIP：172.16.10.8
VIP：172.16.10.9
Keepalived1:172.16.10.1
Keepalived2:172.16.10.2
2、服务器操作系统
Keepalived1 ：Centos 6.4 x86_64
Keepalived2 ：Centos 6.4 x86_64
3、网络拓扑图
4、修改主机名以及hosts文件keepalived1
####keepalived1 server############ sed -i 's@\(HOSTNAME=\).*@\1keepalived1@g'/etc/sysconfig/network hostname keepalived1 [root@keepalived1 ~]# echo "172.16.10.1 keepalived1">> /etc/hosts [root@keepalived1 ~]# echo "172.16.10.2 keepalived2">> /etc/hosts [root@keepalived1 ~]# ssh-keygen -t rs [root@keepalived1 ~]# ssh-copy-id -i .ssh/id_rsa.pub keepalived2 [root@keepalived1 ~]# scp /etc/hosts keepalived1:/etc/ ####keepalived2 server############ sed -i 's@\(HOSTNAME=\).*@\1keepalived2@g'/etc/sysconfig/network hostname keepalived2 [root@keepalived2 ~]# ssh-keygen -t rsa [root@keepalived2 ~]# ssh-copy-id -i .ssh/id_rsa.pub keepalived1
三、编译安装Nginx
[root@keepalived1 ~]# yum install openssl-devel pcre-devel gcc -y [root@keepalived1 ~]# tar xf nginx-1.4.2.tar.gz -C /usr/local/ [root@keepalived1 ~]# cd /usr/local/ [root@keepalived1 local]# groupadd -r nginx [root@keepalived1 local]# useradd -r -g nginx nginx [root@keepalived1 local]# cd nginx-1.4.2/ [root@keepalived1 nginx-1.4.2]# ./conf conf/ configure [root@keepalived1 nginx-1.4.2]# ./configure \ > --prefix=/usr \ > --sbin-path=/usr/sbin/nginx \ > --conf-path=/etc/nginx/nginx.conf \ > --error-log-path=/var/log/nginx/error.log \ > --http-log-path=/var/log/nginx/access.log \ > --pid-path=/var/run/nginx/nginx.pid \ > --lock-path=/var/lock/nginx.lock \ > --user=nginx \ > --group=nginx \ > --with-http_ssl_module \ > --with-http_flv_module \ > --with-http_stub_status_module \ > --with-http_gzip_static_module \ > --http-client-body-temp-path=/var/tmp/nginx/client/ \ > --http-proxy-temp-path=/var/tmp/nginx/proxy/ \ > --http-fastcgi-temp-path=/var/tmp/nginx/fcgi/ \ > --http-uwsgi-temp-path=/var/tmp/nginx/uwsgi \ > --http-scgi-temp-path=/var/tmp/nginx/scgi \ > --with-pcre [root@keepalived1 nginx-1.4.2]# make && make install [root@keepalived1 nginx-1.4.2]# vim /etc/init.d/nginx #!/bin/sh # # nginx - this script starts and stops the nginx daemon # # chkconfig: - 85 15 # description: Nginx is an HTTP(S) server, HTTP(S) reverse \ # proxy and IMAP/POP3 proxy server # processname: nginx # config: /etc/nginx/nginx.conf # config: /etc/sysconfig/nginx # pidfile: /var/run/nginx.pid # Source function library. . /etc/rc.d/init.d/functions # Source networking configuration. . /etc/sysconfig/network # Check that networking is up. [ "$NETWORKING" = "no" ] && exit 0 nginx="/usr/sbin/nginx" prog=$(basename $nginx) NGINX_CONF_FILE="/etc/nginx/nginx.conf" [ -f /etc/sysconfig/nginx ] && . /etc/sysconfig/nginx lockfile=/var/lock/subsys/nginx make_dirs() { # make required directories user=`nginx -V 2>&1 | grep "configure arguments:" | sed 's/[^*]*--user=\([^ ]*\).*/\1/g' -` options=`$nginx -V 2>&1 | grep 'configure arguments:'` for opt in $options; do if [ `echo $opt | grep '.*-temp-path'` ]; then value=`echo $opt | cut -d "=" -f 2` if [ ! -d "$value" ]; then # echo "creating" $value mkdir -p $value && chown -R $user $value fi fi done } start() { [ -x $nginx ] || exit 5 [ -f $NGINX_CONF_FILE ] || exit 6 make_dirs echo -n $"Starting $prog: " daemon $nginx -c $NGINX_CONF_FILE retval=$? echo [ $retval -eq 0 ] && touch $lockfile return $retval } stop() { echo -n $"Stopping $prog: " killproc $prog -QUIT retval=$? echo [ $retval -eq 0 ] && rm -f $lockfile return $retval } restart() { configtest || return $? stop sleep 1 start } reload() { configtest || return $? echo -n $"Reloading $prog: " killproc $nginx -HUP RETVAL=$? echo } force_reload() { restart } configtest() { $nginx -t -c $NGINX_CONF_FILE } rh_status() { status $prog } rh_status_q() { rh_status >/dev/null 2>&1 } case "$1" in start) rh_status_q && exit 0 $1 ;; stop) rh_status_q || exit 0 $1 ;; restart|configtest) $1 ;; reload) rh_status_q || exit 7 $1 ;; force-reload) force_reload ;; status) rh_status ;; condrestart|try-restart) rh_status_q || exit 0 ;; *) echo $"Usage: $0 {start|stop|status|restart|condrestart|try-restart|reload|force-reload|configtest}" exit 2 esac [root@keepalived1 nginx-1.4.2]# chmod +x /etc/init.d/nginx [root@keepalived1 nginx-1.4.2]# service nginx start
注意在此只上传了keepalived1的代码，keepalived2也需要同样的操作
1、修改默认网页以方便后期测试
###############keepalived1###################### [root@keepalived1 ~]# echo "<h1>keepalived1</h1>" > /usr/html/index.html ###############keepalived2###################### [root@keepalived2 ~]# echo "<h1>keepalived2</h1>" > /usr/html/index.html
四、 安装与配置keepalived
1、安装keepalived
###############keepalived1###################### [root@keepalived1 ~]# yum install keepalived -y ###############keepalived2###################### [root@keepalived2 ~]# yum install keepalived -y
2、修改配置文件
[root@keepalived1 keepalived]# grep -v "#" /etc/keepalived/keepalived.conf ! Configuration File for keepalived global_defs { notification_email { root@localhost } notification_email_from Alexandre.Cassen@localhost smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id LVS_DEVEL } vrrp_script chk_nginx { #监控nginx脚本 script "killall -0 nginx" #监控nginx进程 interval 1 #监控间隔 weight -2 #优先级-2 } vrrp_instance VI_1 { state MASTER #主server interface eth0 virtual_router_id 80 priority 100 #优先级 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 172.16.10.8 #定义vip } track_script { chk_nginx #跟踪脚本 } notify_master "/etc/keepalived/notify8.sh master" #定义邮件通知 notify_backup "/etc/keepalived/notify8.sh backup" notify_fault "/etc/keepalived/notify8.sh fault" } vrrp_instance VI_2 { state BACKUP #从server interface eth0 virtual_router_id 81 priority 99 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 172.16.10.9 } track_script { chk_nginx } notify_master "/etc/keepalived/notify9.sh master" #定义邮件通知 notify_backup "/etc/keepalived/notify9.sh backup" notify_fault "/etc/keepalived/notify9.sh fault" } [root@keepalived1 keepalived]#
3、编辑邮件通知脚本（notify8.sh notify9.sh）
#####################notify8.sh############## [root@keepalived1 keepalived]# cat notify8.sh #!/bin/bash # Author: xiaodong <[email protected]> # description: An example of notify script # vip=172.16.10.8 contact='root@localhost' notify() { mailsubject="`hostname` to be $1: $vip floating" mailbody="`date '+%F %H:%M:%S'`: vrrp transition, `hostname` changed to be $1" echo $mailbody | mail -s "$mailsubject" $contact } case "$1" in master) notify master /etc/rc.d/init.d/nginx start exit 0 ;; backup) notify backup /etc/rc.d/init.d/nginx stop exit 0 ;; fault) notify fault exit 0 ;; *) echo 'Usage: `basename $0` {master|backup|fault}' exit 1 ;; esac ####################notfiy9.sh################# [root@keepalived1 keepalived]# cat notify9.sh #!/bin/bash # Author: xiaodong <[email protected]> # description: An example of notify script # vip=172.16.10.9 contact='root@localhost' notify() { mailsubject="`hostname` to be $1: $vip floating" mailbody="`date '+%F %H:%M:%S'`: vrrp transition, `hostname` changed to be $1" echo $mailbody | mail -s "$mailsubject" $contact } case "$1" in master) notify master exit 0 ;; backup) notify backup exit 0 ;; fault) notify fault exit 0 ;; *) echo 'Usage: `basename $0` {master|backup|fault}' exit 1 ;; esac [root@keepalived1 keepalived]# chmod +x notify8.sh [root@keepalived1 keepalived]# chmod +x notify9.sh
4、复制配置文件到keepalived2，并做修改.
[root@keepalived1 keepalived]# scp -p keepalived.conf notify8.sh notify9.sh keepalived2:/etc/keepalived/ [root@keepalived2 keepalived]# grep -v "#" /etc/keepalived/keepalived.conf ! Configuration File for keepalived global_defs { notification_email { root@localhost notification_email_from Alexandre.Cassen@localhost smtp_server 127.0.0.1 smtp_connect_timeout 30 router_id LVS_DEVEL } vrrp_script chk_nginx { script "killall -0 nginx " interval 1 weight -2 } vrrp_instance VI_1 { state BACKUP #改为backup interface eth0 virtual_router_id 80 priority 99 #改为99 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 172.16.10.8 } track_script { chk_nginx } notify_master "/etc/keepalived/notify.sh master" notify_backup "/etc/keepalived/notify.sh backup" notify_fault "/etc/keepalived/notify.sh fault" } vrrp_instance VI_2 { state MASTER #改为MASTER interface eth0 virtual_router_id 81 priority 100 #改为100 advert_int 1 authentication { auth_type PASS auth_pass 1111 } virtual_ipaddress { 172.16.10.9 } track_script { chk_nginx } notify_master "/etc/keepalived/notify9.sh master" notify_backup "/etc/keepalived/notify9.sh backup" notify_fault "/etc/keepalived/notify9.sh fault" }
注释 ：此处使用本地的邮件服务器接受邮件，如果需要用其它邮件服务器请修改contact='root@localhost'
5、启动keepalived服务
###############keepalived1###################### [root@keepalived1 ~]# service keepalived start ###############keepalived2###################### [root@keepalived2 ~]# service keepalived start
6、查看两个节点的vip是否启动正常
五、测试nginx+keepalived的高可用性
1、使用游览器访问测试
2、模拟节点出现故障，nginx服务器是否能自动转移
[root@keepalived1 keepalived]# service keepalived stop
通过以上测试，节点出现故障的时候，服务可以自动转移到备用节点上
3、测试主节点服务down掉之后，备用节点服务是否能正常运行
[root@keepalived1 keepalived]# service keepalived start [root@keepalived1 keepalived]# killall nginx
通过以上测试，实现了Nginx的高可用性，但是，运维人员是否能第一时间得知服务器出现故障，这时候就需要查看邮件了
4、查看邮件是否收到信息
[root@keepalived1 keepalived]# mail #查看邮件命令 Heirloom Mail version 12.4 7/29/08. Type ? for help. "/var/spool/mail/root": 1 message 1 new >N 1 root Wed Sep 25 20:15 18/728 "keepalived1 to be backup: 172.16.10.8 floating" & 1 Message 1: From root@keepalived1.localdomain Wed Sep 25 20:15:46 2013 Return-Path: <root@keepalived1.localdomain> X-Original-To: root@localhost Delivered-To: root@localhost.localdomain Date: Wed, 25 Sep 2013 20:15:46 +0800 To: root@localhost.localdomain Subject: keepalived1 to be backup: 172.16.10.8 floating User-Agent: Heirloom mailx 12.4 7/29/08 Content-Type: text/plain; charset=us-ascii From: root@keepalived1.localdomain (root) Status: R 2013-09-25 20:15:46: vrrp transition, keepalived1 changed to be backup & quit #退出邮件
5、当nginx服务启动之后，主节点恢复
[root@keepalived1 keepalived]# service nginx start [root@keepalived1 keepalived]# mail Heirloom Mail version 12.4 7/29/08. Type ? for help. "/var/spool/mail/root": 2 messages 1 unread 1 root Wed Sep 25 20:15 19/739 "keepalived1 to be backup: 172.16.10.8 floating" >U 2 root Wed Sep 25 20:16 19/738 "keepalived1 to be master: 172.16.10.8 floating" & Message 2: From [email protected] Wed Sep 25 20:16:22 2013 Return-Path: <[email protected]> X-Original-To: root@localhost Delivered-To: [email protected] Date: Wed, 25 Sep 2013 20:16:22 +0800 To: [email protected] Subject: keepalived1 to be master: 172.16.10.8 floating User-Agent: Heirloom mailx 12.4 7/29/08 Content-Type: text/plain; charset=us-ascii From: [email protected] (root) Status: RO 2013-09-25 20:16:22: vrrp transition, keepalived1 changed to be master
Nginx+keepalived的高可用负载均衡配置完成。
本博客至此结束，如有不足之处，望大家多提宝贵意见！！！！
+
+
+
-
-
-