rhel6.5 -----varnish

Varnish是一款高性能的开源HTTP加速器

2.1 总体流程
主进程 fork 子进程，主进程等待子进程的信号，子进程退出后，主进程重新启动子进程
子进程生成若干线程。
Accept 线程：接受请求，将请求挂在 overflow队列上
Work 线程： 多个，从对列上摘除请求，对请求进行处理，直到完成，然后处理下一个
请求
Epoll 线程： 一个请求处理称作一个 session，在 session 周期内，处理完请求后，会交给
Epoll 处理，监听是否还有事件发生。
Expire 线程：对于缓存的对象，根据过期时间，组织成二叉堆，该线程周期检查该堆的
根，处理过期的文件。
线程之间的关系：
2.1.1 accept 线程
监听端口，接受连接。
接受后组织成 struct ses（session 结构） ，看是否有空闲的工作线程，如果有，将请求给它，
pthread_cond_signal 信号通知它没有空闲线程，如果 overflow过大，则放弃该请求。否则，
将其挂在 overflow 上（需要更多工作线程，发通知）。
继续监听 2.1.2 work 线程
从 overflow队列上摘取请求（struct ses），进入状态机处理，处理结束后，通过 pipe通信，
将 struct ses发送给 epoll 线程。
2.1.3 Epoll 线程,得到传过来的 struct ses，若还没有过期，将 socket 放入 epoll 的事件中，事
件发生时，也会将其放入到 overflow中进行

处理过程大致分为如下几个步骤:
(1)Receive 状态,也就是请求处理的入口状态,根据 VCL 规则判断该请求应该是 Pass 或
Pipe,或者进入 Lookup(本地查询)。
(2)Lookup 状态,进入此状态后,会在 hash 表中查找数据,若找到,则进入 Hit 状态,否则进
入 miss 状态。
(3)Pass 状态,在此状态下,会进入后端请求,即进入 fetch 状态。
(4)Fetch 状态,在 Fetch 状态下,对请求进行后端的获取,发送请求,获得数据,并进行本地
的存储。
(5)Deliver 状态, 将获取到的数据发送给客户端,然后完成本次请求。

首先给虚拟机server1安装varnish

[root@server1 ~]# cd /mnt/
[root@server1 mnt]# ls
varnish-3.0.5-1.el6.x86_64.rpm  varnish-libs-3.0.5-1.el6.x86_64.rpm
[root@server1 mnt]# yum install varnish-3.0.5-1.el6.x86_64.rpm varnish-libs-3.0.5-1.el6.x86_64.rpm -y

将端口改为80

[root@server1 mnt]# cd /etc/varnish/
[root@server1 varnish]# ls
default.vcl  secret
[root@server1 varnish]# ll default.vcl 
-rw-r--r-- 1 root root 3119 Dec  2  2013 default.vcl
[root@server1 varnish]# ll secret 
-rw------- 1 root root 37 Jul 28 09:37 secret
[root@server1 varnish]# vim /etc/sysconfig/varnish 

[root@server1 varnish]# vim /etc/security/limits.conf 
# End of file
#
varnish         -       nofile          131073
varnish         -       memlock         82000
varnish         -       nproc           unlimited

打开服务并查看端口

[root@server1 varnish]# /etc/init.d/varnish start
Starting Varnish Cache:                                    [  OK  ]
[root@server1 varnish]# netstat -antlp
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address               Foreign Address             State       PID/Program name   
tcp        0      0 0.0.0.0:80                  0.0.0.0:*                   LISTEN      1858/varnishd       
tcp        0      0 0.0.0.0:22                  0.0.0.0:*                   LISTEN      909/sshd            
tcp        0      0 127.0.0.1:25                0.0.0.0:*                   LISTEN      985/master          
tcp        0      0 127.0.0.1:6082              0.0.0.0:*                   LISTEN      1857/varnishd       
tcp        0      0 172.25.53.1:22              172.25.53.250:41364         ESTABLISHED 1380/sshd           
tcp        0      0 :::80                       :::*                        LISTEN      1858/varnishd       
tcp        0      0 :::22                       :::*                        LISTEN      909/sshd            
tcp        0      0 ::1:25                      :::*                        LISTEN      985/master  

[root@server1 varnish]# vim default.vcl 
添加
backend web1 {
  .host = "172.25.53.2";
  .port = "80";
}

安装第二台虚拟机server2，安装httpd，并打开服务

[root@server2 ~]# yum install httpd -y
[root@server2 ~]# /etc/init.d/httpd start
Starting httpd: httpd: Could not reliably determine the server's fully qualified domain name, using 172.25.53.2 for ServerName
                                                           [  OK  ]

在server2的默认发布目录下添加如下

[root@server2 ~]# cd /var/www/html/
[root@server2 html]# ls
[root@server2 html]# vim index.html
[root@server2 html]# cat index.html 
www.westos.org

更改server1的配置文件，添加为server2的ip，并重启varnish

[root@server1 varnish]# vim default.vcl 
[root@server1 varnish]# /etc/init.d/varnish reload
Loading vcl from /etc/varnish/default.vcl
Current running config name is reload_2018-07-28T11:53:01
Using new config name reload_2018-07-28T13:00:42
VCL compiled.

available       0 boot
available       2 reload_2018-07-28T11:53:01
active          0 reload_2018-07-28T13:00:42

Done

这时候就可通过客户端浏览器用server1 varnish服务器的ip访问server2的发布目录下的文件

我们在主机解析server1 的域名

然后就可以通过ping域名或者浏览器输入域名访问发布目录了

缓存是会过期的，我们现在做的实验是每隔120s缓存更新一次

查看缓存命中情况

###查看缓存命中情况
sub vcl_deliver {
if (obj.hits > 0) {
set resp.http.X-Cache = "HIT from westos cache";
}
else {
set resp.http.X-Cache = "MISS from westos cache";
}
return (deliver);
}
# service varnish reload

[root@server1 varnish]# vim default.vcl 
# Default backend definition.  Set this to point to your content
# server.
# 
backend web1 {
  .host = "172.25.53.2";
  .port = "80";
}


在原基础上添加
sub vcl_deliver {
if (obj.hits > 0) {
set resp.http.X-Cache = "HIT from westos cache";
}
else {
set resp.http.X-Cache = "MISS from westos cache";
}
return (deliver);
}

[root@server1 varnish]# /etc/init.d/varnish reload
Loading vcl from /etc/varnish/default.vcl
Current running config name is reload_2018-07-28T11:53:01
Using new config name reload_2018-07-28T13:00:42
VCL compiled.

available       0 boot
available       2 reload_2018-07-28T11:53:01
active          0 reload_2018-07-28T13:00:42

Done

测试

[root@foundation53 images]# curl -I www.westos.org
HTTP/1.1 200 OK
Server: Apache/2.2.15 (Red Hat)
Last-Modified: Sat, 28 Jul 2018 03:51:28 GMT
ETag: "40029-f-57207249c68a7"
Content-Type: text/html; charset=UTF-8
Content-Length: 15
Accept-Ranges: bytes
Date: Sat, 28 Jul 2018 05:00:51 GMT
X-Varnish: 612366941
Age: 0
Via: 1.1 varnish
Connection: keep-alive
X-Cache: MISS from westos cache            未命中

[root@foundation53 images]# curl -I www.westos.org
HTTP/1.1 200 OK
Server: Apache/2.2.15 (Red Hat)
Last-Modified: Sat, 28 Jul 2018 03:51:28 GMT
ETag: "40029-f-57207249c68a7"
Content-Type: text/html; charset=UTF-8
Content-Length: 15
Accept-Ranges: bytes
Date: Sat, 28 Jul 2018 05:01:09 GMT
X-Varnish: 612366942 612366941
Age: 18
Via: 1.1 varnish
Connection: keep-alive
X-Cache: HIT from westos cache               命中

### 通过 varnishadm 手动清除缓存
# varnishadm ban.url .*$
#清除所有
# varnishadm ban.url /index.html
#清除 index.html 页面缓存
# varnishadm ban.url /admin/$
#清除 admin 目录缓存

[root@foundation53 images]# curl -I www.westos.org/index.html
HTTP/1.1 200 OK
Server: Apache/2.2.15 (Red Hat)
Last-Modified: Sat, 28 Jul 2018 03:51:28 GMT
ETag: "40029-f-57207249c68a7"
Content-Type: text/html; charset=UTF-8
Content-Length: 15
Accept-Ranges: bytes
Date: Sat, 28 Jul 2018 05:05:05 GMT
X-Varnish: 612366950
Age: 0
Via: 1.1 varnish
Connection: keep-alive
X-Cache: MISS from westos cache

###定义多个不同域名站点的后端服务器
backend web1 {
.host = "192.168.0.188";
.port = "80";
}
backend web2 {
.host = "192.168.0.189";
.port = "80";
}
#当访问 www.westos.org 域名时从 web1 上取数据,访问 bbs.westos.org 域名时到 web2 取数据,
访问其他页面报错。
sub vcl_recv {
if (req.http.host ~ "^(www.)?westos.org") {
set req.http.host = "www.westos.org";
set req.backend = web1;
} elsif (req.http.host ~ "^bbs.westos.org") {
set req.backend = web2;
} else {error 404 "westos cache";
}
}
# service varnish reload

创建server3 并安装httpd并打开

[root@server3 ~]# yum install httpd -y
[root@server3 ~]# /etc/init.d/httpd start
Starting httpd: httpd: Could not reliably determine the server's fully qualified domain name, using 172.25.53.3 for ServerName
                                                           [  OK  ]

在server3默认发布目录下发布如下

[root@server3 html]# cd /var/www/html/
[root@server3 html]# ls
index.html
[root@server3 html]# vim index.html 
[root@server3 html]# cat index.html 
bbs.westos.org - server3

在server1添加server3的ip

[root@server1 varnish]# vim default.vcl 

backend web1 {
  .host = "172.25.53.2";
  .port = "80";
}

backend web2 {
  .host = "172.25.53.3";
  .port = "80";
}
sub vcl_recv {
if (req.http.host ~ "^(www.)?westos.org") {
set req.http.host = "www.westos.org";
set req.backend = westos;
return (pass);
} elsif (req.http.host ~ "^bbs.westos.org") {
set req.backend = web2;
} else {error 404 "westos cache";
}
}

[root@foundation53 images]# ping www.westos.org
PING www.westos.org (172.25.53.1) 56(84) bytes of data.
64 bytes from www.westos.org (172.25.53.1): icmp_seq=1 ttl=64 time=0.298 ms
64 bytes from www.westos.org (172.25.53.1): icmp_seq=2 ttl=64 time=0.482 ms
64 bytes from www.westos.org (172.25.53.1): icmp_seq=3 ttl=64 time=0.337 ms
^C
--- www.westos.org ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 2000ms
rtt min/avg/max/mdev = 0.298/0.372/0.482/0.080 ms
[root@foundation53 images]# ping westos.org 
PING www.westos.org (172.25.53.1) 56(84) bytes of data.
64 bytes from www.westos.org (172.25.53.1): icmp_seq=1 ttl=64 time=0.297 ms
64 bytes from www.westos.org (172.25.53.1): icmp_seq=2 ttl=64 time=0.413 ms
64 bytes from www.westos.org (172.25.53.1): icmp_seq=3 ttl=64 time=0.314 ms
^C
--- www.westos.org ping statistics ---
3 packets transmitted, 3 received, 0% packet loss, time 1999ms
rtt min/avg/max/mdev = 0.297/0.341/0.413/0.053 ms
[root@foundation53 images]# ping bbs.westos.org
PING www.westos.org (172.25.53.1) 56(84) bytes of data.
64 bytes from www.westos.org (172.25.53.1): icmp_seq=1 ttl=64 time=0.296 ms
64 bytes from www.westos.org (172.25.53.1): icmp_seq=2 ttl=64 time=0.269 ms
^C
--- www.westos.org ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 999ms
rtt min/avg/max/mdev = 0.269/0.282/0.296/0.021 ms

#定义健康检查
probe healthcheck {
.url = "/index.html"; # 哪个 url 需要 varnish 请求
.interval = 5s; #检查的间隔时间
.timeout = 1s; #等待多长时间探针超时
.window = 5; #维持 5 个 sliding window 的结果
.threshold = 3; #至少有三次 window 是成功的,就宣告 bachend 健康
}

将server3发布配置文件第990行打开，并添加如下

[root@server3 html]# vim /etc/httpd/conf/httpd.conf 

 990 NameVirtualHost *:80
 991 #
 992 # NOTE: NameVirtualHost cannot be used without a port specifier 
 993 # (e.g. :80) if mod_ssl is being used, due to the nature of the
 994 # SSL protocol.
 995 #
 996 
 997 #
 998 # VirtualHost example:
 999 # Almost any Apache directive may go into a VirtualHost container.
1000 # The first VirtualHost section is used for requests without a known
1001 # server name.
1002 #
1003 <VirtualHost *:80>
1004     DocumentRoot /var/www/html
1005     ServerName bbs.westos.org
1006 </VirtualHost>
1007 <VirtualHost *:80>
1008     DocumentRoot /www1
1009     ServerName www.westos.org
1010 </VirtualHost>

[root@server3 html]# ls
index.html
[root@server3 html]# cat index.html 
bbs.westos.org - server3
[root@server3 html]# cat /www1/index.html 
www.westos.org
[root@server3 html]# 
[root@server3 html]# /etc/init.d/httpd restart
Stopping httpd:                                            [  OK  ]
Starting httpd: httpd: Could not reliably determine the server's fully qualified domain name, using 172.25.53.3 for ServerName
                                                           [  OK  ]

并添加域名解析

[root@server3 html]# curl bbs.westos.org
bbs.westos.org - server3
[root@server3 html]# curl www.westos.org
www.westos.org

server1添加如下

[root@server1 varnish]# /etc/init.d/varnish reload
Loading vcl from /etc/varnish/default.vcl
Current running config name is reload_2018-07-28T16:27:28
Using new config name reload_2018-07-28T16:29:45
VCL compiled.

available       0 boot
available       0 reload_2018-07-28T16:27:28
active          0 reload_2018-07-28T16:29:45

Done

[root@foundation53 ~]# curl www.westos.org
www.westos.org -server3
[root@foundation53 ~]# curl www.westos.org
www.westos.org -server2
[root@foundation53 ~]# curl www.westos.org
www.westos.org -server3
[root@foundation53 ~]# curl www.westos.org
www.westos.org -server2