(1) .Varnish Overview
Varnish is a high performance open-source HTTP accelerator, can effectively reduce the load on web servers to improve access speed. According to the official statement, Varnish cache is a type of HTTP reverse proxy.
Varnish author Poul-Henning Kamp is one of the FreeBSD kernel developer, he believes that the current computer compared to 1975, has many complex. In 1975, only two kinds of storage media: memory and hard drive. But in addition to the memory of the computer system main memory, but also includes within CPU Ll, L2 of, even L3 cache. The device also has its own cache on your hard disk, so squid cache object handle their own alternative architectures that can not optimize these circumstances do, but the operating system can learn these cases, so this part of the work should be handed operation processing system, which is the Varnish cache design architecture.
When the Varnish deployment, processing web requests will be some changes. The client's request will first be Varnish accepted. Varnish will analyze the received request, and forwards it to the back-end web server. Backend web server request process routine, and returns the processing result to sequentially Varnish.
But Varnish function is not limited to this. Varnish core functionality is the result of the back-end web server returns cached, if found to follow the same request, Varnish will not forward the request to the web server, but returns the result cache . This will effectively reduce the load of the web server, to enhance the response speed, and may respond more requests per second . Another major reason Varnish fast is its cache memory are all in , which is much faster than on disk. So that the corresponding optimization measures like speed Varnish beyond imagination. But taking into account the actual system memory is usually limited, so it is necessary to manually configure the cache space limit, while avoiding duplicate content caching.
Order processing cache: Request received - Request analysis (analysis of your URL, your first analysis unit) - calculate the hash - Find Cache - freshness detection --- --- cached access to the source - the establishment response message - and log response.
Listening port 6081 , the management process management, child processes Child / Cache , the official website https://www.varnish-cache.org/ .
Comparative (2) .Varnish characteristics and the Squid
Varnish Features:
Memory-based cache, the data will disappear after the restart.
The use of virtual memory mode, good I / O performance.
Support cache set a precise time within 0 to 60 seconds.
VCL (full name varnish config language, which is Varnish own field-specific language) configuration management more flexible.
On 32-bit machines cache file size is the largest 2G.
It has strong management capabilities, such as top, stat, admin, list and so on.
State machine design clever, clear structure.
Use binary heap management cache files, delete achieve positive purpose.
Varnish and Squid comparison:
Same point:
It is a reverse proxy server;
both open-source software;
Varnish compared with Squid advantage of:
Varnish high stability, both at the chance to complete the work load of the same, Squid server failure than Varnish, since the use of Squid should always restart;
Varnish access faster, Varnish adopted the "Visual Page Cache" technology All data can be read directly from the cache memory, while Squid is read from the hard disk, so Varnish faster access speed;
Varnish can support more concurrent connections, because Varnish TCP connection release than Squid fast, so in a highly concurrent connections can support more TCP connections;
Varnish through management port, use regular expressions to clear some cache volume, and Squid can not be done;
Squid belong to a single process using a single-core CPU, but Varnish is open multi-process handling by fork form to do, so it is reasonable to use all cores to handle the corresponding request;
Varnish compared with Squid's shortcomings:
Varnish in a high state concurrent CPU, I / O and memory resources are higher than the overhead Squid;
Varnish Once the process Hang (pending), Crash (crash) or restarted, the cache data will be completely released from memory, then all requests are sent to the back-end server, under high concurrency, it will cause great stress backend server;
in varnish Varnish use different servers each request if the request url by a single HA / F5 (load balancing), is varnish request to the backend server will be penetrated, and the same request will be cached on multiple servers, can also cause waste of resources caching varnish, can also cause performance degradation.
(3) Install Varnish
1) Installation Environment
Source Package Installation youxi1 192.168.1.6
youxi2 192.168.1.7 yum install example, Web backend
youxi3 192.168.1.8 Web backend
2) Installation
youxi1 the source installation varnish6.2.0 (the recommended installation)
//安装依赖包 [root@youxi1 ~]# yum -y install make autoconf automake libedit-devel libtool ncurses-devel pcre-devel pkgconfig python3-docutils python3-sphinx graphviz [root@youxi1 ~]# tar xf varnish-6.2.0.tgz -C /usr/local/src/ [root@youxi1 ~]# cd /usr/local/src/varnish-6.2.0/ [root@youxi1 varnish-6.2.0]# ./configure --prefix=/usr/local/varnish [root@youxi1 varnish-6.2.0]# make && make install [root@youxi1 varnish-6.2.0]# echo $? 0 [root@youxi1 varnish-6.2.0]# cd /usr/local/varnish/ [root@youxi1 varnish]# mkdir etc [root@youxi1 varnish]# cp share/doc/varnish/example.vcl etc/default.vcl //生成vcl配置文件
yum install varnish (simple to understand) on youxi2
[root@youxi2 ~]# vim /etc/yum.repos.d/varnishcache_varnish62.repo [varnishcache_varnish62] name=varnishcache_varnish62 baseurl=https://packagecloud.io/varnishcache/varnish62/el/7/$basearch repo_gpgcheck=1 gpgcheck=0 enabled=1 gpgkey=https://packagecloud.io/varnishcache/varnish62/gpgkey sslverify=1 sslcacert=/etc/pki/tls/certs/ca-bundle.crt metadata_expire=300 [varnishcache_varnish62-source] name=varnishcache_varnish62-source baseurl=https://packagecloud.io/varnishcache/varnish62/el/7/SRPMS repo_gpgcheck=1 gpgcheck=0 enabled=1 gpgkey=https://packagecloud.io/varnishcache/varnish62/gpgkey sslverify=1 = sslcacert / etc / PKI / TLS / certs / CA-bundle.crt metadata_expire = 300 [youxi2 the root @ ~] # yum Clean All && // Clear List yum yum cache and regenerate [root @ youxi2 ~] # yum - y install varnish
3) sites on the Varnish cache youxi2 on configuration youxi1
youxi1 modify configuration files vcl
[root @ youxi1 ~] # vim /usr/local/varnish/etc/default.vcl
backend default {// first row 19 ~ 16
.host = "192.168.1.7"; // change the IP address of the Web site's back-end
. port = "80"; // Review Web site port number of the backend
}
Sub vcl_deliver {// start line 35, a cache hit where
IF (obj.hits> 0) {
SET = the cache-resp.http.X "the hIT Cache ";
}
the else {
SET resp.http.X the Cache-=" Cache Miss ";
}
}
Configuration environment variable
[youxi1 the root @ ~] # Vim /etc/profile.d/varnish.sh Export the PATH = / usr / local / Varnish / bin: / usr / local / Varnish / sbin: $ the PATH [youxi1 the root @ ~] # /. etc / profile.d / varnish.sh // load environment variables
Start Varnish
[root@youxi1 ~]# varnishd -a 192.168.1.6:80,HTTP -f /usr/local/varnish/etc/default.vcl Debug: Version: varnish-6.2.0 revision b14a3d38dbe918ad50d3838b11aa596f42179b54 Debug: Platform: Linux,3.10.0-957.el7.x86_64,x86_64,-jnone,-sdefault,-sdefault,-hcritbit Debug: Child (18374) Started [root@youxi1 ~]# ps aux | grep varnishd root 18364 0.0 0.0 22188 1532 ? SLs 22:59 0:00 varnishd -a 192.168.1.6:80,HTTP -f /usr/local/varnish/etc/default.vcl root 18374 1.8 4.4 1029912 89468 ? SLl 22:59 0:00 varnishd -a 192.168.1.6:80,HTTP -f /usr/local/varnish/etc/default.vcl root 18593 0.0 0.0 112724 992 pts/0 S+ 23:00 0:00 grep --color=auto varnishd [root@youxi1 ~]# firewall-cmd --permanent --zone=public --add-port=80/tcp && firewall-cmd --reload success success
Up test on the back-end Web youxi2
[root@youxi2 ~]# yum -y install httpd [root@youxi2 ~]# echo youxi2 > /var/www/html/index.html [root@youxi2 ~]# systemctl start httpd [root@youxi2 ~]# firewall-cmd --permanent --zone=public --add-port=80/tcp && firewall-cmd --reload success success
The last test
Then use the curl command to make a cache hit test, -I option to take only the information http response headers, do not take web content
[root@youxi1 ~]# curl -I 192.168.1.7 //这是直接访问youxi2 HTTP/1.1 200 OK Date: Sun, 04 Aug 2019 15:14:16 GMT Server: Apache/2.4.6 (CentOS) Last-Modified: Sun, 04 Aug 2019 14:56:47 GMT ETag: "7-58f4bccfca680" Accept-Ranges: bytes Content-Length: 7 Content-Type: text/html; charset=UTF-8 [root@youxi1 ~]# curl -I 192.168.1.6 //第一次访问youxi1 HTTP/1.1 200 OK Date: Sun, 04 Aug 2019 15:14:19 GMT Server: Apache/2.4.6 (CentOS) Last-Modified: Sun, 04 Aug 2019 14:56:47 GMT ETag: "7-58f4bccfca680" Content-Length: 7 Content-Type: text/html; charset=UTF-8 X-Varnish: 12 Age: 0 Via: 1.1 varnish (Varnish/6.2) X-Cache: Miss cache // This is not hit the Accept-Ranges: bytes Connection: the Keep-Alive [root @ youxi1 ~] # curl the -I 192.168.1.6 // second visit youxi1 HTTP / 1.1 200 the OK a Date: Sun, 04 Aug 2019 15:16:39 GMT Server: the Apache / 2.4.6 (CentOS) Last-Modified: Sun, 04 Aug 2019 14:56:47 GMT the ETag: "7-58f4bccfca680" Content-the Length: 7 Content- type: text / HTML; charset = UTF. 8- X-Varnish-: 15 32773 Age: 2 Via: 1.1 Varnish (Varnish / 6.2) X--the cache: cache // this time the hIT a cache hit the Accept-Ranges: bytes Connection: Keep -alive
Cache short time, you can try to configure httpd long link function (profile setting KeepAlive On, and then restart )
4) Configuration Varnish cache multiple sites (youxi1, youxi2 on youxi1)
youxi1 modify configuration files vcl
[youxi1 the root @ ~] # Vim /usr/local/varnish/etc/default.vcl
backend youxi2 {// hostname to the original default
.host = "192.168.1.7";
.port = "80";
}
backend youxi3 {// create a multi-
.host = "192.168.1.8";
.port = "80";
}
Sub vcl_recv {// add vcl_recv in
if (req.http.host ~ "^ (www ) you.?. cn ") {// regular matching
SET req.http.host =" www.you.cn ";
SET req.backend_hint = youxi2; // point youxi2 rear
} elsif (req.http.host ~" ^ bbs.you .cn ") {// regular matching
set req.backend_hint = youxi3; // point youxi3 rear
}
}
Restart Varnish, you need to use the killall command to install the package psmisc
[root@youxi1 ~]# yum -y install psmisc [root@youxi1 ~]# killall varnishd [root@youxi1 ~]# varnishd -a 192.168.1.6:80,HTTP -f /usr/local/varnish/etc/default.vcl Debug: Version: varnish-6.2.0 revision b14a3d38dbe918ad50d3838b11aa596f42179b54 Debug: Platform: Linux,3.10.0-957.el7.x86_64,x86_64,-jnone,-sdefault,-sdefault,-hcritbit Debug: Child (19017) Started
Up test on the back-end Web youxi3
[root@youxi3 ~]# yum -y install httpd [root@youxi3 ~]# echo youxi3 > /var/www/html/index.html [root@youxi3 ~]# systemctl start httpd [root@youxi3 ~]# firewall-cmd --permanent --zone=public --add-port=80/tcp && firewall-cmd --reload success success
youxi1 the Edit / etc / hosts file
[root@youxi1 ~]# vim /etc/hosts 192.168.1.6 www.you.cn 192.168.1.6 bbs.you.cn
test
[root @ youxi1 ~] # curl www.you.cn // first visit, you can see a point is youxi2 youxi2 [root @ youxi1 ~] # curl // second visit to the -I www.you.cn , just take http response header, a cache hit can be seen that the hTTP / 1.1 the OK 200 is a Date: the Sun, 04-Aug 2019 16:09:19 GMT Server: the Apache / 2.4.6 (the CentOS) Last-Modified: the Sun, 04-Aug 2019 GMT 14:56:47 the ETag: "7-58f4bccfca680" the Content-the Length:. 7 the Content-the Type: text / HTML; charset = UTF. 8- X-Varnish-: 32772. 5 Age: 12 is Via: 1.1 Varnish (Varnish / 6.2) X- -Cache: hIT cache // cache hit the Accept-Ranges: bytes Connection: the Keep-Alive [root @ youxi1 ~] # curl bbs.you.cn // first visit, you can see the point that youxi3 youxi3 [root @ youxi1 ~] # curl -I bbs.you.cn // second visit, just take http response header, you can see the cache hit HTTP/1.1 200 OK Date: Sun, 04 Aug 2019 16:09:49 GMT Server: Apache/2.4.6 (CentOS) Last-Modified: Sun, 04 Aug 2019 16:07:43 GMT ETag: "7-58f4ccaa0e583" Content-Length: 7 Content-Type: text/html; charset=UTF-8 X-Varnish: 32774 8 Age: 6 Via: 1.1 varnish (Varnish/6.2) X-Cache: HIT cache Accept-Ranges: bytes Connection: keep-alive
(4) Extended
1) Why use cache:
Accessed data can be accessed again, the hot data in multiple visits.
After the data has been accessed will visit again closest to him or close to the client.
2) As to the cache, high-speed, the best way to be read, is all into memory.
Common memory database, memcached, redis, HANA.
But for the page, put the whole into memory, too unrealistic, disk cache memory + way to store cache.
Key-value, key stored in memory, value storage disk.
3) A data form: key value
Key: access path, URL, a particular feature, the calculated hash result, which is stored in the key memory.
Value: page body, our users get real content, generally stored in a high-speed hard drive.
4) all associated with the cache buffer, is inseparable from two bodies: memory, high-speed hard drive.
5) common terms:
Hit: remove the data from the cache, if it is a web site, then your cache server will be one of the most front-end server.
Hit rate: the number of hits / (hits + the number of non-hit count).
Thermal data: data frequently accessed.
Space memory cache, disk cache space.
Cleanup: regular cleaning, LRU (not commonly used, the oldest of a class of data to be deleted), regularly updated (purge).
Cache object: user information, cookies, transaction information, page memory, all understanding for the object.
Reference: https://www.oschina.net/translate/speed-your-web-site-varnish?print