背景:
公司为节约成本,需要部署反向代理设备,用以缓存视频文件,节省机房出口带宽,增加廉价机房的上联出向流量。本人之前接触较多squid,但由于前公司使用时间较久,个人也是在使用上熟悉,接下来就是自己部署的过程记录,方便大家查阅。本文只解释编译安装的方法。安装环境:
系统环境:Ubuntu 12.04.1 LTS. x86_64 8G内存,12核心。由于是dell服务器,只得系统盘做了raid1,数据盘做了raid5.数据盘挂在了/srv下,整个磁盘大小是40T.可用大小37T。
软件环境:
登录到squid官网,http://www.squid-cache.org/Versions/ 下载最新版本。因为只用到基本功能,因此,只需要下载稳定版本即可。这里我选择的是3.5.22版本,发布时间2016-10.9日,下载链接:http://www.squid-cache.org/Versions/v3/3.5/squid-3.5.22.tar.gz。
软件安装及配置:
下载上边软件链接到本地服务器上,解压后生成一个名为squid-3.5.22的文件夹,该文件便是squid的源码。详细使用方法可以参考这里,http://zyan.cc/book/squid/ 此链接为squid的中文权威指南。对squid的基本使用和原理做了比较详尽的解释。编译安装:
安装前先对ubuntu系统进行软件升级 apt-get update 安装编译需要的环境 apt-get install build-essential 下载安装 wget http://www.squid-cache.org/Versions/v3/3.5/squid-3.5.22.tar.gz tar -zxvf squid-3.5.22.tar.gz cd squid-3.5.22/ ./configure --prefix=/var/www/dream/squid --sysconfdir=/var/www/dream/squid/etc --enable-arp-acl --enable-linux-netfilter --enable-pthreads --enable-err-language="Simplify_Chinese" --enable-default-err-language="Simplify_Chinese" --enable-storeio=ufs,aufs ##确定使用存储的格式,这里由于我们存储较大视频文件10M左右甚至以上,所以这里选择aufs。 --enable-baisc-auth-helpers="NCSA" --enable-underscore make make install ln -s /var/www/dream/squid/sbin/squid /usr/bin/squid ##设置squid成为系统可以识别的系统命令。
配置:
#<common_begin ############################## Common ACL Define ################################# acl localhost src 127.0.0.1/255.255.255.255 acl monitor src 127.0.0.1/255.255.255.255 acl manager proto cache_object acl SSL_ports port 443 acl Safe_ports port 80 # http acl Safe_ports port 21 # ftp acl Safe_ports port 443 # https acl Safe_ports port 70 # gopher acl Safe_ports port 210 # wais acl Safe_ports port 1025-65535 # unregistered ports acl Safe_ports port 280 # http-mgmt acl Safe_ports port 488 # gss-http acl Safe_ports port 591 # filemaker acl Safe_ports port 777 # multiling http acl CONNECT method CONNECT http_access deny !Safe_ports http_access deny CONNECT !SSL_ports acl PURGE method PURGE http_access allow PURGE localhost http_access deny purge http_access allow manager monitor http_access deny manager acl snmppublic snmp_community public snmp_access allow snmppublic localhost acl spider_ua req_header User-Agent -i spider http_access deny spider_ua acl ip_in_fqdn url_regex -i ^http://[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}/.* http_access deny ip_in_fqdn #acl allow_ip src "/var/www/dream/squid/etc/allow_ip" #http_access allow allow_ip #http_reply_access allow allow_ip ######### Device Configuration Define ############### visible_hostname proxy45-236 #cache_dir Directory-Name Mbytes Level-1 Level2 cache_dir aufs /srv/cache1 3000000 128 128 min-size=1000000 maximum_object_size 10240000 KB minimum_object_size 0 KB store_dir_select_algorithm round-robin cache_replacement_policy lru cache_swap_low 80 cache_swap_high 85 reload_into_ims on #request_header_access Via allow all #header_access Via allow all #header_access X-Cache deny all #header_access X-Cache-Lookup deny all #header_access X-Squid-Error deny all dns_timeout 2 minute dns_retransmit_interval 10 seconds negative_dns_ttl 1 minute #refresh_stale_hit 0 minute vary_ignore_expire on request_timeout 40 seconds persistent_request_timeout 5 seconds connect_timeout 1 minute read_timeout 15 minutes server_persistent_connections off client_persistent_connections off collapsed_forwarding on forwarded_for on #ignore_expect_100 on maximum_object_size_in_memory 2 MB cache_mem 512 MB memory_replacement_policy lru #logformat squid_custom_log %ts.%03tu %6tr %{X-Real-IP}>h %Ss/%03Hs %<st %rm %ru %un %Sh/%<A %mt "%{Referer}>h" "%{User-Agent}>h" "%{Cookie}>h" logformat squid_custom_log %ts.%03tu %6tr %>a %Ss/%03Hs %<st %rm %ru %un %Sh/%<A %mt "%{Referer}>h" "%{User-Agent}>h" "%{Cookie}>h" cache_access_log /var/www/dream/squid/var/logs/access.log squid_custom_log strip_query_terms off logfile_rotate 0 cache_log /var/www/dream/squid/var/logs/cache.log cache_store_log none #http_port 80 accel vhost vport #http_port 9090 accel vhost vport=80 allow-direct http_port 80 accel vhost vport allow-direct #http_port 80 icp_port 3130 icp_access allow all snmp_port 3401 cache_effective_user squid cache_effective_group squid pid_filename /var/www/dream/squid/var/run/squid.pid diskd_program /var/www/dream/squid/libexec/diskd unlinkd_program /var/www/dream/squid/libexec/unlinkd half_closed_clients off #client_db off #cachemgr_passwd test4squid config #cache_mgr [email protected] hosts_file /var/www/dream/squid/etc/hosts.conf #dns_nameservers 127.0.0.1 dns_nameservers 8.8.8.8 ################Domain Begin#### #会把Range头去掉,再回后端服务器。所以都能取回整个文件。默认range_offset_limit 0 如果请求带有Range头,则squid会把Range头部转发到后端,不会预取多余的数据。 range_offset_limit -1 #这样的话比如一个1G的文件,用户下载到1M中断了,squid会一直把1G文件下载下来,但是如果文件大小超过"maximum_object_size"大小呢,下载完毕又放弃了。 quick_abort_min -1 ################Domain End##### # Add any of your own refresh_pattern entries above these. coredump_dir /var/www/dream/squid/var/cache/squid refresh_pattern -i ^http 1440 0% 1440 ignore-reload override-lastmod http_access allow all http_reply_access allow all
以上配置可以让服务正常跑起来。
日志定期清理脚本并保留近30天日志
#!/bin/sh set -e yesterday_date=$(date -d "1 day ago" +%Y%m%d) cd /var/www/dream/squid/log/ # rename the current log file without interrupting the logging process mv access.log access.log.$yesterday_date mv cache.log cache.log.$yesterday_date # tell Squid to close the current logs and open new ones /var/www/dream/squid/sbin/squid -k rotate -f /var/www/dream/squid/etc/squid.conf /var/www/dream/squid/sbin/squid -k rotate -f /var/www/dream/squid/etc/squid1.conf /var/www/dream/squid/sbin/squid -k rotate -f /var/www/dream/squid/etc/squid2.conf /var/www/dream/squid/sbin/squid -k rotate -f /var/www/dream/squid/etc/squid3.conf # give Squid some time to finish writing swap.state files sleep 30 mv access.log.$yesterday_date bak/ mv cache.log.$yesterday_date bak/ gzip -9 bak/access.log.$yesterday_date gzip -9 bak/cache.log.$yesterday_date find bak/ -type f -ctime +30 | xargs rm