squid proxy

squid proxy

Course objectives

• Understand the application scenario of squid
• Understand the working principle and the role of squid
• Li squid proxy type (Key)
• Master forward and transparent proxy configuration squid's (Key)
• You can do simple access control for squid server on demand

A, squid basic overview

What 1.squid that? what is it used for?

squid cache (referred to as squid) is the most widely popular, most widely used open-source caching proxy server

2.squid role and application scenarios What?

• It used to make the front of the web cache, to speed up user access to the web
• Within the proxy network users to access Internet resources
• Set access control policy to control the user's online behavior
• The main support http, ftp, etc. Application Protocol
• Official website address: http: //www.squid-cache.org/

3. Common proxy server software

• squid very old, fully functional
• squid varnish superior performance, multi-function relatively free squid
• Other haproxy, nginx, etc.

Works of 4.squid

Advantages: reducing the access request, save bandwidth

Have ACL (access control list) access control list feature, online behavior of clients flexible control

Internal network client has a protective effect

Two, squid proxy type

1. Forward Proxy

2. Transparent Proxy

Forward proxy process to hide the true client's request, the server does not know who the real client, the client requested services are replaced by the proxy server requests. We often say that the agent is forward proxy, forward proxy agent is requesting party is the client.

3. Reverse Proxy

The process of reverse proxy hides the real server, the client does not know who the real service delivery, client service requests are processed proxy server. Reverse proxy is a proxy responder, which is the server.

4. Summary

• Forward and transparent proxy proxy servers and clients belong to the same LAN, on the server end is transparent, the server does not know whom to provide services. Role is to let network users via the Internet on the proxy server, can also increase the access speed, and can control the online behavior within the limits of network users access through a proxy server.
• The process of reverse proxy server hides the real, is transparent to the client, the client does not know the real server to provide access. End load balancing can play a role in enhancing the user's access speed.
• The difference is that the object is not the same proxy, forward proxy object is a client proxy, reverse proxy server is a proxy object.
• Scenarios
  • Forward and transparent proxy: generally used within the company network users to access the Internet, access control based on demand
  • Reverse Proxy: generally used to do before the company web cache server cluster, improve user access efficiency, but also can play a responsible role in a balanced, sustainable end web for the Internet service.

Three, squid proxy server and SNAT | DNAT's the difference

1.SNAT and DNAT

• Network layers: network layer

  LAN users access to the peripheral go SNAT mode, the same data packet, change the source address of the packet header, and then send the packet to the internet

• Role: on the internal network users outside the network (SNAT) and intranet services published to the public network (DNAT)

2. Proxy Mode

• Network layers: application layer

  Packet header information does not change, the data packet to the Internet proxy server, terminal-based application layer filtering.

• Application layer:

  Protocols: http, ftp, pop, smtp, p2p, etc.

• Role: page caching, to accelerate access, ACL resource access control

• Reverse Proxy:

  • Internet users access the LAN server
  • squid do a reverse proxy server, the client request is forwarded to the rear end of the real end servers can do load balancing end results page while caching frequently accessed by the user in order to improve access speeds.

Fourth, understand squid proxy software-related information

1. Install the software

squid在本地镜像里有，只需要配置好本地端yum源即可
yum -y install squid

2. The list of software files

rpm -ql squid
/etc/squid/squid.conf   配置文件
/etc/rc.d/init.d/squid  启动脚本
/usr/sbin/squid     二进制命令
/usr/share/doc/squid-3.1.23 手册
/var/log/squid      日志目录
/var/spool/squid    缓存目录

3. Profile Description

[root@agent2 ~]# vim /etc/squid/squid.conf      简化的配置文件
[root@agent2 ~]# vim /usr/share/doc/squid-3.1.23/squid.conf.documented  查看文档
//设置监听端IP和端口号
http_port 3128
//额外提供给squid使用端内存大小
cache_mem   256MB(默认大小)
//设置squid磁盘缓存最大文件，超过4M端文件不保存到磁盘
maximum_object_size 4MB
//设置squid磁盘缓存最小文件
minimum_object_size 0 KB
//设置squid内存缓存最大文件，超过4M不保存到内存
maximum_object_size_in_memory 4096 KB
//定义squid端cache存放路径、cache目录容量（单位M）、一级缓存目录数量、二级缓存目录数量
cache_dir ufs /var/spool/squid 100 16 256

//log文件日志格式
#logformat combined %>a %ui %un [%tl] "%rm %ru HTTP/%rv" %>Hs %<st "%{Referer}>h" "%{User-Agent}>h" %Ss:%Sh
//log文件存放路径和日志格式
# access_log /var/log/squid/access.log squid
//设置缓存日志
cache_log /var/log/suqid/cache.log
//log轮转60天
logfile_rotate 60
//cache目录使用量大于95%时，开始清理旧的cache
cache_swap_high 95
//cache目录清理到90%时停止
cache_swap_low 90

//定义本地网段
acl localnet src 192.168.1.0/24
//允许本地网段使用
http_access allow localnet
//拒绝所有
http_access deny all
//主机名
visible_hostname xxx.xxxx.xxxx

//管理员邮箱
cache_mgr [email protected]
 

Five, squid proxy to the correct configuration

Requirements: network users specify a proxy server ip and port to access the Internet through a browser-side web services

环境：client：10.1.1.2
squid—server：10.1.1.1和2.2.2.1
web-server：2.2.2.2
（都是仅主机模式）
思路：
1.squid-server需要两个IP，一个公网一个私网
2.client端只需要和squid-server能互通就行，不需要指定gw和dns
3.client端需要在浏览器上设置代理服务器端IP和端口
4.squid-server可以上外网（要能访问2.2.2.2），client不能上外网，目的就是让squid代理内网用户访问互联网web服务
5.搭建一个web服务

步骤：
搭建squid-server服务器
1.安装软件
yum -y install squid

把主机名改为squid-server.itcast.cc 下面要用到，临时改就行，要求不严

2.根据需求修改配置文件
需求：代理内网所有用户访问互联网端web服务
cp /etc/squid/squid.conf /etc/squid/squid.conf.bak
vim /etc/squid/squid.conf
...
#http_access deny all   注释掉这行
http_access allow all   允许所有人来访问代理服务器

cache_dir ufs /var/spool/squid 100 16 256   取消掉前面注释，打开缓存目录
增加以下两行：
visible_hostname squid-server.itcast.cc     定义squid服务器主机名
access_log  /var/log/squid/access.log       定义访问日志

3.启动squid服务
service squid start
init_cache_dir /var/spool/squid... Starting squid: .       [ ok ]

服务器启动后会再缓存目录里产生以下一级、二级缓存目录文件
ls /var/spool/squid/
00 01 02 03 04 05 06 07 08 09 0A 0B 0C 0D 0E 0F  swap.state

查看端口3128已经处于监听状态

注意：如果再次重启会报以下警告：
WARNING cache_mem is larger than total disk cache space!
原因：默认情况下，cache_mem为256M，而squid端配置里磁盘缓存默认为100M
解决：
方法1：调小cache_mem的值
在配置文件中增加 cache_mem 100MB
方法2：调大磁盘缓存
修改配置文件中如下内容：
cache_dir ufs /var/spool/squid 256 16 256   ——>把100换成256
说明：squid缓存从 内存 ————> 磁盘
    cache_mem   /var/spool/squid
    
    
搭建web服务：
web-server：2.2.2.2

rpm -q httpd
做一个网页文件，带图片的，可以展示缓存,放到/var/www/html/里面
启动httpd


客户端测试：
client：10.1.1.2 注意：不需要设置网关和dns
配置代理服务器前：不能够直接访问web服务
ping 2.2.2.2 网络不可达
客户端内火狐浏览器访问http://2.2.2.1不能连接（因为没有打开转发功能echo 1 > /proc/sys/net/ipv4/ip_forward，此处不需要打开）
手动配置代理服务器ip和port
edit-->preferences--advanced-->network-->connection-->setting-->manual proxy configuration
输入代理服务器端内网地址10.1.1.1，port 3128
此时再输入http://2.2.2.2
能够看到自己端网页文件

内网不直接访问web服务器，但是通过代理服务器访问到了，这就是代理

假设web-server宕机，
service httpd stop
客户端重新输入http://2.2.2.2，仍能访问到

Six, squid transparent proxy configuration

Requirements: network users can directly access the Internet web service through a proxy server, do not specify the proxy address and port

思路：
1.squid-server需要两个IP，一个公网一个私网
2.client端浏览器不需要指定代理服务器的IP和端口，但是需要指定网关为squid-server的私网地址
3.squid-server需要借助iptables将内网的80端口重定向到自己的3128端口帮内网用户代理上网
4.搭建一个web服务
（仅主机模式）

步骤：
client端浏览器清空刚刚的代理设置（不要忘记）
client端指定自己的默认网关为squid-server的私有地址
[root@client ~] route add default gw 10.1.1.1

squid-server端配置透明代理服务器并添加iptables规则：
1.修改squid配置文件
vim /etc/squid/squid.conf
http_port 3128 transparent  //在代理端口后面加上transparent（透明），代表透明代理
重启服务：
service squid restart

2.添加防火墙规则
iptables -t nat -A PREROUTING -s 10.1.1.0/24 -p tcp --dport 80 -j REDIRECT --to-port 3128
iptables -t nat -nL

说明：
-j REDIRECT 表示端口重定向，仅用于tcp和udp协议
This is only valid if the rule also specifies -p tcp or -p udp

3.客户端测试验证
1）打开浏览器，输入http://2.2.2.2    可以看到web-server的web页面
curl -I http://2.2.2.2/index.html   //查看是否缓存
。。。
X-Cache:HIT from squid-server.itcast.cc
X-Cache-Lookup:HIT from squid-server.itcast.cc:3128     //HIT代表命中，缓存

。。。
X-Cache:MISS from squid-server.itcast.cc
X-Cache-Lookup:MISS from squid-server.itcast.cc:3128    
2）web-server端，查看/var/log/httpd/access.log日志
3）squid-server端，查看/var/log/squid/access.log日志
4）测试web-server服务停止，squid-server是否缓存

只能上外网，访问页面，不能ping通2.2.2.2
另外：
windows不能访问是因为网关不同，网络不能访问squid-server端

Seven, squid access control

1. Access control configuration syntax

1.禁止IP地址为192.168.0.100的客户机上网
    acl test src 192.168.0.100  //定义来源IP，test可以自己定义，代表定义端IP
    http_access deny test   //拒绝test后面定义端源IP ：192.168.0.100上网
可以把要管理的IP写到一个文件里，然后在src后面跟上“文件路径”

2.禁止192.168.1.0这个网段里端所有客户机上网
    acl test src 192.168.1.0/255.255.255.0
    或者
    acl test src 192.168.1.0/24
    http_access deny test
    
3.禁止用户访问IP为210.21.118.68的网站
    acl test dst 210.21.118.68
    http_access deny test

4.禁止用户访问域名为www.163.com的网站
    acl test dstdomain -i www.163.com   //-i 忽略大小写
    http_access deny test
        
5.禁止用户访问域名包含有163.com的网站
    acl test url_regex -i 163.com   //使用正则表达式匹配URL地址
    http_access deny test
    
6.禁止用户访问包含有sex关键字端URL
    acl test url_regex -i sex
    http_access deny test

7.禁止用户访问多个网址
    acl test dstdom_regex "/etc/squid/baddaturl"
    //如果网址太多，可以写成一个文件，然后在这个文件里一行一个网站写上要
    http_access deny test
    
8.vip用户没有任何限制
    acl vip arp 00：0c:29:79:0C:1A   //绑定mac地址
    http_access allow vip

9.禁止192.168.2.0这个子网内所有客户机在周一到周五的9点到18点上网
    acl test1 src 192.168.2.0/255.255.255.0
    acl test2 time MTWHF 9:00-18:00     /S日/M一/T二/W三/H四/F五/A六/D（周一到周五）
    http_access deny test1 test2
    
hwclock --hctosys   还原时间命令

10.禁止用户下载*.mp3,*.exe,*.zip,*.rar带有定义后缀名的文件
    acl test urlpath_regex -i \.mp3$ \.exe$ \.zip$ \.rar$ \.txt$
    //urlpath_regex 正则表达式匹配URL地址
    http_access deny test

注意：设置acl访问控制规则的时候，有先后顺序。第一条规则匹配上就不再继续往下，所以先把拒绝写在前面

2. Applications

2.1 based on IP address access control

2.2 According domain access control

2.3 access control based on time and date

2.4 access control based on the file name

3. Integrated Case

Eight, reverse proxy (Extended)

Requirements: within a company network set up a web server, Internet users want to access the web service, the better

环境：
client：2.2.2.2  互联网用户
squid-server：2.2.2.1和10.1.1.1   2个IP，一个公网一个私网
web-server：10.1.1.3     公司内网web服务器

还原acl控制

搭建反向代理服务器squid步骤：
1.清空之前的环境
iptables -t nat -F
2.修改squid.conf配置文件来配置反向代理
vim /etc/squid/squid.conf
...
http_port 80 vhost
cache_peer 10.1.1.3 parent 80 0 originserver

解释说明：
less /usr/share/doc/squid-3.1.10/squid.conf.documented
...
cache_peer hostname type http-port icp-port [options]

cache_peer：做反向代理的关键字
hostname：代表后台真正的web服务器
type：代表代理服务器的类型，有PARENT和SIBLING两种：parent代表父亲，sibling代表兄弟伙伴
http-port：后台web服务的端口
icp-port：代表代理服务器之间通讯的端口（用于查询兄弟缓存的端口）
[options]：代表可选项
originserver：代表原始服务器（真正的web服务器）
weight：代表权重
max-coon：代表
...

3.测试验证
在client端：2.2.2.2上直接访问代理服务器：
http://2.2.2.1      （访问代理服务器）