squid proxy and common proxy access

table of Contents

1, Environment Description
2, set up squid service
- 2.1 Installation Services
- 2.2, placed reception
3, configure the proxy
4, configure the proxy when building k8s
5, summary

Squid Cache (referred to as Squid) is a HTTP proxy server software. Squid widely used, can be used as cache server, network traffic may be filtered help secure the ring can also serve as a proxy server in the chain, the proxy forwards the data to an upper or a direct Internet connection. Squid Unix running in one type of system. Because it is open source software, Squid site modify the source code, compiled to native Windows version [2]; users can install Cygwin on Windows, and then compilation in Cygwin in Squid.
Squid has a long history, fully functional. In addition to HTTP, FTP and HTTPS support is quite good, at 3.0 beta also supports IPv6. But Squid superiors agent can not use the SOCKS protocol.

1, Environment Description

This article involved two servers, two servers are within the network ipaddress, respectively

A: 192.168.0.200
B: 192.168.0.100
C: 192.168.0.101

Which Ahas the ability to access the public network, Band Cdoes not have the ability to access the public network.

So, lets Bhave a public network access or the ability to make Band Cimplement certain functions such as yuminstalling software can be used normally, there are two possible options:

plan 1

In the Aopen kernel packet forwarding server to receive Band Cpackets, Band Cwill own gateway configured to Aaddress this Band Cthe ability to access all through Ato forwarding agents realized that this approach is simple to use, but not for cloud vendors Hosting such as Ali cloud provides ECS, because the cloud host ipand NIC configurations are generally not supported by their own definition of

Scenario 2

In Aa server running squidservices, Band Csome operations through squidto achieve forwarding agent

In this paper, Abuilding on a server squidproxy, and in Band Con the server implementation wget, yum, dockerthe use Aof agents, as well as the A, B, Cthree servers as k8snodes in a cluster use kubeadmto build a k8slocal cluster to be configured

2, set up squid service

2.1 Installation Services

AServer access to the Internet, first Aserver yumsource configured not elaborated here, then install squidService

# rpm -qa | grep squid
# yum install -y squid

The default installation of the squidservice only need to configure it, the default is to deny all server connections, only need to modify to allow all servers connected to

# vim /etc/squid/squid.conf
http_access allow all

Start the service and set to boot

# systemctl start squid.service
# systemctl enable squid.service

2.2, placed reception

After this configuration, the squidproxy server can use the default port 3128, but for safety, only allow trusted server connection, usually need to squiduse the configuration to verify the account authorization, by httpd-toolsgenerating the password file, install

# yum install httpd-tools -y

Generate a password file

# mkdir /etc/squid3/
# 生成密码文件，指定文件路径，其中squid是用户名
# htpasswd -cd /etc/squid3/passwords squid
#提示输入密码，不能超过8个字符，输入密码123456

Test password file

# /usr/lib64/squid/basic_ncsa_auth /etc/squid3/passwords                
squid 123456
OK 
# 测试完成，crtl + c 打断

Configured squidto use authentication, modify the configuration file

# vim /etc/squid/squid.conf
# And finally deny all other access to this proxy
auth_param basic program /usr/lib64/squid/basic_ncsa_auth /etc/squid3/passwords #账户密码文件
auth_param basic realm proxy
auth_param basic children 50 #最多 50 个账户同时运行
auth_param basic realm CoolTube Proxy Server #密码框描述
auth_param basic credentialsttl 2 hours #认证持续时间
acl authenticated proxy_auth REQUIRED #对 authenticated 进行外部认证
http_access allow authenticated #允许 authenticated 中的成员访问
http_access deny all #拒绝所有其他访问
visible_hostname squid.CoolTube #代理机名字

# 这里是端口号，可以按需修改
# http_port 3128 这样写会同时监听 ipv6 和 ipv4 的端口，推荐适应下面的配置方法。
http_port 0.0.0.0:3128

Initialize and restart the service

# squid -z
# systemctl restart squid.service

If you turn on the firewall, but also allow the ports in the firewall 3128policy to CentOS7an example

# 把 3128 端口加入防火墙过滤掉
# firewall-cmd --permanent --zone=public --add-port=3128/tcp
# 重启防火墙
# firewall-cmd --reload

If you want to add a user, you only need to generate a password file once again follow the above step

3, configure the proxy

3.1, configure the global proxy

Modify environment variables file and put it into effect

vim /etc/profile
#在最后加入（有认证的情况）
export http_proxy="http://squid:[email protected]:3128"
export https_proxy="http://squid:[email protected]:3128"
如果没有开启认证
http_proxy=http://192.168.0.200:3128
https_proxy=http://192.168.0.200:3128
# source /etc/profile

3.2, configure wget agent

Add wgetthe environment variable file

# vim ~/.wgetrc
http_proxy=http://192.168.0.200:3128
https_proxy=http://192.168.0.200:3128
use_proxy = on
wait = 30
# 验证，如果返回值为0表示wget可用
# wget --spider -T 5 -q -t 2 www.baidu.com | echo $?
0

3.3, yum configuration proxy

Edit the yumconfiguration file, add the following content

# vim /etc/yum.conf
# 有认证的情况
# Proxy
proxy=http://squid:[email protected]:3128
# 没有认证
# Proxy
proxy=http://192.168.0.200:3128

3.4, proxy configuration docker

dockerConfigure the proxy is to ensure dockerfrom outside the network pullmirroring, installed dockerafter the dockernew configuration file and restart the Configurationdocker

# mkdir -p /etc/systemd/system/docker.service.d
# vim /etc/systemd/system/docker.service.d/http-proxy.conf
[Service]
Environment="HTTP_PROXY=http://squid:[email protected]:3128" 
# systemctl daemon-reload
# systemctl restart docker

Verification, by docker pullthe command pullof the public network image

docker pull busybox:latest

3.5 Troubleshooting

If the time to verify the configuration of the above-mentioned agents, there is no return, Caton or timing out, you can see Athat is server squidproxy server logs to check the log generally can view the more obvious errors
cache.log

tail -f /var/log/squid/cache.log 
2020/02/12 22:56:12 kid1| Store logging disabled
2020/02/12 22:56:12 kid1| DNS Socket created at [::], FD 9
2020/02/12 22:56:12 kid1| DNS Socket created at 0.0.0.0, FD 10
2020/02/12 22:56:12 kid1| Adding nameserver 114.114.114.114 from /etc/resolv.conf
2020/02/12 22:56:12 kid1| Adding nameserver 114.114.115.115 from /etc/resolv.conf
2020/02/12 22:56:12 kid1| helperOpenServers: Starting 0/50 'basic_ncsa_auth' processes
2020/02/12 22:56:12 kid1| helperOpenServers: No 'basic_ncsa_auth' processes needed.
2020/02/12 22:56:12 kid1| HTCP Disabled.
2020/02/12 22:56:12 kid1| Finished loading MIME types and icons.
2020/02/12 22:56:12 kid1| Accepting HTTP Socket connections at local=0.0.0.0:3128 remote=[::] FD 11 flags=9

access.log

# tail -f /var/log/squid/access.log 
1581519614.049     47 192.168.0.100 TCP_TUNNEL/200 5383 CONNECT registry.cn-beijing.aliyuncs.com:443 - HIER_DIRECT/47.95.181.38 -
1581542724.596     39 192.168.0.100 TCP_MISS/503 397 HEAD http://url/ - HIER_NONE/- text/html
1581542754.860    256 192.168.0.100 TCP_TUNNEL/200 4460 CONNECT www.baidu.com:443 - HIER_DIRECT/180.101.49.41 -
1581542818.197    107 192.168.0.100 TCP_MISS/200 262 HEAD http://www.baidu.com/ - HIER_DIRECT/180.101.49.41 text/html
1581542820.465     53 192.168.0.100 TCP_MISS/200 262 HEAD http://www.baidu.com/ - HIER_DIRECT/180.101.49.41 text/html
1581542821.292     53 192.168.0.100 TCP_MISS/200 262 HEAD http://www.baidu.com/ - HIER_DIRECT/180.101.49.41 text/html
1581542821.864     54 192.168.0.100 TCP_MISS/200 262 HEAD http://www.baidu.com/ - HIER_DIRECT/180.101.49.41 text/html
1581542822.323     53 192.168.0.100 TCP_MISS/200 262 HEAD http://www.baidu.com/ - HIER_DIRECT/180.101.49.41 text/html
1581542822.764     53 192.168.0.100 TCP_MISS/200 262 HEAD http://www.baidu.com/ - HIER_DIRECT/180.101.49.41 text/html
1581542823.243     53 192.168.0.100 TCP_MISS/200 262 HEAD http://www.baidu.com/ - HIER_DIRECT/180.101.49.41 text/html

4, configure the proxy when building k8s

Use kubeadmor binary to build k8sthe cluster, within the cluster podand servicewill alone occupy a different network, servicethe network is a virtual network segment, this time also need to flow statement would not let go of its http/httpsagents, such statements also apply to those who have not domain resolved through binding hostsdomain to access the file.

If you do not declare, in use kubeadmdeploy k8sa cluster, perform the kubeadm initinitialization of the cluster will see the following prompt execution timeout and fail

[WARNING HTTPProxyCIDR]: connection to "10.1.0.0/12" uses proxy "http://192.168.0.200:3128". This may lead to malfunctional cluster setup. Make sure that Pod and Services IP ranges specified correctly as exceptions in proxy configuration
[WARNING HTTPProxyCIDR]: connection to "10.244.0.0/16" uses proxy "http://192.168.0.200:3128". This may lead to malfunctional cluster setup. Make sure that Pod and Services IP ranges specified correctly as exceptions in proxy configuration

Constantly retry when you can see the proxy server's squidlog and found that there have been log from the kubeadmmachine to the machine where the 6443request sent by the port, but was rejected

1581507643.783      0 192.168.0.100 TCP_DENIED/403 3927 CONNECT 192.168.0.8:6443 - HIER_NONE/- text/html
1581507644.283      0 192.168.0.100 TCP_DENIED/403 3927 CONNECT 192.168.0.8:6443 - HIER_NONE/- text/html
1581507644.783      0 192.168.0.100 TCP_DENIED/403 3927 CONNECT 192.168.0.8:6443 - HIER_NONE/- text/html
1581507645.283      0 192.168.0.100 TCP_DENIED/403 3927 CONNECT 192.168.0.8:6443 - HIER_NONE/- text/html
1581507645.783      0 192.168.0.100 TCP_DENIED/403 3927 CONNECT 192.168.0.8:6443 - HIER_NONE/- text/html
1581507646.283      0 192.168.0.100 TCP_DENIED/403 3927 CONNECT 192.168.0.8:6443 - HIER_NONE/- text/html
1581507646.783      0 192.168.0.100 TCP_DENIED/403 3927 CONNECT 192.168.0.8:6443 - HIER_NONE/- text/html
1581507647.283      0 192.168.0.100 TCP_DENIED/403 3927 CONNECT 192.168.0.8:6443 - HIER_NONE/- text/html

By stepping pit configuration process is correct and feasible:

Modify dockerproxy configuration

# vim /etc/systemd/system/docker.service.d/http-proxy.conf
[Service]
Environment="HTTP_PROXY=http://squid:[email protected]:3128" "NO_PROXY=localhost,127.0.0.1,192.168.0.200,192.168.0.100,192.168.0.101,10.244.0.0/16"
# systemctl daemon-reload
# systemctl restart docker

Modify global proxy configuration

# vim /etc/profile
export http_proxy="http://squid:[email protected]:3128"
export https_proxy="http://squid:[email protected]:3128"
export no_proxy=localhost,127.0.0.1,192.168.0.200,192.168.0.100,192.168.0.101,10.244.0.0/16
# source /etc/profile

Command line statement

export http_proxy="http://squid:[email protected]:3128"
export https_proxy="http://squid:[email protected]:3128"
export no_proxy=localhost,127.0.0.1,192.168.0.200,192.168.0.100,192.168.0.101,10.244.0.0/16

And then execute again kubeadm initinitialize the cluster will be successful

5, summary

This paper is recorded in a particular network environment how to squidconfigure the proxy method of network machines to realize common functions of the Internet, of course, I mentioned here yum, wget, dockerand k8smounted outside the cluster and some common operations also need access agent, here not numerous to mention.

Well, the epidemic is not over, can only work at home late at night insomnia, peace!