It was found that the lanproxy corresponding to a certain cloud server was invalid. During the investigation, it was found that it was caused by the number of TCP TIME_WAIT sockets. (At first I thought it was because the lanproxy service was down or the bandwidth was full, but later I found out it was not)
netstat -an
Description of the problem:
In the high-concurrency Squid server in the Linux system, the number of TCP TIME_WAIT sockets often reaches 20,000 to 30,000, and the server is easily dragged to death.
Solution:
By modifying the Linux kernel parameters, the number of IME_WAIT sockets of the Linux server can be reduced. Add the following lines to
vi /etc/sysctl.conf :
#代码如下:
net.ipv4.tcp_fin_timeout = 30
net.ipv4.tcp_keepalive_time = 1200
net.ipv4.tcp_syncookies = 1
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_tw_recycle = 1
net.ipv4.ip_local_port_range = 1024 65000
net.ipv4.tcp_max_syn_backlog = 8192
net.ipv4.tcp_max_tw_buckets = 5000
Note:
net.ipv4.tcp_syncookies = 1 means enable SYN Cookies. When the SYN waiting queue overflows, enable cookies to handle it, which can prevent a small amount of SYN attacks. The default is 0, which means it is closed;
net.ipv4.tcp_tw_reuse = 1 means enable reuse. Allow TIME-WAIT sockets to be reused for new TCP connections, the default is 0, which means closed;
net.ipv4.tcp_tw_recycle = 1 means to enable the fast recycling of TIME-WAIT sockets in the TCP connection, and the default is 0, which means it is turned off.
net.ipv4.tcp_fin_timeout = 30 means that if the socket is closed by the local end, this parameter determines the time it remains in the FIN-WAIT-2 state.
net.ipv4.tcp_keepalive_time = 1200 indicates the frequency of TCP sending keepalive messages when keepalive is enabled. The default is 2 hours, change it to 20 minutes.
net.ipv4.ip_local_port_range = 1024 65000 Indicates the port range for outgoing connections. Small by default: 32768 to 61000, changed to 1024 to 65000.
net.ipv4.tcp_max_syn_backlog = 8192 indicates the length of the SYN queue, the default is 1024, increasing the queue length to 8192 can accommodate more network connections waiting to be connected.
net.ipv4.tcp_max_tw_buckets = 5000 means that the system maintains the maximum number of TIME_WAIT sockets at the same time. If this number is exceeded, the TIME_WAIT sockets will be cleared immediately and a warning message will be printed. The default is 180000, change it to 5000. For servers such as Apache and Nginx, the parameters in the above lines can reduce the number of TIME_WAIT sockets very well, but for Squid, the effect is not great. This parameter can control the maximum number of TIME_WAIT sockets to prevent the Squid server from being dragged to death by a large number of TIME_WAIT sockets.
Execute the following command to make the configuration take effect:
/sbin/sysctl -p
Original link: Detailed explanation of the method to solve a large number of TIME WAIT under linux https://www.jb51.net/article/37834.htm