Network programming
In tcp applications, the server listens on a fixed port in advance, the client initiates a connection actively, and establishes a tcp connection after a three-way handshake. So for a single machine, what is the maximum number of concurrent tcp connections?
How to identify a TCP connection
Before determining the maximum number of connections, let's take a look at how the system identifies a TCP connection. The system uses a 4-tuple to uniquely identify a TCP connection: {local ip, local port, remote ip, remote port}.
The maximum number of tcp connections for the
client each time the client initiates a tcp connection request, unless the port is bound, the system usually asks the system to select a free local port, which is exclusive and cannot be shared with other tcp connections. The data type of the TCP port is unsigned short, so the maximum number of local ports is 65536. Port 0 has a special meaning and cannot be used. In this way, the available ports are only 65535. Therefore, when all are used as the client side, the maximum number of TCP connections is 65535. These connections can be connected to different server ip.
The maximum number of tcp connections for the
server. The server is usually fixed on a local port to monitor and wait for the client's connection request. Regardless of address reuse (UNIx SO_REUSEADDR option), even if there are multiple ips on the server side, the local listening port is exclusive, so the server-side tcp connection 4-tuple only has remote ip (that is, client ip) and remote port (Client port) is variable, so the maximum tcp connection is the number of client ip × the number of client port. For IPV4, regardless of the ip address classification and other factors, the maximum number of tcp connections is about 2 to the 32nd power (ip number )×2 to the 16th power (the number of ports), that is, the maximum number of tcp connections on the server side is about 2 to the 48th power.
Then how many concurrent TCP connections can be on a single server
The above is the theoretical maximum number of connections for a single machine. In the actual environment, it is limited by machine resources and operating systems, especially on the server side. The maximum number of concurrent tcp connections is far from reaching the theoretical upper limit. The main factors that limit the number of connections under unix/linux are memory and the number of file descriptors allowed (each tcp connection takes up a certain amount of memory, and each socket is a file descriptor). In addition, ports below 1024 are usually reserved port. In the default 2.6 kernel configuration, after testing, each socket occupies between 15-20k of memory.
The parameters that affect the memory occupied by a socket include:
rmem_max
wmem_max
tcp_rmem
tcp_wmem
tcp_mem
grep skbuff /proc/slabinfo
On the server side, by increasing the memory, modifying the maximum number of file descriptors and other parameters, the maximum number of concurrent TCP connections for a single machine exceeds 100,000. Yes, the foreign Urban Airship company has achieved 500,000 concurrency in the product environment. In practical applications, for large-scale network applications, C10K issues need to be considered.
Let me explain in detail the two common senses of file handle restrictions and port restrictions.
Common sense 1: File handle restrictions
Friends who write network server programs under linux must know that every tcp connection occupies a file descriptor. Once this file descriptor is used up, the error returned to us by the new connection is "Socket/File: Can't open so many files".
At this time you need to understand the operating system's limitation on the maximum number of files that can be opened.
Process limit
Executing ulimit -n outputs 1024, indicating that only 1024 files can be opened for one process, so if you use this default configuration, you can have at most thousands of concurrent TCP connections.
Temporary modification: ulimit -n 1000000, but this temporary modification is only valid for the current use environment of the currently logged-in user, and will become invalid after the system restarts or the user logs out.
Modifications that fail after restart (but I tested it under CentOS 6.5, and no failure was found after restart): Edit the /etc/security/limits.conf file, and the modified content is
* soft nofile 1000000
* hard nofile 1000000
Permanent modification: edit /etc/rc.local and add the following content after it
ulimit -SHn 1000000
Global limit
Execute cat /proc/sys/fs/file-nr to output 9344 0 592026, which are: 1. The number of file handles that have been allocated, 2. The number of file handles that have been allocated but not used, and 3. The maximum number of file handles. But in the kernel 2.6 version, the value of the second item is always 0. This is not an error. It actually means that the allocated file descriptors have been used without any waste.
We can increase this value and modify the /etc/sysctl.conf file with root privileges:
fs.file-max = 1000000
net.ipv4.ip_conntrack_max = 1000000
net.ipv4.netfilter.ip_conntrack_max = 1000000
Common sense 2: Port number range restriction?
The port numbers below 1024 on the operating system are reserved by the system, and those from 1024 to 65535 are used by users. Since each TCP connection occupies a port number, we can have more than 60,000 concurrent connections at most. I think there are not a few friends who have this wrong idea, right? (Among them I have always thought so in the past)
Let's analyze it
How to identify a TCP connection: The system uses a 4-tuple to uniquely identify a TCP connection: {local ip, local port, remote ip, remote port}. Well, let's take out the explanation of accept in Chapter 4 of "UNIX Network Programming: Volume One" to take a look at the conceptual things. The second parameter cliaddr represents the client's ip address and port number. As the server, we actually only use this port when bind, indicating that the port number 65535 is not a limitation of the concurrency.
The maximum number of tcp connections for the server: The server is usually fixed on a local port to listen and wait for the client's connection request. Regardless of address reuse (UNIx SO_REUSEADDR option), even if there are multiple ips on the server side, the local listening port is exclusive, so the server-side tcp connection 4-tuple only has remote ip (that is, client ip) and remote port (Client port) is variable, so the maximum tcp connection is the number of client ip × the number of client port. For IPV4, regardless of the ip address classification and other factors, the maximum number of tcp connections is about 2 to the 32nd power (ip number )×2 to the 16th power (the number of ports), that is, the maximum number of tcp connections on the server side is about 2 to the 48th power.