Table of contents
1. Optimization solution for millions of connections on a single machine
1.1. Break through the local file handle limit
First enter the command on the server to see the maximum number of handles that a single process can support.
sh ulimit -n
After entering the command, the number 1024 will appear, indicating the maximum number of files that a process can open in the Linux system. Since opening a TCP connection will create a corresponding file in the Linux system, it is limited by the maximum number of files in this file. . So why is the number of server connections demonstrated earlier finally fixed at 870, which is smaller than 1024? In fact, it is because in addition to the number of connections, the file Class classes opened by the JVM are also counted as files opened in the process, so, 1024 minus The rest of the number of files opened by the JVM is the number of connections that TCP can support. Next, find a way to break through this limitation. First, enter the following command on the server command line to open /etc/security/limits.confthe file.
sh sudo vi /etc/security/limits.conf
Then
add the following two lines of code at the end of this file.
```sh
- hard nofile 1000000
- soft nofile 1000000
``` The first one
represents the current user, hard and soft represent restrictions and warning limits respectively, nofile represents the maximum number of files, and the following number 1 000 000 represents that any user can open 1 million files. This It is also the maximum value supported by the operating system, as shown in the figure below.*
Next, enter the following command.
At sh ulimit -n
this
Rerun the server program and client program separately. At this time, you only need to quietly observe the changes in the number of connections. The final number of connections stayed at 137,920, and an exception was thrown, as shown below.
Current
number of client connections: 137920
Exception in thread “nioEventLoopGroup-2-1” java.lang.InternalError: java.io.FileNotFoundException: /usr/java/jdk1.8.0_121/jre/lib/ext/cldrdata.jar ( Too many open files)
... Why is
this
1.2. Break through the global file handle limit
First enter the following command on the Linux command line to view the number of files that can be opened by all user processes in the Linux system.
sh cat /proc/sys/fs/file-max
Through
the above command, you can see the global limit and find that the result is 10,000. As you can imagine, the number of local file handles cannot be greater than the number of global file handles. Therefore, the global limit on the number of file handles must be increased to break through this limit. First switch to the ROOT user, otherwise you will not have permissions.
sh sudo -s echo 2000> /proc/sys/fs/file-max exit
Let’s
change it to 20 000 to test it and continue experimenting. Start the server program and client program respectively, and find that the number of connections has exceeded the limit of 20,000.
If you use echo to configure /proc/sys/fs/file-maxit before, it will be invalid after restarting the server and will return to the original 10 000. Therefore, directly use the vi command to modify it and enter the following command line.
Add the following content to the end of the file. The results are shown below.sh sodu vi /etc/sysctl.conf /etc/sysctl.confsh fs.file-max=1000000
Next, restart the Linux server, and then start the server program and client program.
sh 当前客户端连接数: 9812451 当前客户端连接数: 9812462 当前客户端连接数: 9812489 当前客户端连接数: 9812501 当前客户端连接数: 9812503 ...
The final
number of connections was fixed at around 980,000. We found that it was mainly limited by the performance of the machine itself. Use htopthe command to check and find that the CPU is close to 100%, as shown in the figure below.
The above is the optimization and performance improvement at the operating system level.