Below these are a few questions to be analyzed.
1. The operating system can support one million connections?
For most of the Linux operating system, by default, it does not support C1000K! Because the operating system contains the maximum number of open files (Max Open Files) limit, into the global system, and process-level restrictions.
Global restrictions
Executed under Linux:
cat /proc/sys/fs/file-nr
Prints line like the following output:
5100 0 101747
The third number 101747 is the current system of global maximum number of open (Max Open Files) files, you can see that only 100,000, so on this server can not support this value C1000K. Many smaller systems, in order to modify the value, modify the file /etc/sysctl.conf as root:
fs.file-max = 1020000
net.ipv4.ip_conntrack_max = 1020000
net.ipv4.netfilter.ip_conntrack_max = 1020000
Process limit
carried out:
ulimit -n
Output:
1024
A description of each process is currently only open Linux system up to 1024 files. To support C1000K, you also need to modify the limit.
Temporary modification
ulimit -n 1020000
However, if you are not root, may not be modified over 1024, you will get an error:
-bash: ulimit: open files: cannot modify limit: Operation not permitted
Permanent modification
/Etc/security/limits.conf edit files, add the following line:
# /etc/security/limits.conf
work hard nofile 1020000
work soft nofile 1020000
The first column represents the work of users work, you can fill *, or root. then save and exit and re-login server.
Note: Linux kernel source there is a constant (NR_OPEN in /usr/include/linux/fs.h), limits the maximum number of open files, such as RHEL 5 is 1048576 (2 ^ 20), so, in order to support C1000K, you You may also need to recompile the kernel.
2. How much memory the operating system needs to maintain one million connections?
Solve the parameter limits of the operating system, then we must look at the occupancy of memory. First, the operating system itself to maintain these connections memory footprint for the Linux operating system, socket (fd) is an integer, so, guess operating system manage one million connections should be occupied by the memory 4M / 8M, and then include some management information, should be about 100M. However, there are memory socket send and receive buffers occupied not analyzed. For this reason, I write the most original C program to verify network:
server
#include
#include
#include
#include
#include
#include
#include
#include
#define MAX_PORTS 10
int main(int argc, char **argv){
struct sockaddr_in addr;
const char *ip = "0.0.0.0";
int opt = 1;
int bufsize;
socklen_t optlen;
int connections = 0;
int base_port = 7000;
if(argc > 2){
base_port = atoi (argv [1]);
}
int server_socks[MAX_PORTS];
for(int i=0; i maxfd){
maxfd = server_socks[i];
}
}
int ret = select(maxfd + 1, &readset, NULL, NULL, NULL);
if(ret < 0){
if(errno == EINTR){
continue;
}else{
printf("select error! %s\n", strerror(errno));
exit(0);
}
}
if(ret > 0){
for(int i=0; i
Note that the server is listening on port 10, which is to facilitate testing. Because only one client test machine, do not connect more with the same IP port to create more than 30,000, so the server listen for 10 ports, so a test between the server machine and can create 300,000 connected.
Client
#include
#include
#include
#include
#include
#include
#include
int main(int argc, char **argv){
if(argc <= 2){
printf("Usage: %s ip port\n", argv[0]);
exit(0);
}
struct sockaddr_in addr;
const char *ip = argv[1];
int base_port = atoi (argv [2]);
int opt = 1;
int bufsize;
socklen_t optlen;
int connections = 0;
bzero(&addr, sizeof(addr));
addr.sin_family = AF_INET;
inet_pton (AF_INET, ip, & addr.sin_addr);
char tmp_data[10];
int index = 0;
while(1){
if(++index >= 10){
index = 0;
}
int port = base_port + index;
printf("connect to %s:%d\n", ip, port);
addr.sin_port = htons((short)port);
int sock;
if ((sock = socket (AF_INET, SOCK_STREAM, 0)) == -1) {
goto sock_err;
}
if(connect(sock, (struct sockaddr *)&addr, sizeof(addr)) == -1){
goto sock_err;
}
connections ++;
printf("connections: %d, fd: %d\n", connections, sock);
if(connections % 10000 == 9999){
printf("press Enter to continue: ");
getchar();
}
usleep(1 * 1000);
/*
bufsize = 5000;
setsockopt(serv_sock, SOL_SOCKET, SO_SNDBUF, &bufsize, sizeof(bufsize));
setsockopt(serv_sock, SOL_SOCKET, SO_RCVBUF, &bufsize, sizeof(bufsize));
*/
}
return 0;
sock_err:
printf("error: %s\n", strerror(errno));
return 0;
}
I tested 100,000 connections that are idle, data is not sent nor received anything. At this time, the process only takes up less than 1MB of memory. However, by comparing the free command before and after the program exits, find the operating system used the 200M (approximately) memory to maintain the 10 million connections! If the connection of a million words, the operating system itself will take up to 2GB of memory! that 2KB per connection.
You can modify
/proc/sys/net/ipv4/tcp_wmem
/proc/sys/net/ipv4/tcp_rmem
To control the transmission and reception buffer size of TCP connection (thanks @egmkang).
3. Application to maintain the connection to the millions how much memory?
Through the above test code can be found, the application to maintain one million free connection, the operating system will only occupy memory, view seen by the ps command, the application itself takes up little memory.
4. one million connected network throughput exceeds the limit?
Suppose one million connections 20 percent active, each connected to data transmission per 1KB, the required network bandwidth is 0.2M x 1KB / sx 8 = 1.6Gbps, the server requires at least 10 Gigabit NIC (10Gbps).
to sum up
Linux systems need to modify the kernel parameters and system configuration to support C1000K. C1000K application requires server requires at least 2GB of memory, if the application itself needs memory, this requirement should be at least 10GB of memory. At the same time, the card should be at least Gigabit NIC.
Of course, this is only theory, practical applications require more memory and CPU resources to handle business data.
reference:
* http://www.cyberciti.biz/faq/linux-increase-the-maximum-number-of-open-files/
* http://www.lognormal.com/blog/2012/09/27/linux-tcpip-tuning/
Well, today's share is over here, if you need more technical articles can access the official website of Marco education in Europe!