Advanced C++ Network Programming _IO Multiplexing

Article directory

- - - The `C10k` problem that cannot be avoided in network programming
    - solution

C++I wrote a blog post about getting started with network programming before :
Getting started with socket network programming

It mainly introduces the use of C++network programming APIinterfaces. The example in this blog post is the simplest and most basic call process. It can only achieve one-to-one communication because it uses a synchronous blocking method.socket

Modern network programming needs to consider concurrency, that is, one-to-many communication status.
It is not feasible to continue to use the previous one-to-one communication model, which can only I/Obe realized by improving the network model.

Unavoidable `C10k`problems in network programming

Please add a picture description

The author who raised this question is Dan Kegelthe Winetricksauthor of the original blog
"The C10K problem" .
Related resource expansion:
High-performance network programming classic: "The C10K problem (English)" [Attachment download]

The original text introduces some I/Oframeworks, among which libeventare the more commonly used network libraries for back-end development.
I have also used this library in projects, and it is one of the ideal solutions to achieve multi-concurrency.

C10kThe essence of the problem is to reduce the server resource consumption in the concurrency state of network programs as much as possible,
such as avoiding or reducing frequent creation and destruction of processes and threads. The solution to this problem is to use thread pool to manage thread resources;
avoid or reduce frequent The copy of the data leads to a zero-copy solution.
In general, the key to solving C10kthe problem is to reduce the consumption of these CPU resources as much as possible!

solution

Here I want to introduce different network I/Omodels one by one to introduce an ideal solution for network programming to deal with multiple concurrent scenarios at this stage.
Take the most basic socketcode that implements the one-to-one communication model as an example to modify.
Here, only tcpthe server code is used as an example. The client uses other software to connect:

/* 同步阻塞基本socket服务端模型 */
#include <stdio.h>
#include <sys/socket.h>
#include <sys/types.h>
#include <netinet/in.h>
#include <fcntl.h>
#include <unistd.h>
#include <pthread.h>

#define BUFFER_LENGTH   128

int main() {    
	int listenfd = socket(AF_INET, SOCK_STREAM, 0);  //
	if (listenfd == -1) {
		return -1;
	}
			
	struct sockaddr_in servaddr;
	servaddr.sin_family = AF_INET;
	servaddr.sin_addr.s_addr = htonl(INADDR_ANY);
	servaddr.sin_port = htons(9999);
	
	if (-1 == bind(listenfd, (struct sockaddr*)&servaddr, sizeof(servaddr))) {
			return -2;
	}
	listen(listenfd, 10);
	
	struct sockaddr_in client;
	socklen_t len = sizeof(client);
	int clientfd = accept(listenfd, (struct sockaddr*)&client, &len);
	
	unsigned char buffer[BUFFER_LENGTH] = {0};
	int ret = recv(clientfd, buffer, BUFFER_LENGTH, 0);
	printf("buffer : %s, ret: %d\n", buffer, ret);
	
	ret = send(clientfd, buffer, ret, 0);
	
	close(clientfd);
	return 0;
}

Use a loop to handle multiple socketconnections

The traditional I/Onetwork model is the blocking I/Omodel, which is the model in the example in the article Introduction to socket network programming .
As mentioned above, this type of model is not suitable for direct application to network programming in multiple concurrent scenarios.
The above code is very simple, and the problem is obvious:

Multiple clients can be connected, but the second client cannot handle sending and receiving data normally;
After the first client is disconnected, the server program also exits directly.

Next, we use loops to deal with the above problems.
Of course, putting the receiving and sending codes directly into the loop like the following will not solve the fundamental problem:

#include <stdio.h>
#include <sys/socket.h>
#include <sys/types.h>
#include <netinet/in.h>
#include <fcntl.h>
#include <unistd.h>
#include <pthread.h>

#define BUFFER_LENGTH   128

int main() {

// block
        int listenfd = socket(AF_INET, SOCK_STREAM, 0);  //
        if (listenfd == -1) return -1;
// listenfd
        struct sockaddr_in servaddr;
        servaddr.sin_family = AF_INET;
        servaddr.sin_addr.s_addr = htonl(INADDR_ANY);
        servaddr.sin_port = htons(9999);

        if (-1 == bind(listenfd, (struct sockaddr*)&servaddr, sizeof(servaddr))) {
				return -2;
        }
        listen(listenfd, 10);

        struct sockaddr_in client;
        socklen_t len = sizeof(client);
        int clientfd = accept(listenfd, (struct sockaddr*)&client, &len);

        while (1) {
                unsigned char buffer[BUFFER_LENGTH] = {0};
                int ret = recv(clientfd, buffer, BUFFER_LENGTH, 0);
                if (ret == 0) {
                        close(clientfd);
                        break;
                }
                printf("buffer : %s, ret: %d\n", buffer, ret);

                ret = send(clientfd, buffer, ret, 0);
        }

        return 0;
}

It is recommended to read the multithreading model
and understand it together with the following code. The multi-process model is directly skipped here because the process occupies a large amount of resources,
and if the aftermath of the process is not done well , zombie processes will be generated . The more such processes will gradually exhaust our system resources.
Therefore, adopting a lightweight multi-threaded model is a better solution at this stage.
First put the code that handles receiving and sending data into the thread, and use the multi-threading model to handle the socketconcurrency scenario of the connection:

#include <stdio.h>
#include <sys/socket.h>
#include <sys/types.h>
#include <netinet/in.h>
#include <fcntl.h>
#include <unistd.h>
#include <pthread.h>

#define BUFFER_LENGTH   128
void *routine(void *arg) {
        int clientfd = *(int *)arg;
        printf("listen --> clientfd: %d\r\n",clientfd);
        while (1) {
                unsigned char buffer[BUFFER_LENGTH] = {0};
                int ret = recv(clientfd, buffer, BUFFER_LENGTH, 0);
                if (ret == 0) {
                        printf("close clientfd: %d\r\n",clientfd);
                        close(clientfd);
                        break;
                }

                printf("buffer : %s, ret: %d clientfd: %d\r\n", buffer, ret,clientfd);

                ret = send(clientfd, buffer, ret, 0); //
        }
}

int main(){
        int listenfd = socket(AF_INET, SOCK_STREAM, 0);  //
        if (listenfd == -1) return -1;

        struct sockaddr_in servaddr;
        servaddr.sin_family = AF_INET;
        servaddr.sin_addr.s_addr = htonl(INADDR_ANY);
        servaddr.sin_port = htons(9999);

        if (-1 == bind(listenfd, (struct sockaddr*)&servaddr, sizeof(servaddr))) {
                return -2;
        }

        listen(listenfd, 10);
        while (1) {
                struct sockaddr_in client;
                socklen_t len = sizeof(client);
                int clientfd = accept(listenfd, (struct sockaddr*)&client, &len);

                pthread_t threadid;
                pthread_create(&threadid, NULL, routine, &clientfd);

        }
        return 0;
}

Program running diagram:
Please add a picture description

Applied to selectthe model to handle multiple socketconnections

The result of using multi-threaded loop processing socketseems to be able to satisfy the one-to-many idea, but it still has some problems.
For example, one thread corresponds to one socketrequest, resulting in a large consumption of server resources.
Frequent creation and destruction of threads is also a big overhead for the system. Of course, we can use the thread pool to avoid frequent creation and destruction of threads,
but the most critical The point is that the multithreading model doesn't solve C10kthe problem yet.

The difference between multi-threaded loop processing socketand multi-threaded loop processing selectis selcetthat threads are multiplexed, which is also I/Oan advantage of the multiplexing model.
The system does not need to frequently create and destroy threads. One thread manages everything socket.
Its advantage over the multi-threaded model is that it simultaneously Can handle more socketconnections:

#include <stdio.h>
#include <sys/socket.h>
#include <sys/types.h>
#include <netinet/in.h>
#include <fcntl.h>
#include <unistd.h>
#include <pthread.h>
#include <string.h>

#define BUFFER_LENGTH 128

int main() {

	int listenfd = socket(AF_INET, SOCK_STREAM, 0);  
	if (listenfd == -1) return -1;

	struct sockaddr_in servaddr;
	servaddr.sin_family = AF_INET;
	servaddr.sin_addr.s_addr = htonl(INADDR_ANY);
	servaddr.sin_port = htons(9999);

	if (-1 == bind(listenfd, (struct sockaddr*)&servaddr, sizeof(servaddr))) {
		return -2;
	}

	listen(listenfd, 10);

	fd_set rfds, wfds, rset, wset;

	FD_ZERO(&rfds);
	FD_SET(listenfd, &rfds);
	FD_ZERO(&wfds);

	int maxfd = listenfd;

	unsigned char buffer[BUFFER_LENGTH] = {0}; // 0 
	int ret = 0;

	while (1) {
		/* 每次调用之前都要初始化fd_set结构体 */
		rset = rfds;
		wset = wfds;

		int nready = select(maxfd+1, &rset, &wset, NULL, NULL);
		if (FD_ISSET(listenfd, &rset)) {
			printf("listenfd --> %d\r\n"，listenfd);

			struct sockaddr_in client;
			socklen_t len = sizeof(client);
			int clientfd = accept(listenfd, (struct sockaddr*)&client, &len);
			
			FD_SET(clientfd, &rfds);

			if (clientfd > maxfd) maxfd = clientfd;
		} 
		
		int i = 0;
		for (i = listenfd+1; i <= maxfd;i ++) {
			if (FD_ISSET(i, &rset)) { //
				memset(buffer, 0, sizeof(buffer));	
				ret = recv(i, buffer, BUFFER_LENGTH, 0);
				if (ret == 0) {
                    printf("close clientfd: %d\r\n",i);
					close(i);
					FD_CLR(i, &rfds);
				} else if (ret > 0) {
					printf("buffer : %s, ret: %d, clientfd: %d\r\n", buffer, ret,i);
					FD_SET(i, &wfds);
				}

				
			} else if (FD_ISSET(i, &wset)) {
				ret = send(i, buffer, ret, 0); // 
				FD_CLR(i, &wfds); //
				FD_SET(i, &rfds);
			}
		}
	}
	return 0;
}

Program running diagram:
Please add a picture description

Use pollmodels to handle multiple socketconnections

The advantages of the above selectmodel are obvious, multiplexing is realized I/O, but there are also many problems, such as the upper limit on the number of monitoring file handles; the structure
must be reinitialized before each call ; it is necessary to traverse the array to monitor state, there is unnecessary consumption.fd_setfd

pollThe model mainly solves the first two problems. It modifies the three structure parameters selectit uses into an array of types. The principle is the same as the model, but theoretically solves the upper limit of the number of file handles and avoids repeated initialization. The problem is that the efficiency of traversing the array monitoring state still exists.fd_setstruct pollfd *selectfd

#include <stdio.h>
#include <sys/socket.h>
#include <sys/types.h>
#include <arpa/inet.h>
#include <unistd.h>
#include <fcntl.h>
#include <poll.h>
#include <string.h>

#define MAX_FD  1024
#define BUFFER_LENGTH 128

int main(){
    //创建一个侦听socket
	int listenfd = socket(AF_INET, SOCK_STREAM, 0);  
	if (listenfd == -1) return -1;
	
	//默认使用阻塞模式
	struct sockaddr_in servaddr;
	servaddr.sin_family = AF_INET;
	servaddr.sin_addr.s_addr = htonl(INADDR_ANY);
	servaddr.sin_port = htons(9999);

	if (-1 == bind(listenfd, (struct sockaddr*)&servaddr, sizeof(servaddr))) {
        printf("bind error.");
		return -2;
	}
    //启动侦听
    if (-1 == listen(listenfd, SOMAXCONN)){
        printf("listen error.");
        close(listenfd);
        return -3;
    }
	
	struct pollfd  fds[MAX_FD] = {0};
    fds[0].fd = listenfd;
    fds[0].events = POLLIN;	
	
	int cur_max_fd = listenfd;
	/* 初始化poll数组 */
	int i = 0;
	for(i=1; i < MAX_FD; i++){
		fds[i].fd = -1;
	}

	unsigned char buffer[BUFFER_LENGTH] = {0}; // 0 
	int ret = 0;
	
	while(1){
		int nready = poll(fds, cur_max_fd+1, -1);
		if (nready < 0){
            perror("poll error\r\n");
            return -4;
        }
		
		if(fds[0].revents & POLLIN){
			struct sockaddr_in client;
			socklen_t len = sizeof(client);
			int clientfd = accept(listenfd, (struct sockaddr*)&client, &len);
			fds[clientfd].fd = clientfd;//将客户端socket加入到集合中
            fds[clientfd].events = POLLIN;
			
			printf("listen --> \r\n");
			if (clientfd > cur_max_fd) {
				cur_max_fd = clientfd;
			}
			if (--nready == 0) continue;
		}
		
		for(i=listenfd+1; i<= cur_max_fd; i++){
			if(fds[i].revents & POLLIN){
				ret = recv(i, buffer, BUFFER_LENGTH, 0);
				if(ret > 0){
					buffer[ret] = '\0';
					printf("buffer : %s, ret: %d, clientfd: %d\r\n", buffer, ret,i);

					send(i, buffer, ret, 0);
				}else if(ret == 0){
					fds[i].fd = -1;
                    printf("close clientfd: %d\r\n",i);
		            close(i);
				}
				if (--nready == 0) break;
			}
		}
	}
	close(listenfd);
	return 0;
}

epoll

epollThe data structure of the red-black tree is used to monitor all fdstate changes. It only returns the state changes fd,
instead of select/polltraversing the entire fdarray by polling and scanning.
It uses three key pairs apito fdmanage, first use to epoll_createcreate an epollobject epfd, then
add the ones that need to be monitored to through, and finally call to wait for data. When the number of client requests reaches , the performance is superior , and this design solves the problem:epoll_ctlsocketepfdepoll_wait
fd10kepollselect/pollC10k

#include <stdio.h>
#include <sys/socket.h>
#include <sys/types.h>
#include <arpa/inet.h>
#include <unistd.h>
#include <fcntl.h>
#include <sys/epoll.h>
#include <string.h>

#define MAX_FD  1024
#define BUFFER_LENGTH 128

int main(){
    //创建一个侦听socket
	int listenfd = socket(AF_INET, SOCK_STREAM, 0);
	if(-1 == listenfd) return -1;
	
	//默认使用阻塞模式
	struct sockaddr_in servaddr;
	servaddr.sin_family = AF_INET;
	servaddr.sin_addr.s_addr = htonl(INADDR_ANY);
	servaddr.sin_port = htons(9999);

	if (-1 == bind(listenfd, (struct sockaddr*)&servaddr, sizeof(servaddr))) {
        printf("bind error.");
		return -2;
	}
    //启动侦听
    if (-1 == listen(listenfd, SOMAXCONN)){
        printf("listen error.");
        close(listenfd);
        return -3;
    }
	
	int epfd = epoll_create(1); //int size

	struct epoll_event events[MAX_FD] = {0};
	struct epoll_event ev;

	ev.events = EPOLLIN;
	ev.data.fd = listenfd;

	epoll_ctl(epfd, EPOLL_CTL_ADD, listenfd, &ev);

	unsigned char buffer[BUFFER_LENGTH] = {0}; // 0 
	int ret = 0;
	while (1) {
		int nready = epoll_wait(epfd, events, MAX_FD, 5);
		if (nready == -1) continue;

		int i = 0;
		for (i = 0;i < nready;i ++) {
			int clientfd =  events[i].data.fd;
			if (clientfd == listenfd) {

				struct sockaddr_in client;
			    socklen_t len = sizeof(client);
				int connfd = accept(listenfd, (struct sockaddr *)&client, &len);

				printf("listen --> \r\n");
				ev.events = EPOLLIN;
				ev.data.fd = connfd;
				epoll_ctl(epfd, EPOLL_CTL_ADD, connfd, &ev);

			} else if (events[i].events & EPOLLIN) {

				ret = recv(clientfd, buffer, BUFFER_LENGTH, 0);
		        if (ret > 0) {
		            buffer[ret] = '\0';
		            //printf("recv msg from client: %s\n", buffer);
					printf("buffer : %s, ret: %d, clientfd: %d\r\n", buffer, ret,clientfd);

					send(clientfd, buffer, ret, 0);
		        } else if (ret == 0) { //
					ev.events = EPOLLIN;
					ev.data.fd = clientfd;

					epoll_ctl(epfd, EPOLL_CTL_DEL, clientfd, &ev);
                    printf("close clientfd: %d\r\n",clientfd);

		            close(clientfd);
		        }
			}
		}
	}
    close(listenfd);
	return 0;
}

reference list

"In-depth understanding of computer systems" - the third part of the exchange and communication between programs
In-depth understanding of epoll for LinuxIO multiplexing
In-depth understanding of the epoll model (especially detailed)
"Graphic System"-9. Network system-Kobayashi coding
"High-performance network programming"- Instant Messenger Blog