Foreword
Sockets are widely used and very important in actual system program development. In practical applications, servers often need to support multiple client connections, so it is particularly important to implement a high-concurrency server model. The high-concurrency server has evolved from a simple loop server model to handle a small number of concurrent network requests to a high-concurrency server model that solves C10K and C10M problems. This article uses a simple multi-threading model to lead everyone to learn how to implement a simple concurrent server by yourself.
C/S architecture
Server-client, that is, Client-Server (C/S) structure. The C/S structure usually adopts a two-layer structure. The server is responsible for data management, and the client is responsible for completing interactive tasks with users.
In the C/S structure, the application program is divided into two parts: the server part and the client part. The server part is the information and functions shared by multiple users, and performs background services, such as controlling the operation of shared databases; the client part is exclusive to users and is responsible for executing foreground functions and has powerful functions in terms of error prompts and online help. function, and can switch freely between subroutines.
As shown in the figure above: This is the function calling relationship that connects the client and the server based on sockets. There is a lot of information on the socket API, so this article will not describe it in detail.
pthread thread library: (POSIX)
The pthread thread library is a commonly used thread library under Linux. You can search for relevant articles about its usage and features. Below is a brief introduction to its usage and compilation.
Thread ID
Threads have IDs, but they are not unique to the system, but are only valid in the process environment. The handle of the thread is of type pthread_t, which cannot be processed as an integer, but is a structure. Two functions are introduced below:
头文件: <pthread.h>
原型: int pthread_equal(pthread_t tid1, pthread_t tid2);
返回值: 相等返回非0, 不相等返回0.
说明: 比较两个线程ID是否相等.
头文件: <pthread.h>
原型: pthread_t pthread_self();
返回值: 返回调用线程的线程ID.
Thread creation
Create a thread in execution, you can assign the thread the work it needs to do (thread execution function), the thread shares the resources of the process. The function pthread_create() that creates the thread
头文件: <pthread.h>
原型: int pthread_create(pthread_t *restrict tidp, const pthread_attr_t *restrict attr, void *(start_rtn)(void), void *restrict arg);
返回值: 成功则返回0, 否则返回错误编号.
参数:
tidp: 指向新创建线程ID的变量, 作为函数的输出.
attr: 用于定制各种不同的线程属性, NULL为默认属性（见下）.
start_rtn: 函数指针, 为线程开始执行的函数名.该函数可以返回一个void *类型的返回值，
而这个返回值也可以是其他类型，并由 pthread_join()获取
arg: 函数的唯一无类型(void)指针参数, 如要传多个参数, 可以用结构封装.
compile
因为pthread的库不是linux系统的库，所以在进行编译的时候要加上 -lpthread
# gcc filename -lpthread //默认情况下gcc使用c库，要使用额外的库要这样选择使用的库
Common web server models
Based on my own understanding, this article mainly uses TCP as an example to summarize the implementation methods of several common network server models, and finally implements a simple command line chat room.
single process loop
The principle of single-line process loop is that the main process does not communicate with the client. The client must first connect to the server. The server accepts a client connection and reads data from the client, then processes and returns the processing result to the client, and then accepts it. The next client's connection request.
Advantages The advantage of the single-threaded loop model is that it is simple and easy to implement, without the troubles and overheads of synchronization and locking.
shortcoming
-
Blocking model, serial processing of network requests;
-
It does not take advantage of multi-core CPUs and network requests are processed serially;
-
Unable to support multiple client connections at the same time;
-
The program operates serially, and the server cannot send and receive data at the same time.
Single thread IO reuse
epoll is commonly used as an IO multiplexing mechanism in Linux high-concurrency servers. The thread registers all the socket read and write events that need to be processed into epoll. When network IO occurs, epoll_wait returns, and the thread checks and processes the request on the socket.
advantage
-
The implementation is simple, reducing lock overhead and thread switching overhead.
shortcoming
-
Only a single-core CPU can be used. If the handle time is too long, the entire service will hang;
-
When the number of clients exceeds a certain number, performance will drop significantly;
-
It is only suitable for scenarios with high IO, low calculation, and short handle processing time.
Multithreading/Multiple processes
The main feature of the multi-thread and multi-process model is that each network request is processed by a process/thread, and blocking system calls are used inside the thread. In terms of the division of thread functions, a separate thread can handle the accept connection, and the remaining threads handle specific Network requests (packet collection, processing, packet sending); multiple processes can also listen and accept network connections individually.
advantage:
1. The implementation is relatively simple; 2. It utilizes CPU multi-core resources.
shortcoming:
1. The thread is still blocked internally. To take an extreme example, if a thread sleeps in the business logic of the handle, the thread will also hang.
Multi-thread/multi-process IO multiplexing
Multi-threaded, multi-process IO taking model, each child process listens to the service, and uses the epoll mechanism to handle the network request of the process. After the child process accept(), it will create a connected descriptor, and then communicate with the client through the connected descriptor. terminal communication. This mechanism is suitable for high concurrency scenarios.
advantage:
-
Support higher concurrency.
shortcoming:
-
Asynchronous programming is unintuitive and error-prone
Multi-threading divides IO roles
The main functions of multi-threaded IO roles are: an accept thread handles the establishment of new connections; an IO thread pool handles network IO; and a handle thread pool handles business logic. Usage scenarios such as: telemarketing applications, thrift TThreadedSelectorServer.
advantage:
-
Divide threads according to different functions, and each thread handles fixed functions, which is more efficient
-
The number of threads can be configured according to business characteristics to optimize performance.
shortcoming:
-
Inter-thread communication requires the introduction of lock overhead
-
The logic is complex and difficult to implement
summary
The common network server models are introduced above, as well as AIO, coroutines, and even other variations, which will not be discussed here. The important thing is to understand the problems faced in each scenario and the characteristics of each model, and to design a solution that fits the application scenario is a good solution.
Multi-threaded concurrent server model
Below we mainly discuss the multi-threaded concurrent server model.
Code structure
The concurrent server code structure is as follows:
thread_func()
{
while(1) {
recv(...);
process(...);
send(...);
}
close(...);
}
main(
socket(...);
bind(...);
listen(...);
while(1) {
accept(...);
pthread_create();
}
}
As can be seen from the above, the server is divided into two parts: main thread and sub-thread.
main thread
The main function is the main thread, and its main tasks are as follows:
-
socket() creates a listening socket;
-
bind() binds port number and address;
-
listen() turns on listening;
-
accept() waits for the client's connection,
-
When a client connects, accept() will create a new socket new_fd;
-
The main thread will create a child thread and pass new_fd to the child thread.
child thread
-
The sub-thread function is thread_func(), which handles all communication tasks with the client through new_fd.
Detailed steps for client to connect to server
Below we look at the step-by-step instructions for the client to connect to the server.
1. Client connects to server
-
The server establishes the listening socket listen_fd and initializes it;
-
The client creates socket fd1;
-
Client client1 connects to the server's listen_fd through socket fd1;
2. The main thread creates sub-thread thread1
-
After the server receives the connection request from client1, the accpet function will return a new socket newfd1;
-
Later, the communication between server and client1 depends on newfd1, and the listening socket listen_fd will continue to monitor the connections of other clients;
-
The main thread creates a sub-thread thread1 through pthead_create() and passes newfd1 to thread1;
-
The communication between server and client1 depends on newfd1 and fd1 respectively.
-
In order to receive the information sent by the server in real time, client1 must also be able to read data from the keyboard. These two operations are blocking. When there is no data, the process will sleep, so a child thread read_thread must be created;
-
The main thread of client1 is responsible for reading data from the keyboard and sending it to the server, and the sub-thread read_thread is responsible for receiving information from the server.
3. client2 connects to the server
-
Client client2 creates socket fd2;
-
Connect the listen_fd of the server through the connect function;
4. The main thread creates sub-thread thread2
-
After the server receives the connection request from client2, the accpet function will return a new socket newfd2;
-
Later, the communication between server and client2 depends on newfd2, and the listening socket listen_fd will continue to monitor the connections of other clients;
-
The main thread creates a sub-thread thread2 through pthead_create() and passes newfd2 to thread2;
-
The communication between server and client1 depends on newfd2 and fd2 respectively.
-
Similarly, client2 must create a child thread read_thread in order to receive the information sent by the server in real time and read data from the keyboard;
-
The main thread of client1 is responsible for reading data from the keyboard and sending it to the server, and the sub-thread read_thread is responsible for receiving information from the server.
As can be seen from the above figure, after each client connects to the server, the server creates a special thread responsible for communicating with the client; each client and server have a fixed pair of fd combinations for connection.
Example
Okay, I’m done with the theory. According to Yiyijun’s practice, I also inherited the teachings of my ancestor: talk is cheap, show you my code. Articles that only write theory without coding are just hooligans.
The main functions of this example are described as follows:
-
Enable multiple clients to connect to the server at the same time;
-
The client can send and receive data independently;
-
After the client sends data to the server, the server will return the data intact to the client.
Service-Terminal
/*********************************************
服务器程序 TCPServer.c
*********************************************/
#include <stdio.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <arpa/inet.h>
#include <errno.h>
#include <string.h>
#include <pthread.h>
#include <stdlib.h>
#define RECVBUFSIZE 2048
void *rec_func(void *arg)
{
int sockfd,new_fd,nbytes;
char buffer[RECVBUFSIZE];
int i;
new_fd = *((int *) arg);
free(arg);
while(1)
{
if((nbytes=recv(new_fd,buffer, RECVBUFSIZE,0))==-1)
{
fprintf(stderr,"Read Error:%s\n",strerror(errno));
exit(1);
}
if(nbytes == -1)
{//客户端出错了 返回值-1
close(new_fd);
break;
}
if(nbytes == 0)
{//客户端主动断开连接，返回值是0
close(new_fd);
break;
}
buffer[nbytes]='\0';
printf("I have received:%s\n",buffer);
if(send(new_fd,buffer,strlen(buffer),0)==-1)
{
fprintf(stderr,"Write Error:%s\n",strerror(errno));
exit(1);
}
}
}
int main(int argc, char *argv[])
{
char buffer[RECVBUFSIZE];
int sockfd,new_fd,nbytes;
struct sockaddr_in server_addr;
struct sockaddr_in client_addr;
int sin_size,portnumber;
char hello[]="Hello! Socket communication world!\n";
pthread_t tid;
int *pconnsocke = NULL;
int ret,i;
if(argc!=2)
{
fprintf(stderr,"Usage:%s portnumber\a\n",argv[0]);
exit(1);
}
/*端口号不对，退出*/
if((portnumber=atoi(argv[1]))<0)
{
fprintf(stderr,"Usage:%s portnumber\a\n",argv[0]);
exit(1);
}
/*服务器端开始建立socket描述符 sockfd用于监听*/
if((sockfd=socket(AF_INET,SOCK_STREAM,0))==-1)
{
fprintf(stderr,"Socket error:%s\n\a",strerror(errno));
exit(1);
}
/*服务器端填充 sockaddr结构*/
bzero(&server_addr,sizeof(struct sockaddr_in));
server_addr.sin_family =AF_INET;
/*自动填充主机IP*/
server_addr.sin_addr.s_addr=htonl(INADDR_ANY);//自动获取网卡地址
server_addr.sin_port =htons(portnumber);
/*捆绑sockfd描述符*/
if(bind(sockfd,(struct sockaddr *)(&server_addr),sizeof(struct sockaddr))==-1)
{
fprintf(stderr,"Bind error:%s\n\a",strerror(errno));
exit(1);
}
/*监听sockfd描述符*/
if(listen(sockfd, 10)==-1)
{
fprintf(stderr,"Listen error:%s\n\a",strerror(errno));
exit(1);
}
while(1)
{
/*服务器阻塞，直到客户程序建立连接*/
sin_size=sizeof(struct sockaddr_in);
if((new_fd = accept(sockfd,(struct sockaddr *)&client_addr,&sin_size))==-1)
{
fprintf(stderr,"Accept error:%s\n\a",strerror(errno));
exit(1);
}
pconnsocke = (int *) malloc(sizeof(int));
*pconnsocke = new_fd;
ret = pthread_create(&tid, NULL, rec_func, (void *) pconnsocke);
if (ret < 0)
{
perror("pthread_create err");
return -1;
}
}
//close(sockfd);
exit(0);
}
client
/*********************************************
服务器程序 TCPServer.c
*********************************************/
#include <stdio.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <arpa/inet.h>
#include <errno.h>
#include <string.h>
#include <pthread.h>
#include <stdlib.h>
#define RECVBUFSIZE 1024
void *func(void *arg)
{
int sockfd,new_fd,nbytes;
char buffer[RECVBUFSIZE];
new_fd = *((int *) arg);
free(arg);
while(1)
{
if((nbytes=recv(new_fd,buffer, RECVBUFSIZE,0))==-1)
{
fprintf(stderr,"Read Error:%s\n",strerror(errno));
exit(1);
}
buffer[nbytes]='\0';
printf("I have received:%s\n",buffer);
}
}
int main(int argc, char *argv[])
{
int sockfd;
char buffer[RECVBUFSIZE];
struct sockaddr_in server_addr;
struct hostent *host;
int portnumber,nbytes;
pthread_t tid;
int *pconnsocke = NULL;
int ret;
//检测参数个数
if(argc!=3)
{
fprintf(stderr,"Usage:%s hostname portnumber\a\n",argv[0]);
exit(1);
}
//argv2 存放的是端口号 ，读取该端口，转换成整型变量
if((portnumber=atoi(argv[2]))<0)
{
fprintf(stderr,"Usage:%s hostname portnumber\a\n",argv[0]);
exit(1);
}
//创建一个 套接子
if((sockfd=socket(AF_INET,SOCK_STREAM,0))==-1)
{
fprintf(stderr,"Socket Error:%s\a\n",strerror(errno));
exit(1);
}
//填充结构体，ip和port必须是服务器的
bzero(&server_addr,sizeof(server_addr));
server_addr.sin_family=AF_INET;
server_addr.sin_port=htons(portnumber);
server_addr.sin_addr.s_addr = inet_addr(argv[1]);//argv【1】 是server ip地址
/*¿Í»§³ÌÐò·¢ÆðÁ¬œÓÇëÇó*/
if(connect(sockfd,(struct sockaddr *)(&server_addr),sizeof(struct sockaddr))==-1)
{
fprintf(stderr,"Connect Error:%s\a\n",strerror(errno));
exit(1);
}
//创建线程
pconnsocke = (int *) malloc(sizeof(int));
*pconnsocke = sockfd;
ret = pthread_create(&tid, NULL, func, (void *) pconnsocke);
if (ret < 0)
{
perror("pthread_create err");
return -1;
}
while(1)
{
#if 1
printf("input msg:");
scanf("%s",buffer);
if(send(sockfd,buffer,strlen(buffer),0)==-1)
{
fprintf(stderr,"Write Error:%s\n",strerror(errno));
exit(1);
}
#endif
}
close(sockfd);
exit(0);
}
To compile and compile threads, you need to use the pthread library. The compilation command is as follows:
-
gcc s.c -o s -lpthread
-
gcc cli.c -oc -lpthread test on this machine first
-
Open a terminal./s 8888
-
Open another terminal./cl 127.0.0.1 8888 and enter a string "qqqqqqq"
-
Open another terminal./cl 127.0.0.1 8888 and enter a string "yikoulinux"
Some readers may notice that the server uses the following code when creating a child thread:
pconnsocke = (int *) malloc(sizeof(int));
*pconnsocke = new_fd;
ret = pthread_create(&tid, NULL, rec_func, (void *) pconnsocke);
if (ret < 0)
{
perror("pthread_create err");
return -1;
}
Why do we need to malloc a piece of memory specifically to store this new socket?
This is a very subtle mistake that many novices make. In the next chapter, I will explain it to you specifically.