1. Previous articles

[High concurrent network communication architecture] 1. The tcp server for single client connection under Linux

[High concurrency network communication architecture] 2. Introduce multi-threading to realize the tcp server of multi-client connection

[High concurrency network communication architecture] 3. Introduce IO multiplexing (select, poll, epoll) to achieve high concurrency tcp server

2. Basic concepts

1 Introduction

In computer science, Reactor (interpretation of "reactor") is a software design pattern, a commonly used event-driven programming model, often used to build high-performance concurrent applications. It describes a design pattern for handling concurrent events, which is realized through event loop and callback mechanism, and is widely used in network programming, server development and other fields.

The Reactor model itself is a pattern or idea, not a specific code framework or library. Different programming languages and platforms can use the Reactor model to implement event-driven applications, such as Node.js, Twisted, Netty, etc., are based on the Reactor model to build high-performance network applications.

In the Reactor model, event-driven is the main programming paradigm to listen to events and call corresponding callback functions accordingly to handle events.

2. Basic framework

Event Source (Event Source) : The event source is the object or component that generates the event. It could be a network socket, a file descriptor, a user input device, etc. The event source is responsible for monitoring and detecting the occurrence of events, and notifying the event dispatcher when the event occurs.

Event Loop : The event loop is a core component of the Reactor model. It is a loop structure responsible for monitoring the occurrence of events and dispatching corresponding event handlers to handle events. The event loop is constantly waiting for the occurrence of an event. Once an event occurs, it will hand over the event to the event dispatcher, and then call the corresponding event handler for processing. The event loop is responsible for managing the order and execution of events and ensuring that the processor does not get blocked.

Event Dispatcher : The event dispatcher is responsible for distributing the events emitted by the event source to the appropriate event handlers. It receives events and, based on the type of event and other rules, chooses the best event handler for that type of event. The event dispatcher functions as an event route to ensure that the event is correctly delivered to the corresponding processor.

Event Handler : An event handler is an object that encapsulates the processing logic of a specific type of event. Each event handler defines how to handle a certain type of event. They encapsulate how to handle events and perform corresponding operations, such as reading network data, writing files, etc.

Multiplexer (Multiplexer) : A multiplexer is used to manage and listen to multiple event sources and notify the event loop when an event occurs. It can achieve efficient event distribution based on the underlying mechanisms provided by the operating system (such as select, poll, epoll).

Reactor (Reactor) : The reactor is an integral part of the event loop, responsible for coordinating the registration and de-registration, distribution and undistribution of events. It can manage multiple event sources and event processors, and provide a unified interface and scheduling mechanism.

Event Queue (Event Queue) : The event queue is used to store and manage pending events. It can be a first-in-first-out (FIFO) queue or other data structure used to ensure that events enter the event loop for processing in order.

Summarize

These components work together to form the basic framework of the Reactor model. They decouple the generation and processing of events, realize efficient event-driven processing in an asynchronous and non-blocking manner, and improve the performance and concurrency of applications.

It should be noted that although the Reactor mode is a basic concurrent programming mode, there may be different variants and improvements depending on the specific implementation and platform, such as multi-threaded Reactor, asynchronous Reactor, etc. Different variants can be selected and optimized according to the needs of the application and the characteristics of the system.

3. Core features

Event-driven : The Reactor pattern is an event-driven design pattern that drives the behavior of the application by listening to the occurrence of events. When an event occurs, Reactor will distribute the event to the corresponding processor for processing according to the event type. This event-driven mechanism enables applications to respond to external events and take appropriate actions according to the type of event.

Non-blocking : The Reactor pattern handles events in a non-blocking manner. The generation and processing of events are performed concurrently, and the event loop and event handler are executed in a non-blocking manner. This means that while one event is being processed, the processing of other events will not be blocked, thereby improving the concurrency performance of the application.

Multiplexing : The Reactor mode uses the multiplexing mechanism (select, poll, epoll, etc.) provided by the operating system to allow an event loop to listen to multiple event sources at the same time. Through this mechanism, the event loop can process multiple events at the same time, and only wake up the corresponding event handler for processing when an event occurs, avoiding the low efficiency of polling.

Scalability : The design of the Reactor pattern makes the application scalable. For different types of events, corresponding event handlers can be created and registered in the event dispatcher. This enables better adaptation to changing workloads by dynamically increasing or decreasing the number of event processors based on the needs of the application.

Flexibility : The Reactor pattern is designed to make applications more flexible and extensible. The generation and processing of events are decoupled. To add a new type of event, you only need to create a corresponding event handler and register it, without modifying the event loop. This flexibility allows applications to easily adapt to new requirements and changes.

High performance : Due to the non-blocking and concurrent processing characteristics of the Reactor pattern, it enables high-performance event processing. The application will not block the processing of other events because of the processing of one event, thus improving the overall performance and throughput.

Summarize

The Reactor pattern is event-driven, non-blocking, multiplexing, scalable, flexible, and high-performance. It can provide an efficient event processing mechanism, enabling applications to process multiple events concurrently, and has good maintainability and scalability.

4. Workflow

Initialization: Create an event loop (Event Loop) and an event distributor (Event Dispatcher). The event loop is a main loop that is responsible for waiting for the arrival of events and dispatching corresponding event handlers to handle the events. The event dispatcher is responsible for passing events from the event loop to the appropriate event handlers.

Register event source and event handler: Register the event source (Event Source) and the corresponding event handler (Event Handler) in the event dispatcher, and establish the association between the event source and the event handler. During registration, each event source and event handler is assigned a unique identifier for subsequent event distribution.

Start the event loop: Start the event loop and enter the event listening state. The event loop starts to wait for the event to occur, and registers the corresponding event handler according to the type of event listened to by the event source.

Waiting for events to occur: The event loop waits for events to occur through the event dispatcher. This can be achieved through blocking methods of event sourcing, non-blocking methods, polling, etc. The event loop waits until at least one event source is notified.

Event notification: When an event occurs from one or more event sources, they notify the event loop and pass the event from the event source to the event loop.

Event distribution: After the event loop receives the event notification, it passes the event to the event dispatcher.

Routing to an event handler: The event dispatcher selects an appropriate event handler to handle the event according to the type of the event and other rules. It looks up the mapping between event sources and handlers based on the event identifier, and passes the event to the matching event handler.

Executing event handlers: Event handlers perform corresponding actions to handle events. This can include reading event data, doing calculations, updating state, etc. Execution of event handlers is usually non-blocking in order to allow the event loop to continue listening for other events.

Return to waiting state: When the event handler finishes processing the event, it returns to the event loop and continues to wait for the next event to occur.

Repeated execution: The event loop repeats the above steps continuously, monitoring and processing events, until the stop condition is met. A stop condition can be an expiration of a wait time, occurrence of a specific event, satisfaction of a specific state, or other conditions defined by the application.

5. Use "network communication" to understand the Reactor model

Suppose you are developing a chat application through which users can communicate with other users in real time. To handle concurrent network connections, you can use the Reactor pattern.

First, you create an event loop (Event Loop) as the main loop and initialize a network socket to listen for incoming connection requests.

When there is a new client connection request, the network socket will fire an event to notify your event loop of the connection request.

Next, you create a connection handler (Connection Handler), which is responsible for handling each client connection. Connection handlers encapsulate the logic for handling connections, such as authentication, receiving and sending messages, and so on.

You register the connection handler to the event dispatcher (Event Dispatcher), and establish the mapping relationship between the socket and the connection handler. In this way, when there is a new connection request, the event dispatcher can pass the connection event to the corresponding connection handler.

After the event loop starts, it waits for a connection request to occur. When a client connects, the socket fires a connect event, which notifies the event loop.

After the event loop receives the connection event, it passes the connection event to the event dispatcher.

The event dispatcher selects an appropriate connection handler to handle the connection based on the type of connection event and other rules. It finds the corresponding connection handler based on the client's identity, request type, etc.

The connection handler performs the corresponding operations, such as authentication, handling the receiving and sending of messages, and so on. These operations are non-blocking, so other client connections can be processed concurrently.

After the processing is completed, the connection handler returns to the event loop and continues to wait for the next connection request.

The event loop continuously monitors connection events, and distributes connection events to corresponding connection handlers for processing. This decoupling design enables you to handle multiple client connections and communications concurrently, improving the responsiveness and scalability of your application.

Summarize

To sum up, Reactor mode has the characteristics of event-driven, non-blocking, multiplexing, scalability, flexibility and high performance. It can provide an efficient event processing mechanism, enabling applications to process multiple events concurrently, and has good maintainability and scalability.

Three, code implementation

1. Use epoll multiplexing to realize the operation process of Reactor mode

Initialize the server: call the init_server function, create a server socket, and perform the following operations on the interface:

Create a listening socket: use socketthe function to create a listening socket, specify the protocol family, socket type and protocol number. For example, socket(AF_INET, SOCK_STREAM, 0)a TCP socket is created.

Bind address and port: Use binda function to bind a socket to a specified IP address and port number. It is necessary to create a sockaddr_instructure and set the corresponding address type, IP address and port number. Bind the socket with the desired address bindby calling the function with the socket descriptor and pointer to the structure as parameters.sockaddr_in

Listen for Connections: Use listenthe function to start listening on a socket. The socket descriptor and the maximum number of waiting connections need to be passed in. This will set the socket into a waiting state and can accept new client connections.

Initialize the event loop mechanism: create an epollinstance, use epoll_createa function to create an epoll instance. The returned file descriptor can be used for subsequent epoll operations.

Create an event handler and add it to the event loop: Create a EventHandlerstructure that contains the file descriptors that need to be monitored and the corresponding event handlers. Using this structure as a parameter, call reactor_add_handlerthe function to add the event handler to the event loop.

Start the event loop: Call reactor_runthe function to start the event loop. In the event loop, the following actions are performed:

Use epoll_waitfunction blocking to wait for the ready event to occur. This function receives epolla file descriptor, an array for getting events, and the length of the array.

Once there is a ready event, traverse the event array and find the corresponding event handler.

Call the processing function of the event handler to handle the event.

Handle connection events: When the listening socket has a new connection, a connection event will be triggered. In the connection event processing function, use acceptthe function to accept the client connection and create a new socket for communicating with the client. Then, create a new event handler and add the socket to the event loop.

Handle read events: When there is data to read, a read event is triggered. In the read event processing function, use readthe function to read the data and process it. If the data is read, corresponding operations can be performed on the data. If the read returns 0, the connection is closed and the socket needs to be removed from the event loop.

Remove the event handler from the event loop: When the connection is closed or an error occurs, the event handler needs to be removed from the event loop. By calling reactor_remove_handlerthe function, the socket is epollremoved from the instance and the corresponding event handler is removed from the array.

Cleanup operations: After the event loop ends, the listening socket and epollinstance are closed, and any other resources are released.

2. Reactor mode implementation code (reference)

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <unistd.h>
#include <errno.h>
#include <sys/epoll.h>

#define MAX_EVENTS 10
#define BUFFER_SIZE 1024

// 定义事件处理器的结构体
typedef struct {
    int fd;                          // 文件描述符
    void (*handler)(int);            // 事件处理函数的指针
} EventHandler;

// 定义事件循环机制以及事件处理器的存储
int epoll_fd;                        // epoll 实例的文件描述符
EventHandler event_handlers[MAX_EVENTS];    // 事件处理器数组
int num_handlers = 0;                // 事件处理器数组中的处理器数量

// 初始化服务端套接字
int init_server(int port);

// 处理连接事件
void handle_accept(int listen_fd);

// 处理读事件
void handle_read(int client_fd);

// 初始化事件循环机制
void reactor_init();

// 添加事件处理器到事件循环中
void reactor_add_handler(int fd, EventHandler event_handler);

// 从事件循环中移除事件处理器
void reactor_remove_handler(int fd);

// 启动事件循环
void reactor_run();

int main(int argc,char *argv[]){

    if(argc < 2)return -1;

    int port = atoi(argv[1]);

    int listen_fd = init_server(port);
    if(listen_fd == -1)return -1;

    // 初始化事件循环机制
    reactor_init();

    // 创建一个事件处理器并添加到事件循环中
    EventHandler event_handler;
    event_handler.fd = listen_fd;
    event_handler.handler = handle_accept;
    reactor_add_handler(listen_fd, event_handler);

    // 启动事件循环
    reactor_run();

    close(listen_fd);

    return 0;
}

// 初始化服务端套接字
int init_server(int port){
    //获取服务端fd,通常为3,前面0,1,2用于指定输入,输出,错误值
    int listen_fd = socket(AF_INET,SOCK_STREAM,0);

    if(-1 == listen_fd){
        printf("Socket error code: %d codeInfo: %s\n", errno, strerror(errno));
        return -1;
    }

    //设置服务端套接字为非阻塞模式
    // int flags = fcntl(sfd,F_GETFL,0);
    // fcntl(sfd,F_SETFL,flags | O_NONBLOCK);

    struct sockaddr_in server_addr;
    memset(&server_addr,0,sizeof(struct sockaddr_in));
    server_addr.sin_family = AF_INET;  //ipv4
    server_addr.sin_addr.s_addr = htonl(INADDR_ANY);   //0.0.0.0
    server_addr.sin_port = htons(port);
    
    //绑定IP和端口号
    if(-1 == bind(listen_fd,(struct sockaddr*)&server_addr,sizeof(struct sockaddr_in)))
    {
        printf("Bind error code: %d codeInfo: %s\n",errno,strerror(errno));
        return -1;
    }

    //监听该套接字上的连接
    if(-1 == listen(listen_fd,SOMAXCONN))
    {
        printf("Listen error code: %d codeInfo: %s\n",errno,strerror(errno));
        return -1;
    }

    printf("Socket init successed: server fd = %d\n",listen_fd);
    return listen_fd;
}

// 处理连接事件
void handle_accept(int listen_fd) {
    struct sockaddr_in client_addr;
    socklen_t addr_len = sizeof(struct sockaddr_in);
    int client_fd = accept(listen_fd, (struct sockaddr *)&client_addr, &addr_len);
    printf("Accepted new connection: client fd = %d\n",client_fd);

    // 添加读事件到事件循环中
    EventHandler event_handler;
    event_handler.fd = client_fd;
    event_handler.handler = handle_read;
    reactor_add_handler(client_fd, event_handler);
}

// 处理读事件
void handle_read(int client_fd) {
    char buffer[BUFFER_SIZE] = {0};
    ssize_t bytes_read = read(client_fd, buffer, BUFFER_SIZE);
    if (bytes_read > 0) {
        printf("Received client fd=%d DataLen: %d Data: %s\n", client_fd, (int)bytes_read, buffer);
    } else if (bytes_read == 0) {
        printf("Connection closed: client fd = %d\n",client_fd);
        // 关闭连接并从事件循环中移除
        reactor_remove_handler(client_fd);
        close(client_fd);
    } else {
        perror("Read error");
        // 关闭连接并从事件循环中移除
        reactor_remove_handler(client_fd);
        close(client_fd);
    }
}

// 初始化事件循环机制
void reactor_init() {
    epoll_fd = epoll_create(1);
    if (epoll_fd < 0) {
        perror("Epoll creation failed");
        exit(1);
    }
    printf("Create epoll successed: epoll fd = %d\n",epoll_fd);
}

// 添加事件处理器到事件循环中
void reactor_add_handler(int fd, EventHandler event_handler) {
    struct epoll_event event;
    event.events = EPOLLIN;          // 只监听读事件
    event.data.fd = fd;
    // 将文件描述符添加到 epoll 实例中进行事件监听
    int ret = epoll_ctl(epoll_fd, EPOLL_CTL_ADD, fd, &event);
    if (ret < 0) {
        perror("Epoll control failed");
        exit(1);
    }
    // 将事件处理器添加到数组中
    event_handlers[num_handlers++] = event_handler;
}

// 从事件循环中移除事件处理器
void reactor_remove_handler(int fd) {
    // 从 epoll 实例中移除文件描述符
    int ret = epoll_ctl(epoll_fd, EPOLL_CTL_DEL, fd, NULL);
    if (ret < 0) {
        perror("Epoll control failed");
        exit(1);
    }
    // 从数组中移除事件处理器
    for (int i = 0; i < num_handlers; i++) {
        if (event_handlers[i].fd == fd) {
            event_handlers[i] = event_handlers[--num_handlers];
            break;
        }
    }
}

// 启动事件循环
void reactor_run() {
    struct epoll_event events[MAX_EVENTS];
    while (1) {
        // 等待事件发生
        int num_ready = epoll_wait(epoll_fd, events, MAX_EVENTS, -1);
        if (num_ready < 0) {
            perror("Epoll wait failed");
            break;
        }

        // 遍历就绪事件，并调用相应的事件处理函数
        for (int i = 0; i < num_ready; i++) {
            int fd = events[i].data.fd;
            // 查找对应的事件处理器并调用其处理函数
            for (int j = 0; j < num_handlers; j++) {
                if (event_handlers[j].fd == fd) {
                    event_handlers[j].handler(fd);
                    break;
                }
            }
        }
    }
    close(epoll_fd);    //清理操作
}

[High concurrent network communication architecture] 4. Efficient event-driven model: Reactor model

1. Previous articles

2. Basic concepts

1 Introduction

2. Basic framework

3. Core features

4. Workflow

5. Use "network communication" to understand the Reactor model

Three, code implementation

1. Use `epoll` multiplexing to realize the operation process of Reactor mode

2. Reactor mode implementation code (reference)

Guess you like

[High concurrent network communication architecture] 4. Efficient event-driven model: Reactor model

1. Previous articles

2. Basic concepts

1 Introduction

2. Basic framework

3. Core features

4. Workflow

5. Use "network communication" to understand the Reactor model

Three, code implementation

1. Use epoll multiplexing to realize the operation process of Reactor mode

2. Reactor mode implementation code (reference)

Guess you like

1. Use `epoll` multiplexing to realize the operation process of Reactor mode