Linux five network IO modes (blocking IO, non-blocking IO, IO multiplexing, signal-driven IO, asynchronous IO)

In Linux network programming, there are five network IO modes, namely blocking IO, non-blocking IO, IO multiplexing, signal-driven IO, and asynchronous IO;

Although it is impossible to know all of them thoroughly, at least you must know a little bit of them all!

Before starting, understand the following synchronous IO and asynchronous IO;

1. Synchronous I/O

        Scenario 1 : Xiao Ming goes to open the water, but there is no water in the boiling water tower at this time, Xiao Ming waits for the boiling water to arrive on the spot, or keeps polling to see if there is boiling water, until there is boiling water to get the water, this is a case of synchronous IO !

        Features of synchronous IO:

                Synchronous IO refers to a user process triggering an I/O operation and waiting or polling to see if the I/O operation is ready.
                The executor of synchronous IO is the initiator of the IO operation.
                Synchronous IO requires the initiator to copy the data from the kernel state to the user state, so it must be blocked here.

2. Asynchronous I/O

        Scene 2 : Xiao Ming goes to turn on the water, but there is no water in the water tower at this time. The aunt of the water tower asks Xiao Ming to put the kettle on the spot. When the water comes, he will help him to fetch the water and call him to fetch it. This is asynchronous IO A case of !

        Features of asynchronous IO:

                Asynchronous IO means that the user process returns immediately after triggering the I/O operation, and continues to do its own thing, and when the I/O operation has been completed, it will be notified of the completion of the I/O; the executor of the asynchronous IO is the kernel
                thread , the kernel thread copies the data from the kernel state to the user state, so there is no blocking here.


Table of contents

1. Block IO

Two, non-blocking IO

Common ways to set non-blocking:

Set port multiplexing

3. IO multiplexing

1. select

2. poll

3. epool

1). epoll_create

2). epoll_ctl

3). epoll_wait

4). Example of echo server based on epoll

4. Horizontal trigger and edge trigger

5. Encapsulate the epoll framework

6. libevent

1). libevent installation

2). Libevent main API introduction

        A. event_base_new

        B. event_new

        C. event_add

        D. event_del

        E. event_base_dispatch

        F. event_base_free

        G. event_set

        H. event_assign

        I. evconnlistener_new_bind

        J. evconnlistener_free

        K. bufferevent_read

        L. bufferevent_write

        M. bufferevent_socket_new

        N. bufferevent_setcb

        O. bufferevent_enable

        P. bufferevent_disable

        Q. bufferevent_socket_connect

        R. bufferevent_get_input 、bufferevent_get_output

        S. Other

3). Echo server case one

4). Echo server case 2

5). Echo server case with caching

6). Client case with caching

7). Libevent monitoring signal case

4. Signal driven IO (not introduced)

5. Asynchronous IO (not introduced)

6. Summary


1. Block IO

Xiao Ming needed boiling water urgently. When he turned on the water, he found that there was no water in the faucet. He waited until it was filled with water and then left. This process can be seen as using the blocking IO model, because if there is no water in the faucet, he has to wait until there is water and the cup is filled before leaving to do other things. Obviously, this IO model is synchronous.

In linux, by default all sockets are blocking IO (blocking IO), a typical read operation process:


Two, non-blocking IO

Xiao Ming once again needed boiling water urgently. After turning on the faucet, he found that there was no water. Because he had other urgent matters, he left immediately. After a while, he took a cup to take a look... During the time of leaving in the middle, Xiao Ming left to pretend The water scene (back to the user process space), can do his own thing. This is the non-blocking IO model. But it is non-blocking only when it checks that there is no data. When the data arrives, it still has to wait to copy the data to the user space (waiting for the water to fill the water glass), so it is still synchronous IO.

When the user thread initiates a read operation, there is no need to wait, but a result is obtained immediately. If the result is an error, it knows the data is not ready yet, so it can send the read operation again. Once the data in the kernel is ready, and it receives the request from the user thread again, it immediately copies the data to the user thread, and then returns.

So in fact, in the non-blocking IO model, user threads need to constantly ask whether the kernel data is ready, that is to say, non-blocking IO will not hand over the CPU, but will always occupy the CPU.

A typical non-blocking IO model is generally as follows:

Common ways to set non-blocking:

Method 1: Specify when creating a socket

// SOCK_NONBLOCK: 非阻塞IO
server_sockfd = socket(AF_INET, SOCK_DGRAM | SOCK_NONBLOCK, 0);   

Method 2: Before reading data, set as follows

fcntl(server_sockfd, F_SETFL, fcntl(server_sockfd, F_GETFL, 0) | O_NONBLOCK);

Need to include the header file #include <fcntl.h>      // fcntl

In the interface for reading data, if there is no data to read, EAGAIN and EWOULDBLOCK will be returned immediately;

So you only need to judge the return value, such as:

int recv_len = recvfrom(server_sockfd, buff, sizeof(buff), 0, (struct sockaddr *)&client_addr, &client_addr_len);
if (recv_len < 0) {
    if (EAGAIN == errno || EWOULDBLOCK == errno) {
        printf("EAGAIN = %d   EWOULDBLOCK = %d   errno = %d\n", EAGAIN, EWOULDBLOCK, errno);
        printf("do something...\n");
        sleep(2);
        continue;
    }

    perror("recvfrom");
    exit(errno);
}

do something... That is, you can do other things, and then come back to see if there is data to read!

Set port multiplexing

int opt = 1;
setsockopt(fd, SOL_SOCKET, SO_REUSEADDR, &opt, sizeof(opt));


3. IO multiplexing

One day, the hot water supply was optimized in the school, and many faucets were added. At this time, Xiao Ming went to fill the water again, and the auntie told him that there was no water in these faucets. You can go to other things and wait until there is water. tell him. So wait and wait (select call), after a while the aunt told him that there is water.

There are two cases here:

        Situation 1 : Auntie only told him that the water was coming, but did not tell Xiao Ming which faucet the water was coming from. He had to try one by one by himself. (select/poll scene);

                That is, there are 1000 sockets, if one of the sockets has a message, the aunt will notify, let us take the traversal one by one, find the socket with data, and then read the data;

        Situation 2 : The dormitory manager will tell Xiao Ming which faucets have water, and Xiao Ming does not need to open them one by one (epoll scene);

                That is, there are 1000 sockets, if one of the sockets has a message, the aunt will notify a specific socket of the message, and we don’t need to traverse one by one, and the efficiency will increase at once;

When the user process calls select, the entire process will be blocked (blocked), and at the same time, the kernel will "monitor" all the sockets that select is responsible for. When the data in any socket is ready, select will return. At this time, the user process calls the read operation again to copy the data from the kernel (kernel) to the user process.

Therefore, the characteristic of IO multiplexing is that through a mechanism, a process can wait for multiple file descriptors at the same time, and any one of these file descriptors (socket descriptors) enters the ready state, and the select() function you can return.

Here you need to use two system calls (select and recvfrom), while blocking IO (blocking IO) only calls one system call (recvfrom). However, the advantage of using select is that it can handle multiple connections at the same time.

If the number of connections processed is not very high, the web server using select/epoll is not necessarily better than the web server using multi-threading + blocking IO (multi-threading + blocking IO), and the delay may be even greater. The advantage of select/epoll is not that it can handle a single connection better, but that it can handle more connections at the same time.

1. select

During a specified period of time, listen to the file descriptors that the user is interested in, such as readable, writable, and abnormal events.

#include <sys/select.h>
#include <sys/time.h>
#include <sys/types.h>
#include <unistd.h>

int select(int nfds, fd_set *readfds, fd_set *writefds, fd_set *exceptfds, struct timeval *timeout);

Description: Listen to multiple (up to 1024) sockets;

parameter

        nfds

                Add 1 to the largest file descriptor, just use FD_SETSIZE;

        readfds

                Used to monitor readable read; if you don't care, use: (fd_set *)0;

        writefds

                Used to monitor writable write; use if you don’t care: (fd_set *)0;

        exceptfds

                Data used to monitor exceptions; if you don’t care, use: (fd_set *)0;

        timeout

                A pointer to the timeval structure , used to determine the maximum time for select to wait for I/O; if it is empty, it will wait forever;

return value

        Greater than 0 : is the total number of ready file handles;

        Equal to 0 : timeout;

        Less than 0 : indicates an error, error: errno;

void FD_CLR ( int fd , fd_set * set );          //  All bits of a fd_set type variable are set to 0
int   FD_ISSET ( int fd , fd_set * set );         // You can use
void FD_SET ( int fd , fd_set * set );         //  Set a certain position of the variable to
void FD_ZERO ( fd_set * set );                 //  Test whether a certain position is set

example:

select_server.c

#include <sys/types.h>
#include <sys/socket.h>
#include <stdio.h>
#include <sys/time.h>
#include <sys/ioctl.h>
#include <unistd.h>
#include <stdlib.h>
#include <netinet/in.h>
#include <string.h>


#define BUFF_MAX	1024


// 英文小写转换成大写
static void str2up(char *str) {
    while (*str) {
        if (*str >= 'a' && *str <= 'z') {
            *str = *str - 'a' + 'A';
        }
 
        str++;
    }
}



int main(void) {
    int server_sockfd, client_sockfd;
    int server_len, client_len;
    struct sockaddr_in server_address;
    struct sockaddr_in client_address;
    int result = 0;
    fd_set readfds, writefds, rfds, wfds;
    char buff[BUFF_MAX] = { '\0' }; 
    

    // 建立服务器socket
    server_sockfd = socket(AF_INET, SOCK_STREAM, 0);	
    server_address.sin_family = AF_INET;
    server_address.sin_addr.s_addr = htonl(INADDR_ANY);
    server_address.sin_port = htons(9100);

    server_len = sizeof(server_address);

    // 绑定
    bind(server_sockfd, (struct sockaddr *)&server_address, server_len);
    
    // 监听,最多监听10个
    listen(server_sockfd, 10);

    // 清零
    FD_ZERO(&readfds);
    FD_ZERO(&writefds);

    // 将服务端socket加入到集合中
    FD_SET(server_sockfd, &readfds);
    FD_SET(server_sockfd, &writefds);

    while (1) {

        int fd;
        int nread;


        // 将需要监视的描述符集拷贝到select查询队列中,select会对其修改,所以一定要分开使用变量
        rfds = readfds;
        wfds = writefds;

        printf("server waiting...\n");

        // 无期限阻塞,并测试文件描述符变动
        result = select(FD_SETSIZE, &rfds, &wfds, (fd_set *)0, (struct timeval *)0);	// FD_SETSIZE,默认最大文件描述符,在这里最大是1024
        if (result < 1) {
            perror("select\n");
            exit(1);

        } else if (0 == result) {
            printf("time out!\n");
        
        }
        
         

        // 扫描所有的文件描述符
        for (fd = 0; fd < FD_SETSIZE; fd++) {
            // 找到相关文件描述符,read
            if (FD_ISSET(fd, &rfds)) {
                // 判断是否为服务器套接字,是则表示客户端请求连接
                if (fd == server_sockfd) {
                    client_len = sizeof(client_address);
                    client_sockfd = accept(server_sockfd, (struct sockaddr *)&client_address, &client_len);
                    
                    // 将客户端socket加入到集合中
                    FD_SET(client_sockfd, &readfds);
                    //FD_SET(client_sockfd, &writefds);

                    printf("adding client on fd %d\n", client_sockfd);
                
                } else { // 客户端socket中有数据请求时
                    // 取得数据量交给nread
                    ioctl(fd, FIONREAD, &nread);

                    // 客户数据请求完毕,关闭套接字,从集合中清除相应描述符
                    if (0 == nread) {
                        close(fd);
                        FD_CLR(fd, &readfds);	// 去掉g关闭的fd
                        printf("remove client on fd %d\n", fd);
                    
                    } else {
                        read(fd, buff, BUFF_MAX);
                        
                        sleep(5);
                        printf("receive:%s\n", buff);
                        printf("serving client on fd %d\n", fd);
                        FD_SET(client_sockfd, &writefds);

                    }
                }
  
            } else if (FD_ISSET(fd, &wfds)) {
                str2up(buff);	// 转化为大写
                write(fd, buff, sizeof(buff));
                memset(buff, 0, BUFF_MAX);
                FD_CLR(fd, &writefds);
               
            } else {
                //printf("其他\n");
            }

        }

    }


    return 0;
}

cleint.c

#include <sys/types.h>
#include <sys/socket.h>
#include <stdio.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <unistd.h>
#include <stdlib.h>
#include <sys/time.h>


#define BUFF_MAX        1024


int main(int argc, char **argv) {
    int client_sockfd;
    int len;
    struct sockaddr_in address; // 服务器网络地址结构体
    int result;
    char buff[BUFF_MAX] = { '\0' };

    if (argc < 2) {
        fprintf(stderr, "missing parameter\n");
        exit(1);
    }

    strcpy(buff, argv[1]);

    // 建立客户端socket
    client_sockfd = socket(AF_INET, SOCK_STREAM, 0);
    address.sin_family = AF_INET;
    address.sin_addr.s_addr = inet_addr("127.0.0.1");
    address.sin_port = htons(9100);

    len = sizeof(address);

    // 连接服务器
    result = connect(client_sockfd, (struct sockaddr *)&address, len);
    if (-1 == result) {
        perror("connect");
        exit(1);
    }


    // 发送数据给服务器
    write(client_sockfd, buff, strlen(buff));

    memset(buff, '\0', BUFF_MAX);
    // 接收服务器发回来的数据
    read(client_sockfd, buff, BUFF_MAX);
    printf("receive:%s\n", buff);
    sleep(3);

    close(client_sockfd);
    return 0;
}

2. poll

Same as select, if no event occurs, it will enter the dormant state, if an event occurs within the specified time, it will return success, and if no event occurs after the specified time, it will return failure. It can be seen that sleeping the process during the waiting period and using event-driven to wake up the process will improve the efficiency of the CPU.

The difference between poll and select: select has a file handle online setting, the value is FD_SETSIZE , while poll has no limit in theory!

#include <poll.h>

int poll(struct pollfd *fds, nfds_t nfds, int timeout);

Description: Wait for an event on the file descriptor;

parameter

        fds

                The file descriptor set to be monitored is specified in the fds parameter, and multiple structures can be passed, that is to say, events generated by multiple drive devices can be monitored, and as long as one of them generates a request event, it can return immediately;

struct pollfd {
    int fd;           /* 文件描述符   open打开的那个 */
    short events;     /* 请求的事件类型,监视驱动文件的事件掩码 */  POLLIN | POLLOUT
    short revents;    /* 驱动文件实际返回的事件 */
}

        The event type events can have the following values:

                POLLIN has data to read                   

                POLLRDNORM has ordinary data readable, equivalent to POLLIN      

                POLLPRI has urgent data to read                 

                POLLOUT write data will not cause blocking                

                An error occurred on the file descriptor specified by POLLER                  

                The file descriptor specified by POLLHUP is hung up                

                POLLNVAL invalid request, cannot open the specified file descriptor              

        ndfs

                Monitor the number of driver files;

        timeout

                Timeout time, the unit is ms;

return value

        An event occurs : return the number of file descriptors whose revents field is not 0;

        timeout : return 0;

        Failure : return -1, and set the error flag errno;

example:

poll_server.c

#include <sys/socket.h>
#include <stdio.h>
#include <sys/time.h>
#include <unistd.h>
#include <stdlib.h>
#include <netinet/in.h>
#include <poll.h>
#include <string.h>


#define BUFF_MAX	1024
#define MAX_FD		10240
struct pollfd fds[MAX_FD];
int cur_max_fd = 1;



// 英文小写转换成大写
void str2up(char *str) {
    while (*str) {
        if (*str >= 'a' && *str <= 'z') {
            *str = *str - 'a' + 'A';
        }
 
        str++;
    }
}


void setMaxFD(int fd) {
    
    if (fd >= cur_max_fd) {
        cur_max_fd = fd + 1;
    }

}



int main(void) {
    int server_sockfd, client_sockfd;
    int server_len, client_len;
    struct sockaddr_in server_address;
    struct sockaddr_in client_address;
    int result = 0;
    char buff[BUFF_MAX] = { '\0' }; 
    

    // 建立服务器socket
    server_sockfd = socket(AF_INET, SOCK_STREAM, 0);	
    server_address.sin_family = AF_INET;
    server_address.sin_addr.s_addr = htonl(INADDR_ANY);
    server_address.sin_port = htons(9100);

    server_len = sizeof(server_address);

    // 绑定
    bind(server_sockfd, (struct sockaddr *)&server_address, server_len);
    
    // 监听,最多监听10个
    listen(server_sockfd, 10);

    // 将服务器添加到监听数组
    fds[server_sockfd].fd = server_sockfd;
    fds[server_sockfd].events = POLLIN | POLLOUT;
    fds[server_sockfd].revents = 0;

    setMaxFD(server_sockfd);

    while (1) {

        int fd = 0, i = 0;
        int nread = 0;

        printf("server waiting...\n");

        // 阻塞等待1秒,监听cur_max_fd个socket
        result = poll(fds, cur_max_fd, 1000);	// 1s
        if (result < 0) {
            perror("poll\n");
            exit(1);

        } else if (0 == result) {
            printf("time out!\n");
        
        } else {
            // 扫描所有的文件描述符
            for (i = 0; i < cur_max_fd; i++) {
                // 找到相关文件描述符
                if (fds[i].revents) {
                    fd = fds[i].fd;

                    // 判断是否为服务器套接字,是则表示客户端请求连接
                    if (fd == server_sockfd) {
                        client_len = sizeof(client_address);
                        client_sockfd = accept(server_sockfd, (struct sockaddr *)&client_address, &client_len);
                        
                        // 将新的客户端添加到监听数组中 
                        fds[client_sockfd].fd = client_sockfd;
                        fds[client_sockfd].events = POLLIN;
                        fds[client_sockfd].revents = 0;

                        setMaxFD(client_sockfd);

                        printf("adding client on fd %d\n", client_sockfd);
                    
                    } else { // 客户端socket中有数据请求时

                        /* 有数据可读 */
                        if (fds[i].revents & POLLIN) {
                            // 取得数据量交给nread
                            nread = read(fd, buff, BUFF_MAX);
                            // 客户数据请求完毕,关闭套接字,从集合中清除相应描述符
                            if (0 == nread) {
                                close(fd);
                                // 关闭后,监听的该socket需要清0
                                memset(&fds[i], 0, sizeof(struct pollfd));
                                printf("remove client on fd %d\n", fd);
                        
                            } else {
                                sleep(1);
                                printf("receive:%s\n", buff);
                                printf("serving client on fd %d\n", fd);

                                // 监听写事件,如果写准备好了,再写
                                fds[i].events = POLLOUT;
                                //write(fd, buff, sizeof(buff));
                            }
     
                        /* 有数据可写 */
                        } else if(fds[i].revents & POLLOUT) {
                            str2up(buff);	// 转化为大写
                            write(fd, buff, strlen(buff));
                            // 设置回继续监听读数据状态
                            fds[i].events = POLLIN;
                            memset(buff, 0, BUFF_MAX);
                        }
                    }     
                }
            }
        }
    }


    return 0;
}

client.c

It is consistent with the client.c code in select!

3. epool

The most commonly used technology in high concurrency is epoll. Epoll is very efficient, much higher than select and poll!

The event queue of epoll is composed of red-black tree, so it gets data super fast!

epoll generally consists of three commonly used functions, epoll_create, epoll_clt and epoll_wait.

1). epoll_create

#include <sys/epoll.h>

int epoll_create(int size);

Description: Create an epoll handle and open an epoll file descriptor;

parameter

        size

                This parameter is meaningless, just fill in an integer greater than 0!

return value

        Success : return epoll file descriptor;

        Failure : return -1, and set the error flag errno;

2). epoll_ctl

#include <sys/epoll.h>

int epoll_ctl(int epfd, int op, int fd, struct epoll_event *event);

Description: Create, modify or delete events to the epoll object;

parameter

        epfd

                epoll file descriptor;

        op

                The values ​​are as follows:

                        EPOLL_CTL_ADD         adds new events to epoll;

                       EPOLL_CTL_MOD         modifies the events in epoll;

                        EPOLL_CTL_DEL           deletes events in epoll;

        fd

                Socket file descriptor, the monitored file descriptor;

        event

                Event, the structure is as follows:

typedef union epoll_data {
    void        *ptr;
    int          fd;
    uint32_t     u32;
    uint64_t     u64;
} epoll_data_t;

struct epoll_event {
    uint32_t     events;      /* Epoll events */
    epoll_data_t data;        /* User data variable */
};

                events value:

                        EPOLLIN indicates that there is data to read (or accept client connections and close connections);

                        EPOLLOUT indicates that there is data to write (the data is ready and can be sent);

                        EPOLLERR indicates that an error occurred in the corresponding connection;

                        EPOLLHUP indicates that the corresponding connection is suspended;

return value

        success : return 0;

        Failure : return -1, and set the error flag errno; 

3). epoll_wait

#include <sys/epoll.h>

int epoll_wait(int epfd, struct epoll_event *events, int maxevents, int timeout);

Description : Collect events that have occurred in the events monitored by epoll (waiting for I/O events on epoll file descriptors);

parameter

        epfd

                epoll file descriptor;

        events

                The epoll_event structure array has been allocated, and epoll will copy the events that occurred to the events array (events cannot be a null pointer, the kernel is only responsible for copying data to the events data, and will not help us allocate in user mode Memory. Kernel is very efficient!);

        maxevents

                The maximum number of events that can be returned this time, usually the maxevents parameter is equal to the size of the pre-allocated events array;

        timeout

                Indicates the most waiting event (in milliseconds) when no event is detected. If timeout is 0, it will return immediately and will not wait; -1 means block indefinitely; 3000 means wait for 3 seconds;

return value

        If it is greater than 0 , it returns the number of file symbols with events;

        Equal to 0 , timeout return;

        Failure : return -1, and set the error flag;

4). Example of echo server based on epoll

The client loops 1000 times to send data to the server, sleeps for 2 milliseconds each time, the server sends it back to the client after receiving it, and the client receives it normally. 

server.c

#include <stdio.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <arpa/inet.h>
#include <netinet/in.h>
#include <stdlib.h>
#include <errno.h>
#include <sys/epoll.h>
#include <unistd.h>
#include <fcntl.h>
#include <string.h>


#define SERVER_PORT 		9100
#define BUFF_SIZE		    1024
#define EPOLL_EVENT_SIZE	1024


int epoll_fd = 0;



typedef struct _ConnectStat {
    char buff[BUFF_SIZE];
    int fd;
    struct epoll_event ev;
} ConnectStat;


// 设置不阻塞
void set_nonblock(int fd) {
    int f1 = fcntl(fd, F_GETFL);
    fcntl(fd, F_SETFL, f1 | O_NONBLOCK);
}



int startup(void) {

    int return_value = -1;
    int ret = 0;
    int listen_socket = 0;	// 服务器套接字
    struct sockaddr_in server_addr;

    // 创建通信套接字
    listen_socket = socket(AF_INET, SOCK_STREAM, 0);
    if (-1 == listen_socket) {
        fprintf(stderr, "socket failed! reason: %s\n", strerror(errno));
        return return_value;
    }

    // 清空标志,写上地址和端口号
    bzero(&server_addr, sizeof(server_addr));

    server_addr.sin_family = AF_INET;	// 选择协议组ipv4
    server_addr.sin_addr.s_addr = htonl(INADDR_ANY);	// 监听本地所有ip地址
    server_addr.sin_port = htons(SERVER_PORT);		// 绑定端口号

    // 绑定
    ret = bind(listen_socket, (struct sockaddr *)&server_addr, sizeof(server_addr));
    if (-1 == ret) {
        fprintf(stderr, "bind failed! reason: %s\n", strerror(errno));
        return return_value;
    }


    // 设置端口复用
    int opt = 1;
    setsockopt(listen_socket, SOL_SOCKET, SO_REUSEADDR, &opt, sizeof(opt));

    // 监听
    ret = listen(listen_socket, 16);
    if (-1 == ret) {
        fprintf(stderr, "listen failed! reason: %s\n", strerror(errno));
        return return_value;
    }

    return listen_socket;
}


ConnectStat *stat_init(int fd) {
    ConnectStat *temp =  (ConnectStat *)malloc(sizeof(ConnectStat));
    if (!temp) {
        fprintf(stderr, "ConnectStat malloc failed! reason: %s\n", strerror(errno));
        return NULL;
    }

    memset(temp, 0, sizeof(ConnectStat));
    temp->fd = fd;
    memset(temp->buff, '\0', sizeof(temp->buff));

    return temp;
}


int connect_handle(int new_fd) {
    ConnectStat *stat = stat_init(new_fd);
    if (!stat) {
        return -1;
    }

    set_nonblock(new_fd);

    stat->ev.events = EPOLLIN;
    stat->ev.data.ptr = stat;

    // 添加到epoll监听池中
    int ret = epoll_ctl(epoll_fd, EPOLL_CTL_ADD, new_fd, &stat->ev);
    if (-1 == ret) {
        fprintf(stderr, "connect_handle epoll_ctl failed! reason: %s\n", strerror(errno));
        return -1;
    }

    return 0;
}



int main(int argc, char **argv) {
    
    int ret = -1;
    //int epoll_fd = 0;
    int listen_sock = 0;

    listen_sock = startup();
    if (-1 == listen_sock) {
        exit(-1);
    }

    // 创建eopll池
    epoll_fd = epoll_create(256);
    if (-1 == epoll_fd) {
        fprintf(stderr, "epoll_create failed! reason: %s\n", strerror(errno));
        exit(1);
    }

   
    ConnectStat *stat = stat_init(listen_sock);
    struct epoll_event ev;
    ev.events = EPOLLIN;	// 读 事件
    ev.data.ptr = stat;

    // 托管
    ret = epoll_ctl(epoll_fd, EPOLL_CTL_ADD, listen_sock, &ev);
    if (-1 == ret) {
        fprintf(stderr, "epoll_ctl failed! reason: %s\n", strerror(errno));
        exit(2);
    }

    struct epoll_event revs[EPOLL_EVENT_SIZE];

    int timeout = 3000;	// 3s
    int num = 0;
    
    while (1) {
        // 监听事件
        num = epoll_wait(epoll_fd, revs, EPOLL_EVENT_SIZE, timeout);
        switch (num) {
            case 0: {
                printf("timeout!\n");
            }
            break;

            case -1: {
                fprintf(stderr, "epoll_wait failed! reason: %s\n", strerror(errno));
            }
            break;

            default: {
                struct sockaddr_in perr;
                socklen_t len = sizeof(perr);

                int i = 0;
                for (; i < num; i++) {
                    // 获取数据结构体
                    ConnectStat *stat = (ConnectStat *) revs[i].data.ptr;
                    if (!stat) {
                        fprintf(stderr, "stat = NULL, i = %d\n", i);
                        continue;
                    }

                    int fd  = stat->fd;    // 获取事件fd
                    if (fd == listen_sock && (revs[i].events & EPOLLIN)) {
                        // 链接客户端
                        int new_fd = accept(listen_sock, (struct sockaddr *)&perr, &len);
                        if (-1 == new_fd) {
                             printf("accept failed! reason: %s\n", strerror(errno));
                        
                         // 客户端连接
                         } else {  
                             printf("get a new client: %s : %d\n", inet_ntoa(perr.sin_addr), ntohs(perr.sin_port));
                             connect_handle(new_fd);
                         }

                    // 有数据 读
                    } else if (revs[i].events & EPOLLIN) {
                        
                         // 读取客户端发送过来的数据
                         int ret = read(fd, stat->buff, sizeof(stat->buff)); 
                         if (ret > 0) {
                             printf("%s\n", stat->buff);

                             // 将当前fd设置为写状态,当数据准备好后,会触发写事件去发送数据
                             stat->ev.events = EPOLLOUT;
                             ret = epoll_ctl(epoll_fd, EPOLL_CTL_MOD, fd, &stat->ev);
                             if (-1 == ret) {
                                 fprintf(stderr, "epoll_ctl failed! reason: %s\n", strerror(errno));
                                 continue;
                             }
                         
                         // 客户端主动断开
                         } else if (0 == ret) {
                             printf("client %d close!\n", fd);
                             epoll_ctl(epoll_fd, EPOLL_CTL_DEL, fd, NULL);
                             close(fd);
                             continue;
                         
                         } else {
                             printf("read failed! reason: %s\n", strerror(errno));
                             continue;
                         }
                    
                     // 有数据 写
                     } else if (revs[i].events & EPOLLOUT) {

                         // 将收到的数据完整的发回给客户端
                         int ret = write(fd, stat->buff, strlen(stat->buff));
                         if (-1 == ret) {
                             fprintf(stderr, "write failed! reason: %s\n", strerror(errno));
                             continue;
                         }

                         // 将当前fd重新设置回监听读数据状态
                         stat->ev.events = EPOLLIN;
                         ret = epoll_ctl(epoll_fd, EPOLL_CTL_MOD, fd, &stat->ev);
                         if (-1 == ret) {
                             fprintf(stderr, "(write)epoll_ctl failed! reason: %s\n", strerror(errno));
                             continue;
                         }

                     } else {
                         printf("--else\n");
                     }
                }
            }
            break;
        }
    }
   
    return 0;
}

client.c

#include <sys/types.h>
#include <sys/socket.h>
#include <stdio.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <unistd.h>
#include <stdlib.h>
#include <sys/time.h>
#include <string.h>

#define BUFF_MAX	1024
#define COUNT		1000

int main(int argc, char **argv) {
    int client_sockfd;
    int len;
    struct sockaddr_in address;	// 服务器网络地址结构体
    int result;
    char buff[BUFF_MAX] = { '\0' };

    if (argc < 2) {
        fprintf(stderr, "missing parameter\n");
        exit(1);
    }

    strcpy(buff, argv[1]);

    // 建立客户端socket
    client_sockfd = socket(AF_INET, SOCK_STREAM, 0);
    address.sin_family = AF_INET;
    address.sin_addr.s_addr = inet_addr("127.0.0.1");
    address.sin_port = htons(9100);

    len = sizeof(address);
    
    // 连接服务器
    result = connect(client_sockfd, (struct sockaddr *)&address, len);
    if (-1 == result) {
        perror("connect");
        exit(1);
    }


    char buff1[BUFF_MAX] = { '\0' };
    int a = COUNT;
    while (a-- >= 0) {

        // 发送数据给服务器
        write(client_sockfd, buff, strlen(buff));   

        //memset(buff, '\0', BUFF_MAX); 
        // 接收服务器发回来的数据
        read(client_sockfd, buff1, BUFF_MAX);
        printf("receive:%s\n", buff1);
        usleep(2000);//  睡眠2毫秒
        memset(buff1, '\0', BUFF_MAX);

    }

    close(client_sockfd);
    return 0;
}

4. Horizontal trigger and edge trigger

1). Horizontal trigger

        Level_triggered (level trigger): When a readable and writable event occurs on the monitored file descriptor, epoll_wait() will notify the handler to read and write. If you don't read and write all the data at once this time (such as the read and write buffer is too small), then the next time you call epoll_wait(), it will also notify you to continue reading and writing on the file descriptor that has not been read and written. , of course, if you don't read or write all the time, it will keep notifying you! ! ! If there are a large number of ready file descriptors in the system that you don't need to read and write, and they will return every time, this will greatly reduce the efficiency of the handler to retrieve the ready file descriptors that it cares about! ! !  

Setting method: the default is horizontal trigger;

That is, the epoll server code above is the default horizontal trigger!

2). Edge trigger

Edge_triggered (edge ​​trigger): When a readable and writable event occurs on the monitored file descriptor, epoll_wait() will notify the handler to read and write. If you haven't read and written all the data this time (such as the read and write buffer is too small), it will not notify you the next time you call epoll_wait(), that is, it will only notify you once until the file descriptor is You will be notified only when there is a second readable and writable event! ! ! This mode is more efficient than level triggering, and the system won't be flooded with a lot of ready file descriptors that you don't care about! ! !

Setting method: stat->_ev.events = EPOLLIN | EPOLLET

In any case, edge triggering is recommended!

Edge case: (echo server)

101234567890 10 is int type ten, and the rest are all data; the client needs to combine such a data structure and then send it to the server, and the server also needs to analyze it after receiving it, and allocate the corresponding memory to read the data after getting the data size;

The client loops two thousand times, sends data to the server, and sleeps for two milliseconds each loop.

server.c

#include <stdio.h>
#include <unistd.h>
#include <sys/types.h>
#include <sys/socket.h>
#include <arpa/inet.h>
#include <netinet/in.h>
#include <stdlib.h>
#include <errno.h>
#include <sys/epoll.h>
#include <unistd.h>
#include <fcntl.h>
#include <string.h>


#define SERVER_PORT 		9100
#define BUFF_SIZE		    8
#define DATA_LEN_BYTES		4
#define EPOLL_EVENT_SIZE	1024
#define DATA_BUFF_SIZE		4096

int epoll_fd = 0;
//char data_buff[DATA_BUFF_SIZE] = { '\0' };


typedef struct _ConnectStat {
    int fd;                     // 客户端socket
    struct epoll_event ev;      
    char *data_buff;            // 存储读取的数据
} ConnectStat;


// 设置不阻塞
void set_nonblock(int fd) {
    int f1 = fcntl(fd, F_GETFL);
    fcntl(fd, F_SETFL, f1 | O_NONBLOCK);
}



int startup(void) {

    int return_value = -1;
    int ret = 0;
    int listen_socket = 0;	// 服务器套接字
    struct sockaddr_in server_addr;

    // 创建通信套接字
    listen_socket = socket(AF_INET, SOCK_STREAM, 0);
    if (-1 == listen_socket) {
        fprintf(stderr, "socket failed! reason: %s\n", strerror(errno));
        return return_value;
    }

    // 清空标志,写上地址和端口号
    bzero(&server_addr, sizeof(server_addr));

    server_addr.sin_family = AF_INET;	// 选择协议组ipv4
    server_addr.sin_addr.s_addr = htonl(INADDR_ANY);	// 监听本地所有ip地址
    server_addr.sin_port = htons(SERVER_PORT);		// 绑定端口号

    // 绑定
    ret = bind(listen_socket, (struct sockaddr *)&server_addr, sizeof(server_addr));
    if (-1 == ret) {
        fprintf(stderr, "bind failed! reason: %s\n", strerror(errno));
        return return_value;
    }


    // 设置端口复用
    int opt = 1;
    setsockopt(listen_socket, SOL_SOCKET, SO_REUSEADDR, &opt, sizeof(opt));

    // 监听
    ret = listen(listen_socket, 16);
    if (-1 == ret) {
        fprintf(stderr, "listen failed! reason: %s\n", strerror(errno));
        return return_value;
    }

    return listen_socket;
}


ConnectStat *stat_init(int fd) {
    ConnectStat *temp =  (ConnectStat *)malloc(sizeof(ConnectStat));
    if (!temp) {
        fprintf(stderr, "ConnectStat malloc failed! reason: %s\n", strerror(errno));
        return NULL;
    }

    memset(temp, 0, sizeof(ConnectStat));
    temp->fd = fd;
    //memset(temp->data_buff, '\0', sizeof(temp->data_buff));
    temp->data_buff = NULL;

    return temp;
}


int connect_handle(int new_fd) {
    ConnectStat *stat = stat_init(new_fd);
    if (!stat) {
        return -1;
    }

    set_nonblock(new_fd);

    stat->ev.events = EPOLLIN | EPOLLET;    // 边缘触发:EPOLLET
    stat->ev.data.ptr = stat;

    // 添加到epoll监听池中
    int ret = epoll_ctl(epoll_fd, EPOLL_CTL_ADD, new_fd, &stat->ev);
    if (-1 == ret) {
        fprintf(stderr, "connect_handle epoll_ctl failed! reason: %s\n", strerror(errno));
        return -1;
    }

    return 0;
}



int main(int argc, char **argv) {
    
    int ret = -1;
    //int epoll_fd = 0;
    int listen_sock = 0;

    listen_sock = startup();
    if (-1 == listen_sock) {
        exit(-1);
    }

    // 创建eopll池
    epoll_fd = epoll_create(256);
    if (-1 == epoll_fd) {
        fprintf(stderr, "epoll_create failed! reason: %s\n", strerror(errno));
        exit(1);
    }

   
    ConnectStat *stat = stat_init(listen_sock);
    struct epoll_event ev;
    ev.events = EPOLLIN | EPOLLET;	// 读 事件, EPOLLET: 边缘触发
    ev.data.ptr = stat;

    // 托管
    ret = epoll_ctl(epoll_fd, EPOLL_CTL_ADD, listen_sock, &ev);
    if (-1 == ret) {
        fprintf(stderr, "epoll_ctl failed! reason: %s\n", strerror(errno));
        exit(2);
    }

    struct epoll_event revs[EPOLL_EVENT_SIZE];

    int timeout = 3000;	// 3s
    int num = 0;
    
    while (1) {
        // 监听事件
        num = epoll_wait(epoll_fd, revs, EPOLL_EVENT_SIZE, timeout);
        switch (num) {
            case 0: {
                printf("timeout!\n");
            }
            break;

            case -1: {
                fprintf(stderr, "epoll_wait failed! reason: %s\n", strerror(errno));
            }
            break;

            default: {
                struct sockaddr_in perr;
                socklen_t len = sizeof(perr);
                int ret = 0;
                char buff[BUFF_SIZE];

                int i = 0;
                for (; i < num; i++) {
                    // 获取数据结构体
                    ConnectStat *stat = (ConnectStat *) revs[i].data.ptr;
                    if (!stat) {
                        fprintf(stderr, "stat = NULL, i = %d\n", i);
                        continue;
                    }

                    int fd  = stat->fd;    // 获取事件fd
                    if (fd == listen_sock && (revs[i].events & EPOLLIN)) {
                        // 链接客户端
                        int new_fd = accept(listen_sock, (struct sockaddr *)&perr, &len);
                        if (-1 == new_fd) {
                             printf("accept failed! reason: %s\n", strerror(errno));
                             continue;
                        
                         // 客户端链接
                         } else {  
                             printf("get a new client: %s : %d\n", inet_ntoa(perr.sin_addr), ntohs(perr.sin_port));
                             connect_handle(new_fd);
                         }

                    // 有数据 读
                    } else if (revs[i].events & EPOLLIN) {

                         memset(buff, '\0', sizeof(buff));  
                         // 读取客户端发送过来的数据
                         ret = read(fd, buff, DATA_LEN_BYTES);    // 读取头部信息,四个字节
                         if (ret > 0) {
                             int buff_size = *(int *)buff;  // 获取int类型数据,即获取数据的大小

                             // 根据接收数据大小分配相应的内存大小
                             stat->data_buff = (char *)malloc(buff_size + 1);
                             memset(stat->data_buff, '\0', buff_size + 1);

                            
                             // 使用循环将数据全都读取出来 
                             while (1) {
                                 ret = read(fd, buff, sizeof(buff));
                                 if (-1 == ret) {
                                     if (errno == EAGAIN) {
                                         //printf("data reive successes!\n");
                                     } else {
                                         printf("data receive failed! reason: %s\n", strerror(errno));
                                     }
                                     break;
                                 }
                                 // 将数据拷贝到大容量数组中保存
                                 //memcpy(data_buff + strlen(data_buff), buff, ret); // 内存拷贝
                                 strncat(stat->data_buff, buff, ret);  // 在字符串尾部追加字符串
                                 memset(buff, 0, sizeof(buff)); 
                             }
                             stat->data_buff[strlen(stat->data_buff)] = '\0';
                             printf("%s\n", stat->data_buff);

                             // 将当前fd设置为写状态,当数据准备好后,会触发写事件去发送数据
                             stat->ev.events = EPOLLOUT | EPOLLET;
                             ret = epoll_ctl(epoll_fd, EPOLL_CTL_MOD, fd, &stat->ev);
                             if (-1 == ret) {
                                 fprintf(stderr, "epoll_ctl failed! reason: %s\n", strerror(errno));
                                 continue;
                             }
                         
                         // 客户端主动断开
                         } else if (0 == ret) {
                             printf("client %d close!\n", fd);
                             ret = epoll_ctl(epoll_fd, EPOLL_CTL_DEL, fd, NULL);
                             if (-1 == ret) {
                                 fprintf(stderr, "epoll_ctl failed! reason: %s\n", strerror(errno));
                                 continue; 
                             }

                             close(fd);
                             free(stat);
                             stat = NULL;
                             continue;
                         
                         } else {
                             printf("read failed! reason: %s\n", strerror(errno));
                             continue;
                         }
                    
                     // 有数据 写
                     } else if (revs[i].events & EPOLLOUT) {

                         // 将收到的数据完整的发回给客户端
                         int ret = write(fd, stat->data_buff, strlen(stat->data_buff));
                         if (-1 == ret) {
                             fprintf(stderr, "write failed! reason: %s\n", strerror(errno));
                             continue;
                         }

                         // 将当前fd重新设置回监听读数据状态
                         stat->ev.events = EPOLLIN | EPOLLET;
                         ret = epoll_ctl(epoll_fd, EPOLL_CTL_MOD, fd, &stat->ev);
                         if (-1 == ret) {
                             fprintf(stderr, "(write)epoll_ctl failed! reason: %s\n", strerror(errno));
                             continue;
                         }

                     } else {
                         printf("--else\n");
                     }
                }
            }
            break;
        }
    }
   
    return 0;
}

client.c

#include <sys/types.h>
#include <sys/socket.h>
#include <stdio.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <unistd.h>
#include <stdlib.h>
#include <sys/time.h>
#include <string.h>

#define BUFF_MAX	1024
#define COUNT		2000
#define DATA_LEN_BYTES  4
 

int main(int argc, char **argv) {
    int client_sockfd;
    int len;
    struct sockaddr_in address;	// 服务器网络地址结构体
    int result;
    char *message = NULL;
    char buff1[BUFF_MAX] = { '\0' };

    if (argc < 2) {
        fprintf(stderr, "missing parameter\n");
        exit(1);
    }

    message = argv[1];
    int ms_len = strlen(message);

    // 组装数据包
    char *buff = (char *)malloc(ms_len + DATA_LEN_BYTES);
    memset(buff, '\0', ms_len + DATA_LEN_BYTES);

    *((int *)buff) = ms_len;
    int size = *((int *)buff);
    printf("size = %d\n", size);
    memcpy(buff + DATA_LEN_BYTES, message, ms_len);



    // 建立客户端socket
    client_sockfd = socket(AF_INET, SOCK_STREAM, 0);
    address.sin_family = AF_INET;
    address.sin_addr.s_addr = inet_addr("127.0.0.1");
    address.sin_port = htons(9100);

    len = sizeof(address);
    
    // 连接服务器
    result = connect(client_sockfd, (struct sockaddr *)&address, len);
    if (-1 == result) {
        perror("connect");
        exit(1);
    }

    

    int a = COUNT;
    while (a-- >= 0) {

        // 发送数据给服务器
        write(client_sockfd, buff, ms_len + DATA_LEN_BYTES); 

        // 接收服务器发回来的数据
        read(client_sockfd, buff1, BUFF_MAX);
        printf("receive:%s\n", buff1);
        usleep(2000); // 休眠2毫秒
        memset(buff1, '\0', BUFF_MAX);
    }

    close(client_sockfd);
    return 0;
}

5. Encapsulate the epoll framework

High performance, high concurrency, encapsulates the framework resources of epoll - CSDN library https://download.csdn.net/download/cpp_learner/87630114

6. libevent

Libevent is a lightweight open source, high-performance event-triggered network library, suitable for various platforms such as windows, linux, bsd, etc., internally uses select, epoll, kqueue and other system calls to manage event mechanisms.

It is used by many open source projects, such as the famous memcached and so on.

Features:

        Event driven, high performance;

        Lightweight, network-focused (compared to ACE);

        Open source, the code is quite refined and easy to read;

        Cross-platform, supports Windows, Linux, BSD and Mac OS;

        Supports multiple I/O multiplexing technologies (epoll, poll, dev/poll, select, kqueue, etc.), abstracts the multiplexing model under different operating systems, and can choose to use different models. Provide services through event functions;

        Support events such as I/O, timers and signals;

Adopt Reactor mode;

Libevent is an implementation of a typical reactor pattern.

Ordinary function call mechanism: the program calls a function, the function executes, the program waits, and the function returns the result to the calling program (if it contains the function return value), that is, it is executed sequentially.

The basic process of the Reactor mode: the application needs to provide the corresponding interface and register to the reactor reactor. If the corresponding event occurs, the reactor will automatically call the corresponding registered interface function (similar to the callback function) to notify you, so libevent is an event-triggered networking library.

Functions of libevent

Libevent provides event notification, io cache event, timer, timeout, asynchronous DNS resolution, event-driven http server and an rpc framework.

        Event notification: The callback function will be executed when the file descriptor is readable and writable.

        IO cache: Cache events provide input and output caches, which can be automatically read and written, and users do not need to directly operate io.

        Timer: libevent provides a timer mechanism that can call a callback function after a certain time interval.

        Signal: Trigger the signal, execute the callback.

        Asynchronous dns resolution: libevent provides a set of dns resolution functions for asynchronous resolution of dns servers.

        Event-driven http server: libevent provides a simple HTTP server that can be integrated into applications.

        RPC client server framework: libevent creates an RPC framework for creating RPC servers and clients, which can automatically encapsulate and unpack data structures.

To put it simply, it is to add the event that needs to be monitored to the listening queue. When an event is triggered, the callback function will be called to process it!

1). libevent installation

Official website: http://libevent.org/

Decompression command: tar -zxvf libevent-2.1.12-stable.tar.gz

Enter the decompressed path, and execute the following commands in sequence:

./configure --disable-openssl
make
make install

Use the command to check whether the installation is successful: ls -la /usr/local/include | grep event

If event2 and event.h appear, it means the installation has been successful

The header file is in the path: cd /usr/local/include

The library file is in the path: cd /usr/local/lib 

2). Libevent main API introduction

        A. event_base_new

struct event_base *event_base_new(void);

The event_base_new () function allocates and returns a new event_base with default settings. The function checks the environment variable and returns a pointer to event_base. Returns NULL if an error occurs. When selecting various methods, the function chooses the fastest method supported by the OS.

Using this function is sufficient for most programs.

The event_base_new () function is declared in #include <event2/event.h> !

        B. event_new

struct event *event_new(struct event_base *base, evutil_socket_t fd, short  what, event_callback_fn cb, void *arg);

Create events using the event_new() interface;

Parameter three (what) uses the following macros:

        EV_TIMEOUT // timeout event

        EV_READ // read event

        EV_WRITE // write event

        EV_SIGNAL // signal event

        EV_PERSIST // continuous monitoring

        EV_ET // edge trigger

Parameter 4 (cb) is a callback function, and a function needs to be passed in. Its type is a function pointer, and the type is as follows:

typedef void (*event_callback_fn) (evutil_socket_t fd, short events,  void *arg);     // arg是传的参数

Parameter 5 (arg) is the parameter for the callback function;

event_new() attempts to allocate and construct a new event for base. The what parameter is a collection of the above flags. If fd is non-negative, it is the file whose read and write events will be observed.

When the event is activated, libevent will call the cb function, passing these parameters: the file descriptor fd, the bit field representing all triggered events, and the arg parameter when constructing the event.

event_new() will return NULL when an internal error occurs, or an invalid argument is passed.

All newly created events are in an initialized and non-pending state, which can be made pending by calling event_add().

To free an event, call event_free(). It is safe to call event_free() on a pending or active event: the function will make the event inactive and non-pending before releasing the event.

void event_free(struct event *event);

example:

struct event *ev_listen = event_new(base, socket, EV_READ | EV_PERSIST, accept_connection, base);

        C. event_add

int event_add(struct event *ev, const struct timeval *timeout);

After constructing an event, you can't actually do anything with it until you add it to the event_base. Add events to event_base using event_add();

Calling event_add() on a non-pending event will make it pending in the configured event_base.

The function returns 0 on success and -1 on failure.

If tv is NULL, the added event will not timeout. Otherwise, tv specifies the timeout value in seconds and microseconds.

If event_add() is called on an event that is already pending, the event will remain pending and will be rescheduled within the specified timeout.

Note : Do not set tv to the time you want the timeout event to execute. If you set "tv->tv_sec = time(NULL)+10;" on January 1, 2010, the timeout event will wait 40 years instead of 10 seconds.

example:

event_add(ev_listen, NULL);

        D. event_del

int event_del(struct event *ev);

Calling event_del() on an already initialized event will make it non-pending and non-active. If the event is not pending or active, the call will have no effect. The function returns 0 on success and -1 on failure.

Note : If the event is removed after the event is activated but before its callback is executed, the callback will not be executed.

The function is defined in <event2/event.h>;

        E. event_base_dispatch

int event_base_dispatch(struct event_base *event_base);

event_base_dispatch () is equivalent to event_base_loop() with no flags set. Therefore, event_base_dispatch() will run until there are no registered events, or event_base_loopbreak() or event_base_loopexit() is called.

The function is defined in <event2/event.h>;

        F. event_base_free

void event_base_free(struct event_base *base);

After using event_base, use event_base_free() to release;

NOTE: This function will not release any events currently associated with event_base, or close their sockets, or release any pointers.

event_base_free () is defined in <event2/event.h>;

        G. event_set

void event_set(struct event *event, evutil_socket_t fd, short what, void (*Callback) (evutil_socket_t, short, void *), void *arg);

Re-set the event;

After calling this function, you must reassign struct event_base * base to him, as follows:

event_set(ev, fd, EV_READ, do_echo_handler, (void *)stat);
ev->ev_base = ev_base;    // 必须重置事件集合

Then you need to re-add ev to the base collection;

event_add(ev, NULL);

        H. event_assign

int event_assign(struct event *event, struct event_base *base, evutil_socket_t fd, short what, void (*Callback) (evutil_socket_t , short , void *), void *arg);

Add an "empty" event to base; 

The arguments to event_assign() are the same as those to event_new(), except that the event argument must point to an uninitialized event. The function returns 0 on success, and -1 if an internal error occurs or an incorrect parameter is used.

warn

Do not call event_assign() on an event already pending in event_base, this can lead to hard-to-diagnose bugs. If initialized and pending, call event_del() before calling event_assign().

example:

event_assign(ev, base, sockfd, EV_READ , do_echo_request, (void*)stat);

        I. evconnlistener_new_bind

struct evconnlistener *evconnlistener_new_bind(struct event_base *base, evconnlistener_cb cb, void *ptr, unsigned flags, int backlog, const struct sockaddr *sa, int socklen);

listen, connect, bind, accept, create a listening object, and listen to the next TCP connection on the specified address;

The base parameter is the event_base used by the listener to monitor the connection. cb is the callback function to be called when a new connection is received; if cb is NULL, the listener is disabled until a callback function is set. The ptr pointer will be passed to the callback function. The flags parameter controls the behavior of the callback function. The backlog is the maximum number of connections that the network stack allows in an unaccepted state at any one time. If the backlog is negative, libevent tries to pick a better value; if it is 0, libevent assumes that listen() has been called on the provided socket.

The provided callback function is called when a new connection is received. The listener parameter is the connection listener that receives connections. The sock parameter is the newly received socket. The addr and len parameters are the address and address length of the receiving connection.

Include the header file: #include <event2/listener.h> 

Parameter two:

        typedef void (*evconnlistener_cb) (struct evconnlistener *listener, evutil_socket_t sock, struct sockaddr *addr, int len, void *ptr);

Parameter four:

        LEV_OPT_REUSEABLE The port can be repeatedly monitored and reused; (commonly used)

        LEV_OPT_CLOSE_ON_FREE The release of the connection listener will close the underlying socket; (commonly used)

        The LEV_OPT_CLOSE_ON_EXEC connection listener will set close-on-exec for the underlying socket;

        LEV_OPT_LEAVE_SOCKETS_BLOCKING set the file descriptor to block

        LEV_OPT_THREAD_SAFE allocation lock, thread safety

        ......

Parameter five:

        backlog: the number of links that can be monitored at the same time; fill in -1 means that the listener will automatically select an appropriate value; fill in 0 and the listener will think that the listen function has been called.

example:

struct sockaddr_in sin;
memset(&sin, 0, sizeof(struct sockaddr_in));
    
sin.sin_family = AF_INET;
sin.sin_port = htons(9999);
//server_addr.sin_addr.s_addr = inet_addr("127.0.0.1");
    
base = event_base_new();
    
struct evconnlistener *listener = evconnlistener_new_bind(base, listener_cb, base,
                                                             LEV_OPT_REUSEABLE | LEV_OPT_CLOSE_ON_FREE,
                                                             10, (struct sockaddr *)&sin,
                                                             sizeof(struct sockaddr_in));      

        J. evconnlistener_free

void evconnlistener_free(struct evconnlistener *lev);

Release the connection listener;

        K. bufferevent_read

size_t bufferevent_read(struct bufferevent *bufev, void *data, size_t size);

Read data from the input buffer of bufferevent;

bufferevent_read() removes at most size bytes of data from the input buffer and stores it in memory at data . The function returns the number of bytes actually removed.

Note that for bufferevent_read(), the memory block at data must have enough space to hold size bytes of data.

        L. bufferevent_write

int bufferevent_write(struct bufferevent *bufev, const void *data, size_t size);

Add data to bufferevent 's output buffer;

bufferevent_write() adds the size byte data starting from data in memory to the end of the output buffer;

Returns 0 on success and -1 on error;

        M. bufferevent_socket_new

struct bufferevent *bufferevent_socket_new(struct event_base *base, evutil_socket_t fd, enum bufferevent_options options);

base is event_base, options is a bitmask representing bufferevent options (BEV_OPT_CLOSE_ON_FREE, etc.), fd is an optional file descriptor representing the socket. If you want to set the file descriptor later, you can set fd to -1.

BEV_OPT_CLOSE_ON_FREE: Release the bufferevent to automatically close the underlying interface;

BEV_OPT_THREADSAFE : Make bufferevent safe under multi-threading;

The function returns a bufferevent on success and NULL on failure.

example:

// 针对已经存在的socket创建bufferevent对象
// BEV_OPT_CLOSE_ON_FREE:如果释放bufferevent对象,则关闭连接
struct bufferevent *bev = bufferevent_socket_new(base, fd, BEV_OPT_CLOSE_ON_FREE);
// BEV_OPT_CLOSE_ON_FREE:释放bufferevent时关闭底层传输端口。这将关闭底层套接字,释放底层bufferevent等。

        N. bufferevent_setcb

void bufferevent_setcb(struct bufferevent *bufev, bufferevent_data_cb readcb,         bufferevent_data_cb writecb, bufferevent_event_cb eventcb, void *cbarg);

typedef void (*bufferevent_data_cb) (struct bufferevent *bev, void *ctx);        // 读写回调

typedef void (*bufferevent_event_cb) (struct bufferevent *bev, short events, void *ctx);        // 事件回调

The events of the event callback are as follows:

BEV_EVENT_EOF: The peer closes the connection

BEV_EVENT_ERROR: An error occurred

BEV_EVENT_TIMEOUT: timeout

BEV_EVENT_CONNECTED: Connection established successfully

The bufferevent_setcb() function modifies one or more callbacks of bufferevent. The readcb, writecb, and eventcb functions will be called when enough data has been read, enough data has been written, or an error occurs, respectively. The first parameter of each callback function is the bufferevent where the event occurred, and the last parameter is the cbarg parameter provided by the user when calling bufferevent_setcb(): through which data can be passed to the callback.

To disable the callback, pass NULL instead of the callback function. Note: All callback functions of bufferevent share a single cbarg , so modifying it will affect all callback functions.

        O. bufferevent_enable

int bufferevent_enable(struct bufferevent *bufev, short event);

The function internally calls event_add to add the corresponding read and write events to the event listening queue;

event:

        EV_READ: read event

        EV_WRITE: write event 

        EV_PERSIST: persistent

example:

bufferevent_enable(bev, EV_READ | EV_PERSIST);  // evnet_add,使bufferevent 生效

        P. bufferevent_disable

 int bufferevent_disable(struct bufferevent *bufev, short event);

 Delete the corresponding read and write events; event is consistent with bufferevent_enable.

example:

bufferevent_disable(bev, EV_READ);  

        Q. bufferevent_socket_connect

int bufferevent_socket_connect(struct bufferevent *bev, struct sockaddr *serv, int socklen);

The client can use this function to connect to the libevent server!

example:

struct bufferevent *bev = bufferevent_socket_new(base, -1, BEV_OPT_CLOSE_ON_FREE);
int status;
struct sockaddr_in server_addr;
    
memset(&server_addr, 0, sizeof(server_addr));
    
server_addr.sin_family = AF_INET;
server_addr.sin_port = htons(8888);
status = inet_aton("127.0.0.1", &server_addr.sin_addr);
    
if (0 == status) {
    errno = EINVAL;
    return -1;
}
    
// 连接到 服务器IP地址和端口,初始化了socket文件描述符
bufferevent_socket_connect(bev, (struct sockaddr *)&server_addr, sizeof(server_addr));

After that, you can use bufferevent_setcb to set the callback function for bev, and then use bufferevent_enable to enable it.

        R. bufferevent_get_input 、bufferevent_get_output

struct evbuffer *bufferevent_get_input(strucgt bufferevent *bufev);

struct evbuffer *bufferevent_get_output(strucgt bufferevent *bufev); 

Get how many bytes are left in the input buffer and output buffer; use the evbuffer_get_length function to get it;

example:

struct evbuffer *input = bufferevent_get_input(bev);
struct evbuffer *output = bufferevent_get_output(bev);
size_t len1 = evbuffer_get_length(input);
size_t len2 = evbuffer_get_length(output);

        S. Other

// 允许多次绑定同一个地址,要用在socket和bind之间
evutil_make_listen_socket_reuseable(listener);
// 关闭套接字
evutil_closesocket(socket);
// 跨平台统一接口,将套接字设置为非阻塞状态
evutil_make_socket_nonblocking(listener);

3). Echo server case one

#include <stdio.h>
#include <string.h>
#include <errno.h>
#include <unistd.h>
#include <stdlib.h>

#include <event.h>
#include <event2/event.h>

#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <sys/stat.h>
#include <sys/types.h>
#include <fcntl.h>
#include <assert.h>



typedef struct _ConnectStat  ConnectStat;

#define BUFLEN  1024

struct _ConnectStat {
	int fd;
	char send_buf[BUFLEN];
    struct event *ev;
};

//echo 服务实现相关代码
ConnectStat * stat_init(int fd);
void do_echo_handler(int fd, short events,  void *arg);
void do_welcome_handler(int fd, short events,  void *arg);
void do_echo_response(int fd, short events,  void *arg);
void accept_connection(int fd, short events, void *arg);


struct event_base *ev_base;


void usage(const char* argv)
{
	printf("%s:[ip][port]\n", argv);
}

// 设置不阻塞
void set_nonblock(int fd)
{
	int fl = fcntl(fd, F_GETFL);
	fcntl(fd, F_SETFL, fl | O_NONBLOCK);
}

int startup(char* _ip, int _port)  //创建一个套接字,绑定,检测服务器
{
	//sock
	//1.创建套接字
	int sock = socket(AF_INET, SOCK_STREAM, 0);
	if (sock < 0)
	{
		perror("sock");
		exit(2);
	}

    // 端口复用
	int opt = 1;
	setsockopt(sock, SOL_SOCKET, SO_REUSEADDR, &opt, sizeof(opt));

	//2.填充本地 sockaddr_in 结构体(设置本地的IP地址和端口)
	struct sockaddr_in local;
	local.sin_port = htons(_port);
	local.sin_family = AF_INET;
	local.sin_addr.s_addr = inet_addr(_ip);

	//3.bind()绑定
	if (bind(sock, (struct sockaddr*)&local, sizeof(local)) < 0)
	{
		perror("bind");
		exit(3);
	}
    
	//4.listen()监听 检测服务器
	if (listen(sock, 5) < 0)
	{
		perror("listen");
		exit(4);
	}
	
	return sock;    //这样的套接字返回
}


ConnectStat * stat_init(int fd) {
	ConnectStat * temp = NULL;
	temp = (ConnectStat *)malloc(sizeof(ConnectStat));

	if (!temp) {
		fprintf(stderr, "malloc failed. reason: %m\n");
		return NULL;
	}

	memset(temp, '\0', sizeof(ConnectStat));
	temp->fd = fd;
	//temp->status = 0;
}

//void do_welcome_handler(int fd, void  *data) {
void do_welcome_handler(int fd, short events,  void *arg) {
	const char * WELCOME= "Welcome.\n";
	int wlen = strlen(WELCOME);
	int n ;
	ConnectStat * stat = (ConnectStat *)(arg);
	
	if( (n = write(fd, WELCOME, wlen)) != wlen ){
		
		if(n<=0){
		    fprintf(stderr, "write failed[len:%d], reason: %s\n",n,strerror(errno));
		} else fprintf(stderr, "send %d bytes only ,need to send %d bytes.\n",n,wlen);
		
	} else {

        event_set(stat->ev, fd, EV_READ, do_echo_handler, (void *)stat);
        stat->ev->ev_base = ev_base;    // 必须重置事件集合
        event_add(stat->ev, NULL);
    }
}


void do_echo_handler(int fd, short events,  void *arg) {
	ConnectStat * stat = (ConnectStat *)(arg);
	char * p = NULL;
	
	assert(stat!=NULL);
	
	
	p = stat->send_buf;
	*p++ = '-';
	*p++ = '>';
	ssize_t _s = read(fd, p, BUFLEN - (p - stat->send_buf) - 1); //2字节"->" +字符结束符.
    if (_s > 0)
    {
		
		*(p+_s) = '\0'; // p[_s] = '\0';
		printf("receive from client: %s", p);
	
		if(!strncasecmp(p, "quit", 4)){//退出.
            event_free(stat->ev);   // 自动解除监听事件,再释放event
            close(fd);
            free(stat);
			return ;
		}

        event_set(stat->ev, fd, EV_WRITE, do_echo_response, (void *)stat);
        stat->ev->ev_base = ev_base;    // 必须重置事件集合
        event_add(stat->ev, NULL);
        
        
	}else if (_s == 0)  //client:close
    {
        fprintf(stderr,"Remote connection[fd: %d] has been closed\n", fd);
        event_free(stat->ev);   // 自动解除监听事件,再释放event
        close(fd);
        free(stat);
    }
    else //err occurred.
    {
        fprintf(stderr,"read faield[fd: %d], reason:%s [%ld]\n",fd , strerror(errno), _s);
    }
}

void do_echo_response(int fd, short events,  void *arg) {
	ConnectStat * stat = (ConnectStat *)(arg);
	int len = strlen(stat->send_buf);
    
	int _s = write(fd, stat->send_buf, len);
    memset(stat->send_buf, '\0', len);
	
	if(_s>0){
		event_set(stat->ev, fd, EV_READ, do_echo_handler, (void *)stat);
        stat->ev->ev_base = ev_base;    // 必须重置事件集合
        event_add(stat->ev, NULL);
		
	}else if(_s==0){
		fprintf(stderr,"Remote connection[fd: %d] has been closed\n", fd);
        event_free(stat->ev);   // 自动解除监听事件,再释放event
        close(fd);
        free(stat);
	}else {
		fprintf(stderr,"read faield[fd: %d], reason:%s [%d]\n",fd ,strerror(errno), _s);
	}
}

//read()
//注册写事件
  
//写事件就绪
//write()

void accept_connection(int fd, short events, void *arg) {
	struct sockaddr_in peer;
	socklen_t len = sizeof(peer);
	
	int new_fd = accept(fd, (struct sockaddr*)&peer, &len);

	if (new_fd > 0)
	{
		ConnectStat *stat = stat_init(new_fd);
		set_nonblock(new_fd);

		printf("new client: %s:%d\n", inet_ntoa(peer.sin_addr), ntohs(peer.sin_port));
        struct event *ev = event_new(ev_base, new_fd, EV_WRITE, do_welcome_handler, (void *)stat);
        stat->ev = ev;
        event_add(ev, NULL);
	} else {
        fprintf(stderr, "error!\n");
    }
}


int main(int argc,char **argv){

	if (argc != 3)     //检测参数个数是否正确
	{
		usage(argv[0]);
		exit(1);
	}

	int listen_sock = startup(argv[1], atoi(argv[2]));      //创建一个绑定了本地 ip 和端口号的套接字描述符
	//初始化异步事件处理框架epoll
	
    ev_base = event_base_new();
	
	//ConnectStat * stat = stat_init(listen_sock);
    struct event *ev_listen = event_new(ev_base, listen_sock, EV_READ|EV_PERSIST, accept_connection, (void*)ev_base);
    event_add(ev_listen, NULL);
    


    event_base_dispatch(ev_base);
    event_base_free(ev_base);

    return 0;
}

4). Echo server case 2

client.c

#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <errno.h>
#include <unistd.h>

#include <stdio.h>
#include <string.h>
#include <stdlib.h>

#include <event.h>
#include <event2/util.h>


int connect_server(const char *server_ip, int port);
void cmd_read_data(int fd, short events, void *arg);
void socket_read_data(int fd, short events, void *arg);


int main(int argc, char **argv) {
    
    if (argc < 3) {
        printf("please input 2 parameters!\n");
        return -1;
    }
    
    // 两个参数依次是服务器的IP地址和端口号
    int sockfd = connect_server(argv[1], atoi(argv[2]));
    if (-1 == sockfd) {
        perror("tcp_connect error!");
        return -1;
    }
    
    printf("connect to server successfully\n");
    
    
    struct event_base *base = event_base_new();
    
    // 监听服务端发送的消息
    struct event *ev_sockfd = event_new(base, sockfd, EV_READ | EV_PERSIST, socket_read_data, NULL);
    event_add(ev_sockfd, NULL);
    
    // 监听终端输入事件
    struct event *ev_cmd = event_new(base, STDIN_FILENO, EV_READ | EV_PERSIST, cmd_read_data, (void *)&sockfd);
    event_add(ev_cmd, NULL);
    
    // 事件循环
    event_base_dispatch(base);
    
    event_base_free(base);
    
    printf("finished\n");
    
    return 0;
}


void cmd_read_data(int fd, short events, void *arg) {
    char msg[1024] = { '\0' };
    
    int ret = read(fd, msg, sizeof(msg) - 1);
    if (0 == ret) {
        printf("connection close. exit!\n");
        exit(1);
    }
    if (ret < 0) {
        perror("read failed!");
        exit (1);
    }
    
    int sockfd = *((int *)arg);
    
    if (msg[ret - 1] == '\n') {
        msg[ret - 1] = '\0';
    } else {
        msg[ret] = '\0';
    }
    
    /*
    int count = 1000;
    while (count-- > 0) {
        write(sockfd, msg, ret);
        
        int len = read(sockfd, msg, sizeof(msg) - 1);
        msg[len] = '\0';
    
        printf("recv from server <<< %s\n", msg);
        
        usleep(10000);
    }
    */
    
    // 把终端的消息发送给服务器端,客户端忽略性能考虑,直接利用阻塞方式发送
    //printf("write to server >>> %s\n", msg);
    ret = write(sockfd, msg, ret);
    if (ret == -1) {
        perror("write to server failed!");
        exit(1);
    }
    
    if (strncmp(msg, "exit", 4) == 0) {   
        memset(msg, 0, sizeof(msg));
        write(sockfd, msg, sizeof(msg));
        usleep(100000); // 100ms
        close(sockfd);
        exit(1);
    }

}


void socket_read_data(int fd, short events, void *arg) {
    char msg[1024] = { '\0' };
    
    // 不考虑一次读不完数据的情况
    int len = read(fd, msg, sizeof(msg) - 1);
    if (0 == len) {
        printf("connection close. exit!\n");
        exit(1);
    } else if (len < 0) {
        perror("read failed!");
        return ;
    }
    
    msg[len] = '\0';
    
    printf("recv from server <<< %s\n", msg);
}


typedef struct sockaddr SA;
int connect_server(const char *server_ip, int port) {
    int sockfd, status, save_errno;
    struct sockaddr_in server_addr;
    
    memset(&server_addr, 0, sizeof(server_addr));
    
    server_addr.sin_family = AF_INET;
    server_addr.sin_port = htons(port);
    status = inet_aton(server_ip, &server_addr.sin_addr);
    
    if (0 == status) {
        errno = EINVAL;
        return -1;
    }
    
    sockfd = socket(PF_INET, SOCK_STREAM, 0);
    
    status = connect(sockfd, (SA *)&server_addr, sizeof(server_addr));
    if (-1 == status) {
        save_errno = errno;
        close(sockfd);
        errno = save_errno;     // the close may be error
        return -1;
    }
    
    return sockfd;
}

server.c

#include <stdio.h>
#include <string.h>
#include <errno.h>
#include <stdlib.h>

#include <unistd.h>
#include <event.h>
#include <event2/event.h>
#include <assert.h>


#define BUFLEN 1024


typedef struct _ConnectStat {
    struct event *ev;
    char buf[BUFLEN];
}ConnectStat;

// echo 服务实现相关代码
ConnectStat *stat_init(struct event *ev);
void accept_connection(int fd, short events, void *arg);
void do_echo_request(int fd, short events, void *arg);
void do_echo_response(int fd, short events, void *arg);
int tcp_server_init(int port, int listen_num);

struct event_base *base;



int main(int argc, char **argv) {
    int listener = tcp_server_init(9999, 1024);
    if (-1 == listener) {
        perror("tcp_server_init error");
        return -1;
    }
    
    base = event_base_new();
    
    // 添加监听客户端请求链接事件
    struct event *ev_listen = event_new(base, listener, EV_READ | EV_PERSIST, accept_connection, base);
    event_add(ev_listen, NULL);
    
    event_base_dispatch(base);
    
    event_base_free(base);
    
    return 0;
}




ConnectStat *stat_init(struct event *ev) {

    ConnectStat *temp =  (ConnectStat *)malloc(sizeof(ConnectStat));
    if (!temp) {
        fprintf(stderr, "malloc failed. reason: %s\n", strerror(errno));
        return NULL;
    }
    
    memset(temp, '\0', sizeof(ConnectStat));
    temp->ev = ev;
    
    return temp;
}







void accept_connection(int fd, short events, void *arg) {

    evutil_socket_t sockfd;

    struct sockaddr_in client;
    socklen_t len = sizeof(client);

    sockfd = accept(fd, (struct sockaddr *)&client, &len);
    if (-1 == sockfd) {
        fprintf(stderr, "accept failed! reason: %s\n", strerror(errno));
        sleep(1);
        return ;
    }


    evutil_make_socket_nonblocking(sockfd);   // 设置为非阻塞
    
    char server_ip[64];
    inet_ntop(AF_INET, &client.sin_addr.s_addr, server_ip, sizeof(server_ip));
    printf("new client: %s:%u\n", server_ip, ntohs(client.sin_port));

    struct event_base *base = (struct event_base *)arg;
    
    // 仅仅是为了动态创建一个event结构体
    struct event *ev = event_new (NULL, -1, 0, NULL, NULL);
    
    ConnectStat *stat = stat_init(ev);
    
    //将动态创建的结构体作为event的回调参数
    event_assign(ev, base, sockfd, EV_READ , do_echo_request, (void*)stat);
 
    event_add(ev, NULL);
}





void do_echo_request(int fd, short events, void *arg) {

    ConnectStat *stat = (ConnectStat *)arg;
    struct event *ev = stat->ev;
    char *msg = stat->buf;
    
    //printf("do echo request ...\n");
    
    int len = read(fd, msg, BUFLEN - 1);
    if (0 == len) {
        fprintf(stdin, "connection break !\n");
        event_free(ev);
        close(fd);
        free(stat);
        return ;
    } else if (len < 0) {
        fprintf(stderr, "read failed! reason: %s\n", strerror(errno));
        return ;
    }
    
    msg[len] = '\0';
    printf("recv from client <<< %s\n", msg);
    
    event_set(ev, fd, EV_WRITE, do_echo_response, (void *)stat);
    ev->ev_base = base;
    event_add(ev, NULL);
}





void do_echo_response(int fd, short events, void *arg) {

    ConnectStat *stat = (ConnectStat *)(arg);
    struct event *ev = NULL;
    
    assert(stat != NULL);
    
    ev = stat->ev;
    
    int len = strlen(stat->buf);
    printf("write to client >>> %s\n", stat->buf);
    
    int _s = write(fd, stat->buf, len);
    if (_s > 0) {
        //printf("write successfully.\n");
    } else if (0 == _s) {   // client close
        fprintf(stderr, "remote connection[fd: %d] has been closed \n", fd);
        event_free(ev);
        close(fd);
        free(stat);
        return ;
    } else {
        fprintf(stderr, "write failed[fd: %d], reason: %s[%d]\n", fd, strerror(errno), _s);
        //event_free(ev);
        //close(fd);
        //free(stat);
        return ;
    }
    
    event_set(ev, fd, EV_READ, do_echo_request, (void *)stat);
    ev->ev_base = base;
    event_add(ev, NULL);
    
    //event_assign(ev, base, fd, EV_READ , do_echo_request, (void*)stat);
}




typedef struct sockaddr SA;
int tcp_server_init(int port, int listen_num) {

    int errno_save;
    evutil_socket_t listener;   // int listener;
    
    listener = socket(AF_INET, SOCK_STREAM, 0);
    if (-1 == listener) {
        return -1;
    }
    
    // 允许多次绑定同一个地址,要用在socket和bind之间
    evutil_make_listen_socket_reuseable(listener);
    
    struct sockaddr_in sin;
    sin.sin_family = AF_INET;
    sin.sin_addr.s_addr = 0;
    sin.sin_port = htons(port);
    
    if (bind(listener, (SA *)&sin, sizeof(sin)) < 0) {
        errno_save = errno;
        evutil_closesocket(listener);
        errno = errno_save;
        return -1;
    }
    
    if(listen(listener, listen_num) < 0) {
        errno_save = errno;
        evutil_closesocket(listener);
        errno = errno_save;
        return -1;
    }
    
    // 跨平台统一接口,将套接字设置为非阻塞状态
    evutil_make_socket_nonblocking(listener);
    
    return listener;
}

5). Echo server case with caching

#include <stdio.h>
#include <string.h>
#include <errno.h>
#include <stdlib.h>
#include <unistd.h>
#include <event.h>
#include <event2/event.h>
#include <event2/listener.h>
#include <assert.h>
#include <arpa/inet.h>

#define BUFLEN  1024


typedef struct _ConnectStat {
    struct bufferevent *bev;
    char buf[BUFLEN];
}ConnectStat;


ConnectStat *stat_init(struct bufferevent *bev);
void do_echo_request(struct bufferevent *bev, void *arg);           // 读数据
void do_echo_response(struct bufferevent *bev, void *arg);          // 写数据
void event_cb(struct bufferevent *bev, short events, void *arg);     // 出错处理函数
int tcp_server_init(int port, int listen_num);
void listener_cb(struct evconnlistener *listener, evutil_socket_t fd, struct sockaddr *sock, int socklen, void *arg);   // 监听函数



struct event_base *base;

int main(int argc, char **argv) {
 
    struct sockaddr_in sin;
    memset(&sin, 0, sizeof(struct sockaddr_in));
    
    sin.sin_family = AF_INET;
    sin.sin_port = htons(8000);
    //server_addr.sin_addr.s_addr = inet_addr("127.0.0.1");
    
    base = event_base_new();
    
    
    // 创建socket,绑定、监听、接受链接
    // 创建监听对象,在指定的地址上监听接下来的TCP连接
    // listen、connect、bind、accept;  LEV_OPT_REUSEABLE:可重用,LEV_OPT_CLOSE_ON_FREE:自动关闭
    struct evconnlistener *listener = evconnlistener_new_bind(base, listener_cb, base,
                                                             LEV_OPT_REUSEABLE | LEV_OPT_CLOSE_ON_FREE,
                                                             1024, (struct sockaddr *)&sin,
                                                             sizeof(struct sockaddr_in));
                                                             
    // 监听集合中的事件
    event_base_dispatch(base);
    
    // 释放
    evconnlistener_free(listener);
    event_base_free(base);
    
    return 0;
}


ConnectStat *stat_init(struct bufferevent *bev) {
    ConnectStat *temp = NULL;
    temp = (ConnectStat *)malloc(sizeof(ConnectStat));
    
    if (!temp) {
        fprintf(stderr, "malloc failed. reason: %s\n", strerror(errno));
        return NULL;
    }
    
    memset(temp, '\0', sizeof(ConnectStat));
    temp->bev = bev;
    
    return temp;
}


void do_echo_request(struct bufferevent *bev, void *arg) {
    ConnectStat *stat = (ConnectStat *)arg;
    char *msg = stat->buf;
    
    // 从缓冲区中获取数据
    size_t len = bufferevent_read(bev, msg, BUFLEN);
    msg[len] = '\0';
    
    printf("recv from client <<< %s\n", msg);
    
    // 将数据添加到缓冲区
    bufferevent_write(bev, msg, strlen(msg));
}


void do_echo_response(struct bufferevent *bev, void *arg) {
    return ;
}

void event_cb(struct bufferevent *bev, short events, void *arg) {
    ConnectStat *stat = (ConnectStat *)arg;  
        
    // 获取输入缓冲区和输出缓冲区还剩多少字节
    struct evbuffer *input = bufferevent_get_input(bev);
    struct evbuffer *output = bufferevent_get_output(bev);
    int finished = 0;
    
    if (events & BEV_EVENT_EOF) {
        size_t len1 = evbuffer_get_length(input);
        size_t len2 = evbuffer_get_length(output);
        printf("Input buffer have %lu data.  Output buffer have %lu data.\n", (unsigned long)len1, (unsigned long)len2);
        finished = 1;
        
        printf("connect cloase\n");
    
    } else if (events & BEV_EVENT_ERROR) {
        finished = 1;       
        printf("connect cloase\n");
    }
    
    if (finished) {
        // 自动close套接字和free读写缓冲区
        bufferevent_free(bev);// 释放bufferevent对象
        free(stat);
    }
}


typedef struct sockaddr SA;
int tcp_server_init(int port, int listen_num) {
    int errno_save;
    evutil_socket_t listener;   // int listener
    
    listener = socket(AF_INET, SOCK_STREAM, 0);
    if (-1 == listener) {
        return -1;
    }
    
    // 允许多次绑定同一个地址,要用在socket和bind之间
    evutil_make_listen_socket_reuseable(listener);
    
    struct sockaddr_in sin;
    sin.sin_family = AF_INET;
    sin.sin_addr.s_addr = 0;
    sin.sin_port = htons(port);
    
    if (bind(listener, (SA *)&sin, sizeof(sin)) < 0) {
        errno_save = errno;
        evutil_closesocket(listener);
        errno = errno_save;
        
        return -1;
    }
    
    if (listen(listener, listen_num) < 0) {
        errno_save = errno;
        evutil_closesocket(listener);
        errno = errno_save;
        
        return -1;
    }
    
    // 跨平台统一接口,将套接字设置为非阻塞状态
    evutil_make_socket_nonblocking(listener);
    
    return listener;
}


// 一个客户端连接上服务器此函数就会被调用;当此函数被调用时,libevent已经帮我们accept了这个客户端
void listener_cb(struct evconnlistener *listener, evutil_socket_t fd, struct sockaddr *sock, int socklen, void *arg) {
    printf("accept a client %d \n", fd);
    
    struct event_base *base = (struct event_base *)arg;
    
    // 针对已经存在的socket创建bufferevent对象
    // BEV_OPT_CLOSE_ON_FREE:如果释放bufferevent对象,则关闭连接
    struct bufferevent *bev = bufferevent_socket_new(base, fd, BEV_OPT_CLOSE_ON_FREE);
    // BEV_OPT_CLOSE_ON_FREE:释放bufferevent时关闭底层传输端口。这将关闭底层套接字,释放底层bufferevent等。
    
    ConnectStat *stat = stat_init(bev);
    
    // 给bufferevent设置回调函数
    // bufferevent对象、读事件回调函数、写事件回调函数、其他事件回调函数、参数
    bufferevent_setcb(bev, do_echo_request, do_echo_response, event_cb, stat);  // evnet_set
    
    bufferevent_enable(bev, EV_READ | EV_PERSIST);  // evnet_add,使bufferevent 生效
    
    
    bufferevent_write(bev, "hellowrold", strlen("Welcome"));
}

The client can use the above one to test, or use the client in the sixth point below to test!

6). Client case with caching

#include <sys/types.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <errno.h>
#include <unistd.h>

#include <stdio.h>
#include <string.h>
#include <stdlib.h>

#include <event.h>
#include <event2/util.h>
#include <event2/event.h>
#include <event2/listener.h>


int connect_server(const char *server_ip, int port, struct bufferevent *bev);
void cmd_read_data(int fd, short events, void *arg);
void do_read_msg(struct bufferevent *bev, void *arg);
void event_cb(struct bufferevent *bev, short events, void *arg);

int main(int argc, char **argv) {
    
    if (argc < 3) {
        printf("please input 2 parameters!\n");
        return -1;
    }
    
    struct event_base *base = event_base_new();
    
    // 创建并且初始化buffer缓冲区
    struct bufferevent *bev = bufferevent_socket_new(base, -1, BEV_OPT_CLOSE_ON_FREE);
    
    // 两个参数依次是服务器的IP地址和端口号
    int result = connect_server(argv[1], atoi(argv[2]), bev);
    if (-1 == result) {
        perror("tcp_connect error!");
        return -1;
    }
    
    
    // 监听终端输入事件
    struct event *ev_cmd = event_new(base, STDIN_FILENO, EV_READ | EV_PERSIST, cmd_read_data, (void *)bev);
    event_add(ev_cmd, NULL);
    
    
    
    // 事件循环
    event_base_dispatch(base);
    
    event_base_free(base);
    
    printf("finished\n");
    
    return 0;
}


void cmd_read_data(int fd, short events, void *arg) {
    char msg[1024] = { '\0' };
    
    int ret = read(fd, msg, sizeof(msg) - 1);
    if (0 == ret) {
        printf("connection close. exit!\n");
        exit(1);
    }
    if (ret < 0) {
        perror("read failed!");
        exit (1);
    }
    
    struct bufferevent *bev = (struct bufferevent *)arg;
    
    if (msg[ret - 1] == '\n') {
        msg[ret - 1] = '\0';
    } else {
        msg[ret] = '\0';
    }
    
    // 把终端的消息发送给服务器端
    // 将数据添加到写缓冲区
    bufferevent_write(bev, msg, strlen(msg));
    
    if (strncmp(msg, "exit", 4) == 0) {   
        memset(msg, 0, sizeof(msg));

        // 自动close套接字和free读写缓冲区
        bufferevent_free(bev);// 释放bufferevent对象

        usleep(100000); // 100ms
        exit(1);
    }
}


typedef struct sockaddr SA;
int connect_server(const char *server_ip, int port, struct bufferevent *bev) {
    int status;
    struct sockaddr_in server_addr;
    
    memset(&server_addr, 0, sizeof(server_addr));
    
    server_addr.sin_family = AF_INET;
    server_addr.sin_port = htons(port);
    status = inet_aton(server_ip, &server_addr.sin_addr);
    
    if (0 == status) {
        errno = EINVAL;
        return -1;
    }
    
    // 连接到 服务器IP地址和端口,初始化了socket文件描述符
    bufferevent_socket_connect(bev, (struct sockaddr *)&server_addr, sizeof(server_addr));
    
    // 设置buffer的回调函数
    bufferevent_setcb(bev, do_read_msg, NULL, event_cb, NULL);
    bufferevent_enable(bev, EV_READ | EV_PERSIST);  // evnet_add,使bufferevent 生效
    
    return 0;
}

void do_read_msg(struct bufferevent *bev, void *arg) {
    char msg[1024] = { '\0' };
    int msg_len = sizeof(msg);
    
    // 从缓冲区中获取数据
    size_t len = bufferevent_read(bev, msg, msg_len);
    msg[len] = '\0';
    
    printf("recv from client <<< %s\n", msg);
}


void event_cb(struct bufferevent *bev, short events, void *arg) {
    
    struct evbuffer *input = bufferevent_get_input(bev);
    struct evbuffer *output = bufferevent_get_output(bev);
    int finished = 0;
    
    if (events & BEV_EVENT_EOF) {
        size_t len1 = evbuffer_get_length(input);
        size_t len2 = evbuffer_get_length(output);
        printf("Input buffer have %lu data.  Output buffer have %lu data.\n", (unsigned long)len1, (unsigned long)len2);
        finished = 1;
        
        printf("connect cloase\n");
    
    } else if (events & BEV_EVENT_ERROR) {
        finished = 1;       
        printf("connect cloase\n");
    
    } else if (events & BEV_EVENT_CONNECTED) {
        printf("The client has connected to server.\n");
    }
    
    if (finished) {
        // 自动close套接字和free读写缓冲区
        bufferevent_free(bev);// 释放bufferevent对象
    }
}

You can use the server in point 5 above to test!

7). Libevent monitoring signal case

Listen to the Ctrl + C signal here 

#include <stdio.h>
#include <signal.h>
#include <event.h>


int signal_count = 0;

void signal_handler(evutil_socket_t fd, short events, void *arg) {
    printf("收到信号 %d\n", fd);
    
    struct event *ev = (struct event *)arg;
    
    signal_count++;
    if (signal_count >= 2) {
        // 把事件从集合中删除
        event_del(ev);
    }
}


int main(int argc, char **argv) {
    // 创建事件集合
    struct event_base *base = event_base_new();
    
    // 创建事件
    struct event ev;
    // 把事件和信号绑定
    event_assign(&ev, base, SIGINT, EV_SIGNAL | EV_PERSIST, signal_handler, &ev);   // 监听ctrl+c信号
    
    // 事件添加到集合中
    event_add(&ev, NULL);
    
    // 事件集合循环,当集合中没有事件了,此函数结束返回
    event_base_dispatch(base);
    
    
    // 释放集合
    event_base_free(base);
    
    return 0;
}


4. Signal driven IO (not introduced)

When using signal-driven I/O, when the network socket is readable, the kernel notifies the application process by sending a SIGIO signal, so the application can start reading data. This method is not asynchronous I/O, because the actual work of reading data into the application process cache is still the responsibility of the application itself. 

......


5. Asynchronous IO (not introduced)

When the user process initiates a read operation, after the kernel receives the read operation, it will return immediately, so it will not block the user process, and then it will wait for the data preparation to be completed, and then copy the data to the user memory, after completion , which sends a signal to the user process, telling the user process that the read operation is complete.

......


6. Summary

So far, the more commonly used IO modes have been introduced, and the knowledge points and codes have been issued;

To be honest, I don’t know how to use these things in specific projects, after all, I have never been in contact with enterprise projects developed by servers;

Accumulate the knowledge points first, and if you have the opportunity to enter the Linux server development position in the future, you will also have the corresponding knowledge support!

Guess you like

Origin blog.csdn.net/cpp_learner/article/details/129431294