Linux High Concurrency Server-Part 2

• See the blog for part one

Data link layer protocol-Ethernet frame protocol-MAC address encapsulation

Network interface layer protocol-ARP protocol

encapsulation

• The transport layer adds TCP/UDP data to the header, then adds the application data together for encapsulation, and so on for other layers.

Divide

The process of network communication

• If QQ on one computer wants to send a message to QQ on another computer, the application layer contains the message data and the protocol header of QQ. These two are encapsulated and then transmitted down.
• If UDP is used for transmission, a 16-bit source port number, a 16-bit destination port number, a 16-bit UDP length, a 16-bit UDP checksum and data are required, so these must be added on the basis of the upper application layer.
• The content encapsulated in the upper layer is used as the data of the next layer. The network layer uses the IP protocol. Data such as version number, header length, IP address (source and destination) need to be added, forming IP header + IP data. report format
• Then passed to the data link layer, through the Ethernet frame protocol, plus MAC header, CRC check, etc.

• Check whether the MAC address in the message is consistent with the destination MAC address. If they are consistent, receive the message, and so on from bottom to top.
• Search based on MAC address. How to find MAC address? The ARP protocol is used
• The ARP protocol finds the MAC address based on the IP
• ARP packet occupies 28 bytes
• Also add the Ethernet Frame Protocol
• Q: Do I need to change the IP address to a broadcast address when sending arp broadcast?
• A: "When the host sends information, it broadcasts the ARP request containing the target IP address to all hosts on the LAN and receives the return message to determine the target's physical address; after receiving the return message, it stores the IP address and physical address. Enter the local ARP cache and keep it for a certain period of time. The next request will directly query the ARP cache to save resources." The IP address mentioned above is the IP address that the ARP protocol will search for. It has nothing to do with the broadcast address sent. Don't get confused.

Socket communication

• Inter-communication is also implemented, but it needs to be on different hosts.
• A socket is a combination of IP address and port, and also includes the API of the protocol stack.
  

Introduction

• Socket is end-to-end communication. There is no need to consider the network layer, data link layer, etc. In the past, when transmitting between each layer, the data header of the layer must be added. Now socket directly includes the entire
• I don’t care about how to decapsulate the bottom layer of the socket.

Endianness

• The accumulator of a modern CPU can load (at least) 4 bytes at a time (considering 32-bit machines here), that is, an integer. Then the order in which these 4 bytes are arranged in memory will affect the value of the integer loaded into the accumulator. This is a byte order problem. In various computer architectures, the storage mechanisms for bytes, words, etc. are different, which leads to a very important issue in the field of computer communication, that is, the information units (bits, bytes, words, doubles) exchanged by the communicating parties. words, etc.) should be transmitted in what order. If agreed rules are not reached, the communicating parties will not be able to perform correct encoding/decoding, resulting in communication failure.
• Byte order, as the name suggests, is the order in which data larger than one byte is stored in memory (of course there is no need to talk about the order of data of one byte)
• Byte order is divided into Big-Endian and Little-Endian. Big endian byte order means that the highest byte of an integer (23 _{31bit) is stored at the low address of the memory, and the low byte (0} 7bt) is stored at the high address of the memory; little endian byte order refers to the integer The high-order byte is stored at the high address of the memory, and the low-order byte is stored at the low address of the memory.
  

big endian little endian

/*  
字节序：字节在内存中存储的顺序。
小端字节序：数据的高位字节存储在内存的高位地址，低位字节存储在内存的低位地址
大端字节序：数据的低位字节存储在内存的高位地址，高位字节存储在内存的低位地址
*/

// 通过代码检测当前主机的字节序
#include <stdio.h>

int main() {
    
    

    union {
    
    
    short value;    // 2字节
    char bytes[sizeof(short)];  // char[2]
} test;

    test.value = 0x0102;
    if((test.bytes[0] == 1) && (test.bytes[1] == 2)) {
    
    
        printf("大端字节序\n");
    } else if((test.bytes[0] == 2) && (test.bytes[1] == 1)) {
    
    
        printf("小端字节序\n");
    } else {
    
    
        printf("未知\n");
    }

    return 0;
}

• Little endian byte order: the high bits of the data are stored in the high bits of the memory
• Big endian: the high bits of data are stored in the low bits of memory
  

Byte order conversion

• When formatted data is passed directly between two hosts using different endianness, the receiving end must interpret it incorrectly. The way to solve the problem is: the sending end always converts the data to be sent into big-endian byte order data before sending it, and the receiving end knows that the data sent by the other party is always in big-endian byte order, so the receiving end can The byte order adopted by itself determines whether to convert the received data (little endian machine converts, big endian machine does not convert)
• Network byte order is a data representation format specified in TCP/IP. It has nothing to do with specific CPU types, operating systems, etc., thus ensuring that data can be correctly interpreted when transmitted between different hosts. Network byte order Use big-endian sorting
• BSD Socket provides an encapsulated conversion interface for programmer convenience. Including conversion functions from host byte order to network byte order: htons, htonl; conversion functions from network byte order to host byte order: ntohs, ntohl

/*

    网络通信时，需要将主机字节序转换成网络字节序（大端字节序），
	（主机字节序可能是小端，，也可能是大端）
    另外一段获取到数据以后根据情况将网络字节序转换成主机字节序。

    // 转换端口
    uint16_t htons(uint16_t hostshort);		// 主机字节序 - 网络字节序
    uint16_t ntohs(uint16_t netshort);		// 主机字节序 - 网络字节序

    // 转IP
    uint32_t htonl(uint32_t hostlong);		// 主机字节序 - 网络字节序
    uint32_t ntohl(uint32_t netlong);		// 主机字节序 - 网络字节序

*/

#include <stdio.h>
#include <arpa/inet.h>

int main() {
    
    

    // htons 转换端口
    unsigned short a = 0x0102;
    printf("a : %x\n", a);
    unsigned short b = htons(a);
    printf("b : %x\n", b);

    printf("=======================\n");

    // htonl  转换IP
    char buf[4] = {
    
    192, 168, 1, 100};
    int num = *(int *)buf;
    int sum = htonl(num);
    unsigned char *p = (char *)&sum;

    printf("%d %d %d %d\n", *p, *(p+1), *(p+2), *(p+3));

    printf("=======================\n");

    // ntohl
    unsigned char buf1[4] = {
    
    1, 1, 168, 192};
    int num1 = *(int *)buf1;
    int sum1 = ntohl(num1);
    unsigned char *p1 = (unsigned char *)&sum1;
    printf("%d %d %d %d\n", *p1, *(p1+1), *(p1+2), *(p1+3));
    
     // ntohs


    return 0;
}

Socket address

• // The socket address is actually a structure that encapsulates information such as port number and IP. This socket address needs to be used in subsequent socket-related APIs.
• // Client->Server (IP, Port)
  

Universal Socket address

• In the socket network programming interface, the socket address is represented by the structure sockaddr, which is defined as follows:

#include <bits/socket.h> struct sockaddr {
      
      
	sa_family_t sa_family; 
	char sa_data[14];//只用了6个字节来表示IPv4的地址数据，2+4,2位端口号，4位为IP，剩下8个空在那里
};

typedef unsigned short int sa_family_t;

• The sa_family member is a variable of address family type (sa_family_t). Address family types usually correspond to protocol family types. Common protocol families (also called domain) and corresponding address families are as follows:
  | Protocol Family | Address Family | Description |
  | — | — | — |
  | PF_UNIX | AF_UNIX | UNIX local domain protocol family |
  | PF_INET | AF_INET | TCP/IPv4 protocol suite |
  | PF_INET6 | AF_INET6 | TCP/IPv6 protocol suite|

• The macros PF_* and AF_* are both defined in the bits/socket.h header file, and the latter has exactly the same value as the former, so the two are usually mixed.

• The sa_data member is used to store the socket address value. However, the address values ​​of different protocol families have different meanings and lengths, as shown below:
  | Protocol family | Address value meaning and length |
  | — | — |
  | PF_UNIX | The path name of the file, the length can reach 108 bytes |
  | PF_INET | 16bit port number and 32bit IPv4 address, 6 bytes in total |
  | PF_INET6 | 16bit port number, 32bit flow identifier, 128bit IPv6 address, 32bit range ID, 26 bytes in total |

• As can be seen from the above table, the 14-byte sa_data simply cannot accommodate the address values ​​of most protocol families. Therefore, Linux defines the following new universal socket address structure. This structure not only provides enough space to store address values, but is also memory aligned.

#include <bits/socket.h> struct sockaddr_storage
{
    
    
	sa_family_t sa_family; 
	unsigned long int   ss_align;
	char ss_padding[ 128 - sizeof(__ss_align) ];
};


typedef unsigned short int sa_family_t;

Private Socket Address

• Many network programming functions were born earlier than the IPv4 protocol. At that time, they all used the structsockaddr structure. For forward compatibility, now sockaddr has degenerated into the role of (void *), passing an address to the function. As for whether this function is sockaddr_in or sockaddr_in6 , determined by the address family, and then the function is forced to convert the type into the required address type internally.

• The UNIX local domain protocol family uses the following dedicated socket address structure:

#include <sys/un.h> struct sockaddr_un
{
    
    
sa_family_t sin_family; char sun_path[108];
};

• The TCP/IP protocol family has two dedicated socket address structures, sockaddr_in and sockaddr_in6, which are used for IPv4 and IPv6 respectively:

• All special socket address (and sockaddr_storage) type variables need to be converted to the general socket address type sockaddr when actually used (just force conversion), because the address parameter type used by all socket programming interfaces is sockaddr.

IP address conversion (string IP-integer, host, network byte order conversion)

• IP addresses are expressed in dotted decimal notation
• Usually, people are accustomed to using readable strings to represent IP addresses, such as dotted decimal strings to represent IPv4 addresses, and hexadecimal strings to represent IPv6 addresses. But in programming, we need to convert them into integers (binary numbers) before they can be used. On the contrary, when recording logs, we need to convert the IP address represented by an integer into a readable string. The following three functions can be used to convert between IPv4 addresses represented as dotted decimal strings and IPv4 addresses represented as network byte order integers:

#include <arpa/inet.h>
in_addr_t inet_addr(const char *cp);
int inet_aton(const char *cp, struct in_addr *inp); 
char *inet_ntoa(struct in_addr in);

• The following pair of updated functions can also perform the same functions as the previous three functions, and they work with both IPv4 addresses and IPv6 addresses:

#include <arpa/inet.h>
// p:点分十进制的IP字符串，n:表示network，网络字节序的整数 
//将点分十进制的IP地址转换为网络字节序的整数
int inet_pton(int af, const char *src, void *dst);
af:地址族： AF_INET AF_INET6 
src:需要转换的点分十进制的IP字符串 
dst:转换后的结果保存在这个里面，传出参数

//  将网络字节序的整数，转换成点分十进制的IP地址字符串
const char *inet_ntop(int af, const void *src, char *dst, socklen_t size); 
af:地址族： AF_INET AF_INET6
src: 要转换的IP的整数的地址
dst: 转换成IP地址字符串保存的地方
size：第三个参数的大小（数组的大小）
返回值：返回转换后的数据的地址（字符串），和 dst 是一样的

/*
    #include <arpa/inet.h>
    // p:点分十进制的IP字符串，n:表示network，网络字节序的整数
    int inet_pton(int af, const char *src, void *dst);
        af:地址族： AF_INET  AF_INET6
        src:需要转换的点分十进制的IP字符串
        dst:转换后的结果保存在这个里面

    // 将网络字节序的整数，转换成点分十进制的IP地址字符串
    const char *inet_ntop(int af, const void *src, char *dst, socklen_t size);
        af:地址族： AF_INET  AF_INET6
        src: 要转换的ip的整数的地址
        dst: 转换成IP地址字符串保存的地方
        size：第三个参数的大小（数组的大小）
        返回值：返回转换后的数据的地址（字符串），和 dst 是一样的

*/

#include <stdio.h>
#include <arpa/inet.h>

int main() {
    
    

    // 创建一个ip字符串,点分十进制的IP地址字符串
    char buf[] = "192.168.1.4";
    unsigned int num = 0;

    // 将点分十进制的IP字符串转换成网络字节序的整数
    inet_pton(AF_INET, buf, &num);
    unsigned char * p = (unsigned char *)&num;
    printf("%d %d %d %d\n", *p, *(p+1), *(p+2), *(p+3));


    // 将网络字节序的IP整数转换成点分十进制的IP字符串
    char ip[16] = "";
    const char * str =  inet_ntop(AF_INET, &num, ip, 16);
    printf("str : %s\n", str);
    printf("ip : %s\n", str);
    printf("%d\n", ip == str);

    return 0;
}

TCP communication process

// TCP communication process
// Server side (role that passively accepts connections)

1. Create a socket for listening (socket())

• Bind IP and port number (bind())
• Listening: Listening for client connections (listen())
• Socket: This socket is actually a file descriptor

2. Bind this listening file descriptor to the local IP and port (the IP and port are the address information of the server)

• The client uses this IP and port when connecting to the server (accept())

3. Set up monitoring, and the monitoring fd starts to work. Monitoring is to monitor whether there is data in the client's read buffer.

4. Blocking and waiting. When a client initiates a connection, unblock and accept the client's connection, you will get a socket
   (fd) for communicating with the client.
5. communication

• Receive data
• Send data (send data to another host's read buffer)

6. Communication ends, disconnect

// client

1. Create a socket (fd) for communication
2. To connect to the server, you need to specify the IP and port of the connected server.
3. The connection is successful and the client can communicate directly with the server

• Receive data
• send data

4. Communication ends, disconnect

• When a client monitors the server, it will return a new file descriptor for communication. You cannot use the monitored fd for communication. If you use the monitored file descriptor, it will be messy.
• Q: I don’t understand why the server side needs to bind the file descriptor of the server-side socket to the local IP and port number, but the client does not need it? Is this using the local computer as the server? How is the client's IP address specified?
• A: If the client is not bound, the system will automatically allocate a socket, but the server must be bound because send() requires a destination address as a parameter. Similar to when you make a phone call, you don’t need to know your own number, but you must know the other party’s number.

socket function

Client-server communication

Service-Terminal

// TCP 通信的服务器端

#include <stdio.h>
#include <arpa/inet.h>
#include <unistd.h>
#include <string.h>
#include <stdlib.h>

int main() {
    
    

    // 1.创建socket(用于监听的套接字)
    int lfd = socket(AF_INET, SOCK_STREAM, 0);

    if(lfd == -1) {
    
    
        perror("socket");
        exit(-1);
    }

    // 2.绑定
    struct sockaddr_in saddr;
    saddr.sin_family = AF_INET;
    // inet_pton(AF_INET, "192.168.193.128", saddr.sin_addr.s_addr);
    saddr.sin_addr.s_addr = INADDR_ANY;  // 0.0.0.0
    saddr.sin_port = htons(9999);
    int ret = bind(lfd, (struct sockaddr *)&saddr, sizeof(saddr));

    if(ret == -1) {
    
    
        perror("bind");
        exit(-1);
    }

    // 3.监听
    ret = listen(lfd, 8);
    if(ret == -1) {
    
    
        perror("listen");
        exit(-1);
    }

    // 4.接收客户端连接
    struct sockaddr_in clientaddr;
    int len = sizeof(clientaddr);
    int cfd = accept(lfd, (struct sockaddr *)&clientaddr, &len);
    
    if(cfd == -1) {
    
    
        perror("accept");
        exit(-1);
    }

    // 输出客户端的信息
    char clientIP[16];
    inet_ntop(AF_INET, &clientaddr.sin_addr.s_addr, clientIP, sizeof(clientIP));
    unsigned short clientPort = ntohs(clientaddr.sin_port);
    printf("client ip is %s, port is %d\n", clientIP, clientPort);

    // 5.通信
    char recvBuf[1024] = {
    
    0};
    while(1) {
    
    
        
        // 获取客户端的数据
        int num = read(cfd, recvBuf, sizeof(recvBuf));
        if(num == -1) {
    
    
            perror("read");
            exit(-1);
        } else if(num > 0) {
    
    
            printf("recv client data : %s\n", recvBuf);
        } else if(num == 0) {
    
    
            // 表示客户端断开连接
            printf("clinet closed...");
            break;
        }

        char * data = "hello,i am server";
        // 给客户端发送数据
        write(cfd, data, strlen(data));
    }
   
    // 关闭文件描述符
    close(cfd);
    close(lfd);

    return 0;
}

client

// TCP通信的客户端

#include <stdio.h>
#include <arpa/inet.h>
#include <unistd.h>
#include <string.h>
#include <stdlib.h>

int main() {
    
    

    // 1.创建套接字
    int fd = socket(AF_INET, SOCK_STREAM, 0);
    if(fd == -1) {
    
    
        perror("socket");
        exit(-1);
    }

    // 2.连接服务器端
    struct sockaddr_in serveraddr;
    serveraddr.sin_family = AF_INET;
    inet_pton(AF_INET, "192.168.193.128", &serveraddr.sin_addr.s_addr);
    serveraddr.sin_port = htons(9999);
    int ret = connect(fd, (struct sockaddr *)&serveraddr, sizeof(serveraddr));

    if(ret == -1) {
    
    
        perror("connect");
        exit(-1);
    }

    
    // 3. 通信
    char recvBuf[1024] = {
    
    0};
    while(1) {
    
    

        char * data = "hello,i am client";
        // 给客户端发送数据
        write(fd, data , strlen(data));

        sleep(1);
        
        int len = read(fd, recvBuf, sizeof(recvBuf));
        if(len == -1) {
    
    
            perror("read");
            exit(-1);
        } else if(len > 0) {
    
    
            printf("recv server data : %s\n", recvBuf);
        } else if(len == 0) {
    
    
            // 表示服务器端断开连接
            printf("server closed...");
            break;
        }

    }

    // 关闭连接
    close(fd);

    return 0;
}

operation result

• Adjust to send data every 1 second

TCP three-way handshake

• TCP is a connection-oriented unicast protocol. The three-way handshake is the behavior of the protocol when establishing a connection, not the behavior of the programmer. The same goes for the four-way handshake.
• The three-way handshake occurs on the client connection
• TCP is a connection-oriented unicast protocol. Before sending data, both communicating parties must establish a connection with each other. The so-called "connection" is actually a piece of information about each other stored in the memory of the client and the server, such as IP address, port number, etc.
• TCP can be viewed as a byte stream that handles packet loss, duplication, and errors at the IP layer or below. During the establishment process of the connection, both parties need to exchange some connection parameters. These parameters can be placed in the TCP header
• TCP provides a reliable, connection-oriented, byte stream, transport layer service, using a three-way handshake to establish a connection. Use four waves to close a connection

• The three-way handshake can confirm the reception and transmission between the client and the server.

• How to ensure that the data sent is complete?
• When sending, they are sent in a certain order, so how to ensure that the order of receiving is consistent?
• The completeness and sequence can be viewed through the serial number and confirmation number.

First handshake:
1. The client sets the SYN flag position to 1.
2. Generates a random 32-bit sequence number seq=]. Data can be carried after this serial number (data size).
3. The first handshake cannot carry
The second handshake of data :
1. The server receives the client’s connection: ACK=1
2. The server will send back a confirmation sequence number: ack = client’s sequence number + data length + SYN/FIN (calculated in one byte)
3. The server will initiate a connection request to the client: SYN=1
4. The server will generate a random sequence number: seq=K
The third handshake:
1. The client responds to the server's connection request: ACK=1
2. The client reply is received Server-side data: ack = server-side serial number + data length + SYN/FIN (calculated as one byte

TCP sliding window

• Sliding window is a flow control technology. In early network communications, the communicating parties directly sent data without considering network congestion. Since everyone does not know the network congestion situation and sends data at the same time, the intermediate nodes are blocked and packets are dropped, and no one can send data, so a sliding window mechanism is developed to solve this problem. The sliding window protocol is a technique used to improve throughput by allowing the sender to transmit additional packets before receiving any responses. The receiver tells the sender how many packets it can send at a certain time (called the window size)
• TCP uses a sliding window for transmission control. The size of the sliding window means how much buffer the receiver has available to receive data. The sender can use the size of the sliding window to determine how many bytes of data should be sent. When the sliding window is 0, the sender generally cannot send any more datagrams. The sliding window is a bearer structure in TCP that implements ACK confirmation, flow control, and congestion control.

• The sender's buffer, the buffer is understood as memory. The white grid is the free space; the gray grid is the data that has been sent but has not been received; the purple grid is the data that has not been sent yet.
• In the buffer of the receiver, the white grid is the free space; the purple grid is the received data.

• The understanding of the first statement, and so on for other statements: the three-way handshake client sends a SYN flag, 0 (0) means that the random sequence number is 0, and 0 data is sent, win4096 means that the sliding window is 4096, which means that the buffer can Received 4096 pieces of data, mss1460 means that the maximum data in a message segment is 1460, win6144 means that the maximum sliding window of the receiving end is 6144, the largest piece of data is mss1024, and then an ACK 8001 was returned. Why is it ACK8001? Because the serial number you brought is 8000, the flag is 1, add 1 to get 8001, and so on for the rest.

• The window is understood as the size of the buffer
• The size of the sliding window changes as data is sent and received
• Both parties in the communication have buffers for sending and receiving data.
• server:
  • Send buffer (window of send buffer)
  • Receive buffer (window of receive buffer)
• Client:
  • Send buffer (window of send buffer)
  • Receive buffer (window of receive buffer)

TCP waves four times

• When the communicating parties disconnect, calling close() in the program will use the TCP protocol to wave four times.
• Both the client and the server can actively initiate disconnection. Whoever calls close() first will initiate the disconnection.
• Because when TCP connects

TCP communication-multi-process implementation of concurrent server

• To implement TCP communication server to handle concurrent tasks, use multi-threads or multi-processes to solve the problem
• Idea:
  • 1 parent process, multiple child processes
  • The parent process is responsible for waiting for and accepting connections from clients
  • Subprocess: Complete communication, accept a client connection, and create a subprocess for communication.

// TCP通信的客户端
#include <stdio.h>
#include <arpa/inet.h>
#include <unistd.h>
#include <string.h>
#include <stdlib.h>

int main() {
    
    

    // 1.创建套接字
    int fd = socket(AF_INET, SOCK_STREAM, 0);
    if(fd == -1) {
    
    
        perror("socket");
        exit(-1);
    }

    // 2.连接服务器端
    struct sockaddr_in serveraddr;
    serveraddr.sin_family = AF_INET;
    inet_pton(AF_INET, "192.168.193.128", &serveraddr.sin_addr.s_addr);
    serveraddr.sin_port = htons(9999);
    int ret = connect(fd, (struct sockaddr *)&serveraddr, sizeof(serveraddr));

    if(ret == -1) {
    
    
        perror("connect");
        exit(-1);
    }
    
    // 3. 通信
    char recvBuf[1024];
    int i = 0;
    while(1) {
    
    
        
        sprintf(recvBuf, "data : %d\n", i++);
        
        // 给服务器端发送数据
        write(fd, recvBuf, strlen(recvBuf)+1);

        int len = read(fd, recvBuf, sizeof(recvBuf));
        if(len == -1) {
    
    
            perror("read");
            exit(-1);
        } else if(len > 0) {
    
    
            printf("recv server : %s\n", recvBuf);
        } else if(len == 0) {
    
    
            // 表示服务器端断开连接
            printf("server closed...");
            break;
        }

        sleep(1);
    }

    // 关闭连接
    close(fd);

    return 0;
}

#include <stdio.h>
#include <arpa/inet.h>
#include <unistd.h>
#include <stdlib.h>
#include <string.h>
#include <signal.h>
#include <wait.h>
#include <errno.h>

void recyleChild(int arg) {
    
    
    while(1) {
    
    
        int ret = waitpid(-1, NULL, WNOHANG);
        if(ret == -1) {
    
    
            // 所有的子进程都回收了
            break;
        }else if(ret == 0) {
    
    
            // 还有子进程活着
            break;
        } else if(ret > 0){
    
    
            // 被回收了
            printf("子进程 %d 被回收了\n", ret);
        }
    }
}

int main() {
    
    

    struct sigaction act;
    act.sa_flags = 0;
    sigemptyset(&act.sa_mask);
    act.sa_handler = recyleChild;
    // 注册信号捕捉
    sigaction(SIGCHLD, &act, NULL);
    

    // 创建socket
    int lfd = socket(PF_INET, SOCK_STREAM, 0);
    if(lfd == -1){
    
    
        perror("socket");
        exit(-1);
    }

    struct sockaddr_in saddr;
    saddr.sin_family = AF_INET;
    saddr.sin_port = htons(9999);
    saddr.sin_addr.s_addr = INADDR_ANY;

    // 绑定
    int ret = bind(lfd,(struct sockaddr *)&saddr, sizeof(saddr));
    if(ret == -1) {
    
    
        perror("bind");
        exit(-1);
    }

    // 监听
    ret = listen(lfd, 128);
    if(ret == -1) {
    
    
        perror("listen");
        exit(-1);
    }

    // 不断循环等待客户端连接
    while(1) {
    
    

        struct sockaddr_in cliaddr;
        int len = sizeof(cliaddr);
        // 接受连接
        int cfd = accept(lfd, (struct sockaddr*)&cliaddr, &len);
        if(cfd == -1) {
    
    
            if(errno == EINTR) {
    
    
                continue;
            }
            perror("accept");
            exit(-1);
        }

        // 每一个连接进来，创建一个子进程跟客户端通信
        pid_t pid = fork();
        if(pid == 0) {
    
    
            // 子进程
            // 获取客户端的信息
            char cliIp[16];
            inet_ntop(AF_INET, &cliaddr.sin_addr.s_addr, cliIp, sizeof(cliIp));
            unsigned short cliPort = ntohs(cliaddr.sin_port);
            printf("client ip is : %s, prot is %d\n", cliIp, cliPort);

            // 接收客户端发来的数据
            char recvBuf[1024];
            while(1) {
    
    
                int len = read(cfd, &recvBuf, sizeof(recvBuf));

                if(len == -1) {
    
    
                    perror("read");
                    exit(-1);
                }else if(len > 0) {
    
    
                    printf("recv client : %s\n", recvBuf);
                } else if(len == 0) {
    
    
                    printf("client closed....\n");
                    break;
                }
                write(cfd, recvBuf, strlen(recvBuf) + 1);
            }
            close(cfd);
            exit(0);    // 退出当前子进程
        }

    }
    close(lfd);
    return 0;
}

• The child process cannot be recycled in the parent process, so you need to register signal capture and use the sigchld signal to capture

TCP communication-multi-threading to implement concurrent server

• Multiple threads share the same virtual address space

#include <stdio.h>
#include <arpa/inet.h>
#include <unistd.h>
#include <stdlib.h>
#include <string.h>
#include <pthread.h>

struct sockInfo {
    
    
    int fd; // 通信的文件描述符
    struct sockaddr_in addr;//客户端信息
    pthread_t tid;  // 线程号
};

struct sockInfo sockinfos[128];

void * working(void * arg) {
    
    
    // 子线程和客户端通信   cfd 客户端的信息 线程号
    // 获取客户端的信息
    struct sockInfo * pinfo = (struct sockInfo *)arg;

    char cliIp[16];
    inet_ntop(AF_INET, &pinfo->addr.sin_addr.s_addr, cliIp, sizeof(cliIp));
    unsigned short cliPort = ntohs(pinfo->addr.sin_port);
    printf("client ip is : %s, prot is %d\n", cliIp, cliPort);

    // 接收客户端发来的数据
    char recvBuf[1024];
    while(1) {
    
    
        int len = read(pinfo->fd, &recvBuf, sizeof(recvBuf));

        if(len == -1) {
    
    
            perror("read");
            exit(-1);
        }else if(len > 0) {
    
    
            printf("recv client : %s\n", recvBuf);
        } else if(len == 0) {
    
    
            printf("client closed....\n");
            break;
        }
        write(pinfo->fd, recvBuf, strlen(recvBuf) + 1);
    }
    close(pinfo->fd);
    return NULL;
}

int main() {
    
    

    // 创建socket
    int lfd = socket(PF_INET, SOCK_STREAM, 0);
    if(lfd == -1){
    
    
        perror("socket");
        exit(-1);
    }

    struct sockaddr_in saddr;
    saddr.sin_family = AF_INET;
    saddr.sin_port = htons(9999);
    saddr.sin_addr.s_addr = INADDR_ANY;

    // 绑定
    int ret = bind(lfd,(struct sockaddr *)&saddr, sizeof(saddr));
    if(ret == -1) {
    
    
        perror("bind");
        exit(-1);
    }

    // 监听
    ret = listen(lfd, 128);
    if(ret == -1) {
    
    
        perror("listen");
        exit(-1);
    }

    // 初始化数据
    int max = sizeof(sockinfos) / sizeof(sockinfos[0]);
    for(int i = 0; i < max; i++) {
    
    
        bzero(&sockinfos[i], sizeof(sockinfos[i]));//初始化成0
        sockinfos[i].fd = -1;
        sockinfos[i].tid = -1;
    }

    // 循环等待客户端连接，一旦一个客户端连接进来，就创建一个子线程进行通信
    while(1) {
    
    

        struct sockaddr_in cliaddr;//保存客户端接收到的信息
        int len = sizeof(cliaddr);
        // 接受连接
        int cfd = accept(lfd, (struct sockaddr*)&cliaddr, &len);

        struct sockInfo * pinfo;
        for(int i = 0; i < max; i++) {
    
    
            // 从这个数组中找到一个可以用的sockInfo元素
            if(sockinfos[i].fd == -1) {
    
    
                pinfo = &sockinfos[i];
                break;
            }
            if(i == max - 1) {
    
    
                sleep(1);
                i--;
            }
        }

        pinfo->fd = cfd;
        memcpy(&pinfo->addr, &cliaddr, len);

        // 创建子线程
        pthread_create(&pinfo->tid, NULL, working, pinfo);

        pthread_detach(pinfo->tid);
    }

    close(lfd);
    return 0;
}

TCP state transition

• The three-way handshake is initiated by the client first. It calls the connect() function to actively connect to the server, and the bottom layer starts the first handshake.

half closed

• After waving four times, the client calls close() and an ACK will be returned. If the server does not call close(), it will not wave for the third time. If the server does not call close() at this time, then at this time This state is a semi-closed state, that is, only one side is closed. One end cannot send data and can only receive data. Therefore, if there is such a need, the semi-closed state can be used.
• From a program perspective, you can use the API to control the semi-closed connection state.

#include <sys/socket.h>
int shutdown(int sockfd, int how); 
sockfd: 需要关闭的socket的描述符
how:	允许为shutdown操作选择以下几种方式:
SHUT_RD(0)：  关闭sockfd上的读功能，此选项将不允许sockfd进行读操作。
该套接字不再接收数据，任何当前在套接字接受缓冲区的数据将被无声的丢弃掉。
SHUT_WR(1):  关闭sockfd的写功能，此选项将不允许sockfd进行写操作。进程不能在对此套接字发出写操作。
SHUT_RDWR(2):关闭sockfd的读写功能。相当于调用shutdown两次：首先是以SHUT_RD,然后以 SHUT_WR。

• Use close to terminate a connection, but it only reduces the descriptor's reference count and does not directly close the connection. The connection is closed only when the descriptor's reference count is 0. shutdown closes the descriptor directly, regardless of the descriptor's reference count. You can also choose to terminate the connection in one direction, only reading or only writing.
• Notice:

1. If there are multiple processes sharing a socket, each time close is called, the count will be decremented by 1 until the count reaches 0, that is, all processes have called close, and the socket will be released.
2. In multi-process, if one process calls shutdown(sfd, SHUT_RDWR), other processes will not be able to communicate. But if a process close(sfd) will not affect other processes
   

Port reuse

parameter:

• sockfd: the file descriptor to be operated on
• level: level - SOL_SOCKET (level of port multiplexing)
• optname: the name of the option
  • SO_REUSEADDR
  • SO_REUSEPORT
• optval: port multiplexing value (shaped)
  • 1: Can be reused
  • 0: Cannot be reused
• optlen: size of optval parameter
• Port reuse, the setting time is before the server binds the port
• First call the setsockopt() function to set up port reuse.
• Then use bind() to bind

Common command netstat parameters for viewing network-related information
:
-a all sockets
-p displays the name of the program using the socket
- directly uses the P address without going through the domain name server

#include <stdio.h>
#include <arpa/inet.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>

int main() {
    
    

    // 创建socket
    int fd = socket(PF_INET, SOCK_STREAM, 0);
    if(fd == -1) {
    
    
        perror("socket");
        return -1;
    }

    struct sockaddr_in seraddr;
    inet_pton(AF_INET, "127.0.0.1", &seraddr.sin_addr.s_addr);
    seraddr.sin_family = AF_INET;
    seraddr.sin_port = htons(9999);

    // 连接服务器
    int ret = connect(fd, (struct sockaddr *)&seraddr, sizeof(seraddr));

    if(ret == -1){
    
    
        perror("connect");
        return -1;
    }

    while(1) {
    
    
        char sendBuf[1024] = {
    
    0};
        fgets(sendBuf, sizeof(sendBuf), stdin);

        write(fd, sendBuf, strlen(sendBuf) + 1);

        // 接收
        int len = read(fd, sendBuf, sizeof(sendBuf));
        if(len == -1) {
    
    
            perror("read");
            return -1;
        }else if(len > 0) {
    
    
            printf("read buf = %s\n", sendBuf);
        } else {
    
    
            printf("服务器已经断开连接...\n");
            break;
        }
    }

    close(fd);

    return 0;
}

#include <stdio.h>
#include <ctype.h>
#include <arpa/inet.h>
#include <unistd.h>
#include <stdlib.h>
#include <string.h>

int main(int argc, char *argv[]) {
    
    

    // 创建socket
    int lfd = socket(PF_INET, SOCK_STREAM, 0);

    if(lfd == -1) {
    
    
        perror("socket");
        return -1;
    }

    struct sockaddr_in saddr;
    saddr.sin_family = AF_INET;
    saddr.sin_addr.s_addr = INADDR_ANY;
    saddr.sin_port = htons(9999);
    
    //int optval = 1;
    //setsockopt(lfd, SOL_SOCKET, SO_REUSEADDR, &optval, sizeof(optval));

    int optval = 1;
    setsockopt(lfd, SOL_SOCKET, SO_REUSEPORT, &optval, sizeof(optval));

    // 绑定
    int ret = bind(lfd, (struct sockaddr *)&saddr, sizeof(saddr));
    if(ret == -1) {
    
    
        perror("bind");
        return -1;
    }

    // 监听
    ret = listen(lfd, 8);
    if(ret == -1) {
    
    
        perror("listen");
        return -1;
    }

    // 接收客户端连接
    struct sockaddr_in cliaddr;
    socklen_t len = sizeof(cliaddr);
    int cfd = accept(lfd, (struct sockaddr *)&cliaddr, &len);
    if(cfd == -1) {
    
    
        perror("accpet");
        return -1;
    }

    // 获取客户端信息
    char cliIp[16];
    inet_ntop(AF_INET, &cliaddr.sin_addr.s_addr, cliIp, sizeof(cliIp));
    unsigned short cliPort = ntohs(cliaddr.sin_port);

    // 输出客户端的信息
    printf("client's ip is %s, and port is %d\n", cliIp, cliPort );

    // 接收客户端发来的数据
    char recvBuf[1024] = {
    
    0};
    while(1) {
    
    
        int len = recv(cfd, recvBuf, sizeof(recvBuf), 0);
        if(len == -1) {
    
    
            perror("recv");
            return -1;
        } else if(len == 0) {
    
    
            printf("客户端已经断开连接...\n");
            break;
        } else if(len > 0) {
    
    
            printf("read buf = %s\n", recvBuf);
        }

        // 小写转大写
        for(int i = 0; i < len; ++i) {
    
    
            recvBuf[i] = toupper(recvBuf[i]);
        }

        printf("after buf = %s\n", recvBuf);

        // 大写字符串发给客户端
        ret = send(cfd, recvBuf, strlen(recvBuf) + 1, 0);
        if(ret == -1) {
    
    
            perror("send");
            return -1;
        }
    }
    
    close(cfd);
    close(lfd);

    return 0;
}

• Why are there two sockets on the server side? One is used for monitoring and the other is used for communication, so the status of the two is also different.

• When the state is in timewait, there will be twice the message waiting time.
• If the port is not released within 1 minute, the binding cannot continue.

• After adding port reuse, no errors are reported and can be bound.

I/O multiplexing (I/O multiplexing)

• Both parties in the communication have SOCKET, which corresponds to the buffer in the kernel. IO is the operation of the buffer. For example, writing is from the write buffer. When writing, data is sent to the read buffer of the other party. Reading is from the read buffer. Read data
• I/O multiplexing allows the program to monitor multiple file descriptors at the same time, which can improve the performance of the program . The system calls to implement I/O multiplexing under Linux mainly include select, poll and epoll.
• If there are many clients sending me messages, and I don’t know which client sent them, do I have to traverse their file descriptors? Yes, this was the previous approach, traversing the file descriptors one by one, but now using IO multiplexing, you can monitor at the same time
• IO multiplexing is to see whether there is data in the read and write buffer of the file descriptor
  

I/O model

blocking wait

• The read function, get function, accept(), and recv() are all blocked. Read is always waiting for data, which is inefficient.

• No time slice is taken up when blocking
• Use multi-threading and multi-process to solve concurrency problems
  

BIO model

Non-blocking, busy polling

NIO model

IO multiplexing

• Delegate kernel completion

• There used to be multiple paths, but now there is only one path. Originally, it needed to be traversed one by one to see if the file descriptor had data. Now, it is handed over to the file descriptor and let him traverse it by himself.
  

select

• Main idea:
  • First, you need to construct a list of file descriptors and add the file descriptors to be monitored to the list.
  • Call a system function to monitor the file descriptors in the list. The function does not return until one or more of these descriptors perform I/O operations: a. This function is blocking b. The function blocks the
    file
    descriptor The detection operation is done by the kernel
  • On return, it tells the process how many (which) descriptors require I/O operations

• The following 1024 bits, each bit represents a file descriptor
• The read buffer is tested to see if there is data in it, and the write buffer is tested to see if there is free space in it . If there is free space, data can be written into it. If there is free data, the corresponding flag position is set to 1.
• In the same way, the same is true for the detection of read buffers. Set the corresponding flag position to 1, and then detect when it encounters a 1. When it is 0, it will not detect it. As long as it is called once, it will know which file descriptors have data

Work content

• Regarding the explanation of why select needs +1, because select traverses from the beginning, from 0 to 101, if the length is only 101, then it can only traverse from 0 to 100, so +1 is needed
• reads is a pointer passed in and out of parameters. It monitors which bit has changed. For example, if it monitors 3, its position is 1. The user mode can traverse this collection. If it traverses to 3, if there is data, it will be described from the file 3. character to read the data, and there is no need to read the rest if there is no data. What is used here is to fd_set readsmonitor the read buffer. During detection, the collection will be copied from the user state to the kernel state, and then the collection will be traversed to see which one needs to be detected. , it is necessary to detect if the flag bit is 1. If there is data in 3 and 4 at this time, the flag bit remains at 1. If there is no data in 100 and 101 at this time, the flag bit is changed from 1 to 0, indicating that no data has arrived. Then copy from the kernel state to the user state. Now we know that 3 has data, and then traverse it again, and then at 3, read and recv are used to read the data. To determine whether the flag bit is 1, you can FD_ISSET(3,&reads)use
• In the next cycle, if the client corresponding to 10 ends and is disconnected, it will no longer be detected by the kernel and will be FD_CLR(100,&reads)cleared.
  

Code

#include <stdio.h>
#include <arpa/inet.h>
#include <unistd.h>
#include <stdlib.h>
#include <string.h>
#include <sys/select.h>

int main() {
    
    

    // 创建socket
    int lfd = socket(PF_INET, SOCK_STREAM, 0);
    struct sockaddr_in saddr;
    saddr.sin_port = htons(9999);
    saddr.sin_family = AF_INET;
    saddr.sin_addr.s_addr = INADDR_ANY;

    // 绑定
    bind(lfd, (struct sockaddr *)&saddr, sizeof(saddr));

    // 监听
    listen(lfd, 8);

    // 创建一个fd_set的集合，存放的是需要检测的文件描述符
    fd_set rdset, tmp;
    FD_ZERO(&rdset);
    FD_SET(lfd, &rdset);
    int maxfd = lfd;

    while(1) {
    
    

        tmp = rdset;

        // 调用select系统函数，让内核帮检测哪些文件描述符有数据
        int ret = select(maxfd + 1, &tmp, NULL, NULL, NULL);
        if(ret == -1) {
    
    
            perror("select");
            exit(-1);
        } else if(ret == 0) {
    
    
            continue;
        } else if(ret > 0) {
    
    
            // 说明检测到了有文件描述符的对应的缓冲区的数据发生了改变
            if(FD_ISSET(lfd, &tmp)) {
    
    
                // 表示有新的客户端连接进来了
                struct sockaddr_in cliaddr;
                int len = sizeof(cliaddr);
                int cfd = accept(lfd, (struct sockaddr *)&cliaddr, &len);

                // 将新的文件描述符加入到集合中
                FD_SET(cfd, &rdset);

                // 更新最大的文件描述符
                maxfd = maxfd > cfd ? maxfd : cfd;
            }

            for(int i = lfd + 1; i <= maxfd; i++) {
    
    
                if(FD_ISSET(i, &tmp)) {
    
    
                    // 说明这个文件描述符对应的客户端发来了数据
                    char buf[1024] = {
    
    0};
                    int len = read(i, buf, sizeof(buf));
                    if(len == -1) {
    
    
                        perror("read");
                        exit(-1);
                    } else if(len == 0) {
    
    
                        printf("client closed...\n");
                        close(i);
                        FD_CLR(i, &rdset);
                    } else if(len > 0) {
    
    
                        printf("read buf = %s\n", buf);
                        write(i, buf, strlen(buf) + 1);
                    }
                }
            }

        }

    }
    close(lfd);
    return 0;
}

#include <stdio.h>
#include <arpa/inet.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>

int main() {
    
    

    // 创建socket
    int fd = socket(PF_INET, SOCK_STREAM, 0);
    if(fd == -1) {
    
    
        perror("socket");
        return -1;
    }

    struct sockaddr_in seraddr;
    inet_pton(AF_INET, "127.0.0.1", &seraddr.sin_addr.s_addr);
    seraddr.sin_family = AF_INET;
    seraddr.sin_port = htons(9999);

    // 连接服务器
    int ret = connect(fd, (struct sockaddr *)&seraddr, sizeof(seraddr));

    if(ret == -1){
    
    
        perror("connect");
        return -1;
    }

    int num = 0;
    while(1) {
    
    
        char sendBuf[1024] = {
    
    0};
        sprintf(sendBuf, "send data %d", num++);
        write(fd, sendBuf, strlen(sendBuf) + 1);

        // 接收
        int len = read(fd, sendBuf, sizeof(sendBuf));
        if(len == -1) {
    
    
            perror("read");
            return -1;
        }else if(len > 0) {
    
    
            printf("read buf = %s\n", sendBuf);
        } else {
    
    
            printf("服务器已经断开连接...\n");
            break;
        }
        // sleep(1);
        usleep(1000);
    }

    close(fd);

    return 0;
}

• Achieve multi-client connection through select, this is IO multiplexing technology

poll

• Improve select and encapsulate fds into a structure array, which is a collection of file descriptors that need to be detected.

• Focus on detecting read events
• When both read and write events are detected,myfd.events = POLLIN | POLLOUT;

#include <stdio.h>
#include <arpa/inet.h>
#include <unistd.h>
#include <stdlib.h>
#include <string.h>
#include <poll.h>


int main() {
    
    

    // 创建socket
    int lfd = socket(PF_INET, SOCK_STREAM, 0);
    struct sockaddr_in saddr;
    saddr.sin_port = htons(9999);
    saddr.sin_family = AF_INET;
    saddr.sin_addr.s_addr = INADDR_ANY;

    // 绑定
    bind(lfd, (struct sockaddr *)&saddr, sizeof(saddr));

    // 监听
    listen(lfd, 8);

    // 初始化检测的文件描述符数组
    struct pollfd fds[1024];
    for(int i = 0; i < 1024; i++) {
    
    
        fds[i].fd = -1;
        fds[i].events = POLLIN;
    }
    fds[0].fd = lfd;//要监听的文件描述符
    int nfds = 0;

    while(1) {
    
    

        // 调用poll系统函数，让内核帮检测哪些文件描述符有数据
        int ret = poll(fds, nfds + 1, -1);
        if(ret == -1) {
    
    
            perror("poll");
            exit(-1);
        } else if(ret == 0) {
    
    
            continue;
        } else if(ret > 0) {
    
    
            // 说明检测到了有文件描述符的对应的缓冲区的数据发生了改变
            if(fds[0].revents & POLLIN) {
    
    
                // 表示有新的客户端连接进来了
                struct sockaddr_in cliaddr;
                int len = sizeof(cliaddr);
                int cfd = accept(lfd, (struct sockaddr *)&cliaddr, &len);

                // 将新的文件描述符加入到集合中
                for(int i = 1; i < 1024; i++) {
    
    
                    if(fds[i].fd == -1) {
    
    
                        fds[i].fd = cfd;
                        fds[i].events = POLLIN;
                        break;
                    }
                }

                // 更新最大的文件描述符的索引
                nfds = nfds > cfd ? nfds : cfd;
            }

            for(int i = 1; i <= nfds; i++) {
    
    
                if(fds[i].revents & POLLIN) {
    
    
                    // 说明这个文件描述符对应的客户端发来了数据
                    char buf[1024] = {
    
    0};
                    int len = read(fds[i].fd, buf, sizeof(buf));
                    if(len == -1) {
    
    
                        perror("read");
                        exit(-1);
                    } else if(len == 0) {
    
    
                        printf("client closed...\n");
                        close(fds[i].fd);
                        fds[i].fd = -1;
                    } else if(len > 0) {
    
    
                        printf("read buf = %s\n", buf);
                        write(fds[i].fd, buf, strlen(buf) + 1);
                    }
                }
            }

        }

    }
    close(lfd);
    return 0;
}

epoll

• First create an epoll instance in the kernel area
• The structure of the kernel area can be operated through the file descriptor.

• Operate the kernel directly without copying or switching from user mode to kernel mode.
• rbr is a red-black tree data structure, which replaces the original linear array structure. It speeds up traversal and greatly improves efficiency.
• rdlist is a double linked list

#include <stdio.h>
#include <arpa/inet.h>
#include <unistd.h>
#include <stdlib.h>
#include <string.h>
#include <sys/epoll.h>

int main() {
    
    

    // 创建socket
    int lfd = socket(PF_INET, SOCK_STREAM, 0);
    struct sockaddr_in saddr;
    saddr.sin_port = htons(9999);
    saddr.sin_family = AF_INET;
    saddr.sin_addr.s_addr = INADDR_ANY;

    // 绑定
    bind(lfd, (struct sockaddr *)&saddr, sizeof(saddr));

    // 监听
    listen(lfd, 8);

    // 调用epoll_create()创建一个epoll实例
    int epfd = epoll_create(100);//参数随便给一个值

    // 将监听的文件描述符相关的检测信息添加到epoll实例中
    struct epoll_event epev;
    epev.events = EPOLLIN;
    epev.data.fd = lfd;
    epoll_ctl(epfd, EPOLL_CTL_ADD, lfd, &epev);
    //第一个是epfd，第二个是要操作的类型，第三个是要添加的文件描述符信息，第四个是event

    struct epoll_event epevs[1024];//把发生改变的文件描述符信息装到这里，即内核检测完的数据，直接遍历就行

    while(1) {
    
    

        int ret = epoll_wait(epfd, epevs, 1024, -1);//-1表示阻塞
        if(ret == -1) {
    
    
            perror("epoll_wait");
            exit(-1);
        }

        printf("ret = %d\n", ret);

        for(int i = 0; i < ret; i++) {
    
    

            int curfd = epevs[i].data.fd;

            if(curfd == lfd) {
    
    
                // 监听的文件描述符有数据达到，有客户端连接
                struct sockaddr_in cliaddr;
                int len = sizeof(cliaddr);
                int cfd = accept(lfd, (struct sockaddr *)&cliaddr, &len);//返回的通信文件描述符，这不能和监听文件描述符混用

                epev.events = EPOLLIN; //EPOLLIN | EPOLLOUT
                epev.data.fd = cfd;
                epoll_ctl(epfd, EPOLL_CTL_ADD, cfd, &epev);//添加到实例当中
            } else {
    
    //我们的逻辑是只有读检测到了事件，才能顺应接下来的逻辑，如果是写到来，则不适用
                //EPOLLIN | EPOLLOUT，所以要加上这个判断，要根据返回来的events进行具体操作
                if(epevs[i].events & EPOLLOUT) {
    
    
                    continue;
                }   
                // 有数据到达，需要通信
                char buf[1024] = {
    
    0};
                int len = read(curfd, buf, sizeof(buf));
                if(len == -1) {
    
    
                    perror("read");
                    exit(-1);
                } else if(len == 0) {
    
    
                    printf("client closed...\n");
                    epoll_ctl(epfd, EPOLL_CTL_DEL, curfd, NULL);//关闭连接删除用NULL
                    close(curfd);
                } else if(len > 0) {
    
    //说明读到数据了，然后把数据写出来
                    printf("read buf = %s\n", buf);
                    write(curfd, buf, strlen(buf) + 1);
                }

            }

        }
    }

    close(lfd);
    close(epfd);
    return 0;
}

epoll working mode

• Horizontal triggering is the default working mode. The default goose working mode will always notify you as long as there is data in the buffer.
• Edge triggering only supports non-blocking and only notifies once.

LT level trigger

#include <stdio.h>
#include <arpa/inet.h>
#include <unistd.h>
#include <stdlib.h>
#include <string.h>
#include <sys/epoll.h>

int main() {
    
    

    // 创建socket
    int lfd = socket(PF_INET, SOCK_STREAM, 0);
    struct sockaddr_in saddr;
    saddr.sin_port = htons(9999);
    saddr.sin_family = AF_INET;
    saddr.sin_addr.s_addr = INADDR_ANY;

    // 绑定
    bind(lfd, (struct sockaddr *)&saddr, sizeof(saddr));

    // 监听
    listen(lfd, 8);

    // 调用epoll_create()创建一个epoll实例
    int epfd = epoll_create(100);

    // 将监听的文件描述符相关的检测信息添加到epoll实例中
    struct epoll_event epev;
    epev.events = EPOLLIN;
    epev.data.fd = lfd;
    epoll_ctl(epfd, EPOLL_CTL_ADD, lfd, &epev);

    struct epoll_event epevs[1024];

    while(1) {
    
    

        int ret = epoll_wait(epfd, epevs, 1024, -1);
        if(ret == -1) {
    
    
            perror("epoll_wait");
            exit(-1);
        }

        printf("ret = %d\n", ret);

        for(int i = 0; i < ret; i++) {
    
    

            int curfd = epevs[i].data.fd;

            if(curfd == lfd) {
    
    
                // 监听的文件描述符有数据达到，有客户端连接
                struct sockaddr_in cliaddr;
                int len = sizeof(cliaddr);
                int cfd = accept(lfd, (struct sockaddr *)&cliaddr, &len);

                epev.events = EPOLLIN;
                epev.data.fd = cfd;
                epoll_ctl(epfd, EPOLL_CTL_ADD, cfd, &epev);
            } else {
    
    
                if(epevs[i].events & EPOLLOUT) {
    
    
                    continue;
                }   
                // 有数据到达，需要通信
                char buf[5] = {
    
    0};
                int len = read(curfd, buf, sizeof(buf));
                if(len == -1) {
    
    
                    perror("read");
                    exit(-1);
                } else if(len == 0) {
    
    
                    printf("client closed...\n");
                    epoll_ctl(epfd, EPOLL_CTL_DEL, curfd, NULL);
                    close(curfd);
                } else if(len > 0) {
    
    
                    printf("read buf = %s\n", buf);
                    write(curfd, buf, strlen(buf) + 1);
                }

            }

        }
    }

    close(lfd);
    close(epfd);
    return 0;
}

• It can be seen from the running effect that as long as there is data in the buffer, this ret will always be updated.
  

ET edge trigger

• Used in conjunction with non-blocking APIs

#include <stdio.h>
#include <arpa/inet.h>
#include <unistd.h>
#include <stdlib.h>
#include <string.h>
#include <sys/epoll.h>
#include <fcntl.h>
#include <errno.h>

int main() {
    
    

    // 创建socket
    int lfd = socket(PF_INET, SOCK_STREAM, 0);
    struct sockaddr_in saddr;
    saddr.sin_port = htons(9999);
    saddr.sin_family = AF_INET;
    saddr.sin_addr.s_addr = INADDR_ANY;

    // 绑定
    bind(lfd, (struct sockaddr *)&saddr, sizeof(saddr));

    // 监听
    listen(lfd, 8);

    // 调用epoll_create()创建一个epoll实例
    int epfd = epoll_create(100);

    // 将监听的文件描述符相关的检测信息添加到epoll实例中
    struct epoll_event epev;
    epev.events = EPOLLIN;
    epev.data.fd = lfd;
    epoll_ctl(epfd, EPOLL_CTL_ADD, lfd, &epev);

    struct epoll_event epevs[1024];

    while(1) {
    
    

        int ret = epoll_wait(epfd, epevs, 1024, -1);
        if(ret == -1) {
    
    
            perror("epoll_wait");
            exit(-1);
        }

        printf("ret = %d\n", ret);

        for(int i = 0; i < ret; i++) {
    
    

            int curfd = epevs[i].data.fd;

            if(curfd == lfd) {
    
    
                // 监听的文件描述符有数据达到，有客户端连接
                struct sockaddr_in cliaddr;
                int len = sizeof(cliaddr);
                int cfd = accept(lfd, (struct sockaddr *)&cliaddr, &len);

                // 设置cfd属性非阻塞
                int flag = fcntl(cfd, F_GETFL);
                flag | O_NONBLOCK;
                fcntl(cfd, F_SETFL, flag);

                epev.events = EPOLLIN | EPOLLET;    // 设置边沿触发
                epev.data.fd = cfd;//有数据只会通知一次
                epoll_ctl(epfd, EPOLL_CTL_ADD, cfd, &epev);
            } else {
    
    
                if(epevs[i].events & EPOLLOUT) {
    
    
                    continue;
                }  

                // 循环读取出所有数据
                char buf[5];
                int len = 0;
                while( (len = read(curfd, buf, sizeof(buf))) > 0) {
    
    
                    // 打印数据
                    // printf("recv data : %s\n", buf);
                    write(STDOUT_FILENO, buf, len);
                    write(curfd, buf, len);
                }
                if(len == 0) {
    
    
                    printf("client closed....");
                }else if(len == -1) {
    
    
                    if(errno == EAGAIN) {
    
    
                        printf("data over.....");
                    }else {
    
    
                        perror("read");
                        exit(-1);
                    }
                    
                }

            }

        }
    }

    close(lfd);
    close(epfd);
    return 0;
}

• Since the ret triggered by the edge will only be updated once, after completing the sub-loop, you jump back to the while loop and will be blocked in int ret = epoll_wait(epfd, epevs, 1024, -1);it, so it will not be displayed later.
• After improvement

Connection refused error

• This is due to 2MSL, please wait 1 minute
• Because the client was blocked here just now, the server stops first. Whoever stops first will have a TIME_WAIT state. You have to wait for 1 minute and then connect again.

UDP

• UDP communicates through datagrams and does not need to ensure data security.
• The UDP Socket is not like the TCP Socket, which has monitoring and communication. It is directly a socket for communication.

UDP communication

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <arpa/inet.h>

int main() {
    
    

    // 1.创建一个通信的socket
    int fd = socket(PF_INET, SOCK_DGRAM, 0);
    
    if(fd == -1) {
    
    
        perror("socket");
        exit(-1);
    }   

    struct sockaddr_in addr;
    addr.sin_family = AF_INET;
    addr.sin_port = htons(9999);
    addr.sin_addr.s_addr = INADDR_ANY;

    // 2.绑定
    int ret = bind(fd, (struct sockaddr *)&addr, sizeof(addr));
    if(ret == -1) {
    
    
        perror("bind");
        exit(-1);
    }

    // 3.通信
    while(1) {
    
    
        char recvbuf[128];
        char ipbuf[16];

        struct sockaddr_in cliaddr;
        int len = sizeof(cliaddr);

        // 接收数据
        int num = recvfrom(fd, recvbuf, sizeof(recvbuf), 0, (struct sockaddr *)&cliaddr, &len);

        printf("client IP : %s, Port : %d\n", 
            inet_ntop(AF_INET, &cliaddr.sin_addr.s_addr, ipbuf, sizeof(ipbuf)),
            ntohs(cliaddr.sin_port));

        printf("client say : %s\n", recvbuf);

        // 发送数据
        sendto(fd, recvbuf, strlen(recvbuf) + 1, 0, (struct sockaddr *)&cliaddr, sizeof(cliaddr));

    }

    close(fd);
    return 0;
}

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <arpa/inet.h>

int main() {
    
    

    // 1.创建一个通信的socket
    int fd = socket(PF_INET, SOCK_DGRAM, 0);
    
    if(fd == -1) {
    
    
        perror("socket");
        exit(-1);
    }   

    // 服务器的地址信息
    struct sockaddr_in saddr;
    saddr.sin_family = AF_INET;
    saddr.sin_port = htons(9999);
    inet_pton(AF_INET, "127.0.0.1", &saddr.sin_addr.s_addr);

    int num = 0;
    // 3.通信
    while(1) {
    
    

        // 发送数据
        char sendBuf[128];
        sprintf(sendBuf, "hello , i am client %d \n", num++);
        sendto(fd, sendBuf, strlen(sendBuf) + 1, 0, (struct sockaddr *)&saddr, sizeof(saddr));

        // 接收数据
        int num = recvfrom(fd, sendBuf, sizeof(sendBuf), 0, NULL, NULL);
        printf("server say : %s\n", sendBuf);

        sleep(1);
    }

    close(fd);
    return 0;
}

• UDP does not require multi-threading/multi-process, so multiple clients can communicate with the server.

broadcast

• Used in LAN
• Send once and all computers in the LAN can receive the message

• The broadcast address is to set all the flag bits of the host address to 1, which is 255
• Now the client also needs to bind the port used by the server, and the IP is used as the broadcast address.
• Do two more things. The server sets the broadcast attribute, and the client binds the port number of the server's broadcast data.

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <arpa/inet.h>

int main() {
    
    

    // 1.创建一个通信的socket
    int fd = socket(PF_INET, SOCK_DGRAM, 0);
    if(fd == -1) {
    
    
        perror("socket");
        exit(-1);
    }   

    // 2.设置广播属性
    int op = 1;
    setsockopt(fd, SOL_SOCKET, SO_BROADCAST, &op, sizeof(op));
    
    // 3.创建一个广播的地址
    struct sockaddr_in cliaddr;
    cliaddr.sin_family = AF_INET;
    cliaddr.sin_port = htons(9999);
    inet_pton(AF_INET, "192.168.193.255", &cliaddr.sin_addr.s_addr);

    // 3.通信
    int num = 0;
    while(1) {
    
    
       
        char sendBuf[128];
        sprintf(sendBuf, "hello, client....%d\n", num++);
        // 发送数据
        sendto(fd, sendBuf, strlen(sendBuf) + 1, 0, (struct sockaddr *)&cliaddr, sizeof(cliaddr));
        printf("广播的数据：%s\n", sendBuf);
        sleep(1);
    }

    close(fd);
    return 0;
}

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <arpa/inet.h>

int main() {
    
    

    // 1.创建一个通信的socket
    int fd = socket(PF_INET, SOCK_DGRAM, 0);
    if(fd == -1) {
    
    
        perror("socket");
        exit(-1);
    }   

    struct in_addr in;

    // 2.客户端绑定本地的IP和端口
    struct sockaddr_in addr;
    addr.sin_family = AF_INET;
    addr.sin_port = htons(9999);
    addr.sin_addr.s_addr = INADDR_ANY;

    int ret = bind(fd, (struct sockaddr *)&addr, sizeof(addr));
    if(ret == -1) {
    
    
        perror("bind");
        exit(-1);
    }

    // 3.通信
    while(1) {
    
    
        
        char buf[128];
        // 接收数据
        int num = recvfrom(fd, buf, sizeof(buf), 0, NULL, NULL);
        printf("server say : %s\n", buf);

    }

    close(fd);
    return 0;
}

• If the client is opened late, data may be lost and is unsafe, as shown in the figure. If the client is opened late, reception will start at 28

• A computer cannot be bound repeatedly, but it can be bound again in a new computer, as shown in the figure, two different hosts.

Multicast (multicast)

• Identifies a group of IP interfaces
• There are a few more steps to be added to the broadcast
• Set up multicast on the server and join the multicast group on the client

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <arpa/inet.h>

int main() {
    
    

    // 1.创建一个通信的socket
    int fd = socket(PF_INET, SOCK_DGRAM, 0);
    if(fd == -1) {
    
    
        perror("socket");
        exit(-1);
    }   

    // 2.设置多播的属性，设置外出接口
    struct in_addr imr_multiaddr;
    // 初始化多播地址
    inet_pton(AF_INET, "239.0.0.10", &imr_multiaddr.s_addr);
    setsockopt(fd, IPPROTO_IP, IP_MULTICAST_IF, &imr_multiaddr, sizeof(imr_multiaddr));
    
    // 3.初始化客户端的地址信息
    struct sockaddr_in cliaddr;
    cliaddr.sin_family = AF_INET;
    cliaddr.sin_port = htons(9999);
    inet_pton(AF_INET, "239.0.0.10", &cliaddr.sin_addr.s_addr);

    // 3.通信
    int num = 0;
    while(1) {
    
    
       
        char sendBuf[128];
        sprintf(sendBuf, "hello, client....%d\n", num++);
        // 发送数据
        sendto(fd, sendBuf, strlen(sendBuf) + 1, 0, (struct sockaddr *)&cliaddr, sizeof(cliaddr));
        printf("组播的数据：%s\n", sendBuf);
        sleep(1);
    }

    close(fd);
    return 0;
}

#include <stdio.h>
#include <stdlib.h>
#include <unistd.h>
#include <string.h>
#include <arpa/inet.h>

int main() {
    
    

    // 1.创建一个通信的socket
    int fd = socket(PF_INET, SOCK_DGRAM, 0);
    if(fd == -1) {
    
    
        perror("socket");
        exit(-1);
    }   

    struct in_addr in;
    // 2.客户端绑定本地的IP和端口
    struct sockaddr_in addr;
    addr.sin_family = AF_INET;
    addr.sin_port = htons(9999);
    addr.sin_addr.s_addr = INADDR_ANY;

    int ret = bind(fd, (struct sockaddr *)&addr, sizeof(addr));
    if(ret == -1) {
    
    
        perror("bind");
        exit(-1);
    }

    struct ip_mreq op;
    inet_pton(AF_INET, "239.0.0.10", &op.imr_multiaddr.s_addr);
    op.imr_interface.s_addr = INADDR_ANY;

    // 加入到多播组
    setsockopt(fd, IPPROTO_IP, IP_ADD_MEMBERSHIP, &op, sizeof(op));

    // 3.通信
    while(1) {
    
    
        
        char buf[128];
        // 接收数据
        int num = recvfrom(fd, buf, sizeof(buf), 0, NULL, NULL);
        printf("server say : %s\n", buf);

    }

    close(fd);
    return 0;
}

• Both hosts can receive

local socket communication

• The function is the communication of local processes (in a host). The above are called network sockets, that is, process communication between different hosts.
• The implementation process is similar to the network socket implementation, using TCP

• Use AF_LOCAL to communicate locally
• Locally use sockaddr_un
• The principle of local sockets is very similar to that of famous pipes

#include <stdio.h>
#include <string.h>
#include <unistd.h>
#include <stdlib.h>
#include <arpa/inet.h>
#include <sys/un.h>

int main() {
    
    

    unlink("server.sock");

    // 1.创建监听的套接字
    int lfd = socket(AF_LOCAL, SOCK_STREAM, 0);//本地套接字用LOCAL
    if(lfd == -1) {
    
    
        perror("socket");
        exit(-1);
    }

    // 2.绑定本地套接字文件
    struct sockaddr_un addr;
    addr.sun_family = AF_LOCAL;
    strcpy(addr.sun_path, "server.sock");//因为数组名是指针常量，是不能被修改的，所以用strcpy
    int ret = bind(lfd, (struct sockaddr *)&addr, sizeof(addr));
    if(ret == -1) {
    
    
        perror("bind");
        exit(-1);
    }

    // 3.监听
    ret = listen(lfd, 100);
    if(ret == -1) {
    
    
        perror("listen");
        exit(-1);
    }

    // 4.等待客户端连接
    struct sockaddr_un cliaddr;
    int len = sizeof(cliaddr);
    int cfd = accept(lfd, (struct sockaddr *)&cliaddr, &len);
    if(cfd == -1) {
    
    
        perror("accept");
        exit(-1);
    }

    printf("client socket filename: %s\n", cliaddr.sun_path);

    // 5.通信
    while(1) {
    
    

        char buf[128];
        int len = recv(cfd, buf, sizeof(buf), 0);

        if(len == -1) {
    
    
            perror("recv");
            exit(-1);
        } else if(len == 0) {
    
    
            printf("client closed....\n");
            break;
        } else if(len > 0) {
    
    
            printf("client say : %s\n", buf);
            send(cfd, buf, len, 0);
        }

    }

    close(cfd);
    close(lfd);

    return 0;
}

#include <stdio.h>
#include <string.h>
#include <unistd.h>
#include <stdlib.h>
#include <arpa/inet.h>
#include <sys/un.h>

int main() {
    
    

    unlink("client.sock");

    // 1.创建套接字
    int cfd = socket(AF_LOCAL, SOCK_STREAM, 0);
    if(cfd == -1) {
    
    
        perror("socket");
        exit(-1);
    }

    // 2.绑定本地套接字文件
    struct sockaddr_un addr;
    addr.sun_family = AF_LOCAL;
    strcpy(addr.sun_path, "client.sock");
    int ret = bind(cfd, (struct sockaddr *)&addr, sizeof(addr));
    if(ret == -1) {
    
    
        perror("bind");
        exit(-1);
    }

    // 3.连接服务器
    struct sockaddr_un seraddr;
    seraddr.sun_family = AF_LOCAL;
    strcpy(seraddr.sun_path, "server.sock");
    ret = connect(cfd, (struct sockaddr *)&seraddr, sizeof(seraddr));
    if(ret == -1) {
    
    
        perror("connect");
        exit(-1);
    }

    // 4.通信
    int num = 0;
    while(1) {
    
    

        // 发送数据
        char buf[128];
        sprintf(buf, "hello, i am client %d\n", num++);
        send(cfd, buf, strlen(buf) + 1, 0);
        printf("client say : %s\n", buf);

        // 接收数据
        int len = recv(cfd, buf, sizeof(buf), 0);

        if(len == -1) {
    
    
            perror("recv");
            exit(-1);
        } else if(len == 0) {
    
    
            printf("server closed....\n");
            break;
        } else if(len > 0) {
    
    
            printf("server say : %s\n", buf);
        }

        sleep(1);

    }

    close(cfd);
    return 0;
}

• Corresponding pseudo files will be generated

• Each time you run it, delete the dummy file first and then rebind it.

Web Server

Blocking/non-blocking, synchronous/asynchronous

• Mainly aimed at network IO
• two stages

• The sockfd here can be blocking or non-blocking.
• EINTER, EAGAIN, etc. are not errors, so they need to be judged.
• buf is read by ourselves, so it is synchronized
• For example, if you buy a ticket and go to the airport to print your boarding pass, this is synchronous; asynchronous means when the ticket comes out and the airport expresses it to you, it is asynchronous; asynchronous is more efficient and takes other people's time
• When dealing with IO, both blocking and non-blocking are synchronous IO. Only when special API is used is asynchronous IO.
  

IO model (an extension of IO multiplexing)

• Is the IO multiplexing API synchronous or asynchronous? Using epoll can only detect the arrival of data, and the specific read operations (read, recv) are all synchronous IO. Remember one sentence, when dealing with IO, blocking and non-blocking are both synchronous IO. Only when special API is used is asynchronous IO, so the API of IO multiplexing is synchronous.

Blocking

• The caller calls a function and waits for the function to return. During this period, he does nothing and constantly checks whether the function has returned.

You must wait for this function to return before proceeding to the next step. This is an attribute of the file descriptor.

Non-Blocking (NIO)

• Non-blocking waiting, checking whether the I/O event is ready every once in a while. Don't be ready to do other things. Non-blocking I/O execution system calls always return immediately, regardless of whether the event has occurred. If the event has not occurred, -1 is returned. At this time, the two situations can be distinguished based on errno. For accept, recv, and send, the event has not occurred. errno is usually set to EAGAIN

IO multiplexing

• Linux uses the select/poll/epoll function to implement the IO multiplexing model. These functions will also block the process, but the difference from blocking IO is that these functions can block multiple IO operations at the same time. Moreover, the IO functions of multiple read operations and write operations can be detected at the same time. The IO operation function is not actually called until there is data to read or write.
• The purpose of IO reuse is to detect multiple client events at one time (in a single process)

Signal-Driven

• Linux uses sockets for signal-driven IO and installs a signal processing function. The process continues to run without blocking. When the IO event is ready, proceed

The process receives the SIGIO signal and then handles the IO event

• The kernel is asynchronous in the first stage and synchronous in the second stage; the difference from non-blocking IO is that it provides a message notification mechanism, which does not require the user process to continuously poll and check, reducing the number of system API calls and improving efficiency

Asynchronous

• In Linux, you can call the aio_read function to tell the kernel the descriptor buffer pointer and buffer size, file offset and notification method, and then return immediately. When the kernel copies the data to the buffer, it notifies the application.

Introduction to Web Server (Web Server)

• A WebServer is a server software (program), or the hardware (computer) that runs the server software. Its main function is to communicate with the client (usually a browser) through the HTTP protocol to receive, store, and process HTTP requests from the client, make HTTP responses to its requests, and return to the client its requested information. content (files, web pages, etc.) or return an Error message

• Typically users use a web browser to communicate with the corresponding server. Type "domain name" or "IP address:port number" in the browser, and the browser will first resolve your domain name into the corresponding IP address or directly send an HTTP request to the corresponding Web server based on your IP address. This process first establishes a connection with the target Web server through the three-way handshake of the TCP protocol, and then uses the HTTP protocol to generate an HTTP request message for the target Web server, which is sent to the target Web server through TCP, IP and other protocols.

HTTP

• Hypertext Transfer Protocol (HTTP) is a simple request-response protocol that usually runs on top of TCP. It specifies what kind of messages the client may send to the server and what kind of response it gets. The headers of request and response messages are given in ASCII form; the message contents have a MIME-like format. HTTP is the basis for data communication on the World Wide Web

principle

• The HTTP protocol defines how a Web client requests a Web page from a Web server, and how the server delivers the Web page to the client. The HTTP protocol uses a request/response model. The client sends a request message to the server. The request message contains the request method, URL, protocol version, request header and request data. The server responds with a status line, which includes the protocol version, success or error code, server information, response headers, and response data.
• The following are the steps for HTTP request/response:

1. Client Connection to Web Server
   An HTTP client, usually a browser, establishes a TCP socket connection with the Web server's HTTP port (default is 80). For example, http://www.baidu.com. (URL)
2. To send an HTTP request
   through a TCP socket, the client sends a text request message to the Web server. A request message consists of four parts: request line, request header, blank line and request data.
3. The server accepts the request and returns an HTTP response.
   The Web server parses the request and locates the requested resource. The server writes a copy of the resource to the TCP socket, which is read by the client. A response consists of 4 parts: status line, response header, blank line and response data.
4. Release the connection TCP connection.
   If the connection mode is close, the server actively closes the TCP connection, and the client passively closes the connection and releases the TCP connection. If the connection mode is keepalive, the connection will be maintained for a period of time, and requests can continue to be received during this time;
5. Client Browser Parses HTML Content
   The client browser first parses the status line to see a status code indicating whether the request was successful. Each response header is then parsed, and the response header tells the following HTML document, which is a number of bytes, and the character set of the document. The client browser reads the response data HTML, formats it according to the syntax of HTML, and displays it in the browser window.
   For example: type the URL in the browser address bar and press Enter, you will go through the following process:
6. The browser requests the DNS server to resolve the IP address corresponding to the domain name in the URL;
7. After parsing the IP address, establish a TCP connection with the server based on the IP address and the default port 80;
8. The browser issues an HTTP request to read a file (the file corresponding to the part after the domain name in the URL), and the request message is sent to the server as the data of the third message of the TCP three-way handshake;
9. The server responds to the browser request and sends the corresponding HTML text to the browser;
10. Release the TCP connection;
11. The browser converts the HTML text and displays the content

• The HTTP protocol is an application layer protocol based on the TCP/IP protocol, based on the request-response model. The HTTP protocol stipulates that a request is issued from the client, and finally the server responds to the request and returns. In other words, communication must be established first from the client, and the server will not send a response until it receives the request.
  

HTTP request message

• You can see the request message through the browser

HTTP response message

HTTP request

• The HTTP/1.1 protocol defines a total of eight methods (also called "actions") to operate specified resources in different ways:

1. GET: Makes a "show" request to the specified resource. The use of the GET method should only be used to read data and should not be used in operations that produce "side effects", such as in Web Applications. One reason is that GET may be
   accessed .
2. HEAD: Like the GET method, it makes a request for the specified resource to the server. It's just that the server will not return the text part of the resource. The advantage is that using this method can obtain the "information about the resource" (metainformation or metadata) without having to transmit the entire content.
3. POST: Submit data to the specified resource and request the server for processing (such as submitting a form or uploading a file). The data is included in the request article. This request may create a new resource or modify an existing resource, or both.
4. PUT: Upload the latest content to the specified resource location.
5. DELETE: Request the server to delete the resource identified by Request-URI.
6. TRACE: Echo the request received by the server, mainly used for testing or diagnosis.
7. OPTIONS: This method allows the server to return all HTTP request methods supported by the resource. Use '*' to replace the resource name and send an OPTIONS request to the Web server to test whether the server function is functioning properly.
8. CONNECT: Reserved in the HTTP/1.1 protocol for proxy servers that can change connections to pipelines. Typically used for connections to SSL encrypted servers (via unencrypted HTTP proxies).

• Commonly used GET/POST

Basic framework of server programming

• Although there are many types of server programs, their basic frameworks are the same. The difference lies in the logical processing.

module	Function
I/O processing unit	Handle client connections, read and write network data
logic unit	Business process or thread
network storage unit	Database, file or cache
request queue	Communication methods between units

• The I/O processing unit is the server's module that manages client connections. It usually completes the following tasks: waits for and accepts new client connections, receives client data, and returns server response data to the client. However, the sending and receiving of data may not necessarily be performed in the I/O processing unit, but may also be performed in the logical unit. The specific location depends on the event processing mode.
• A logical unit is usually a process or thread. It analyzes and processes customer data, and then passes the results to the I/O processing unit or directly to the client (which method is used depends on the event processing mode). Servers usually have multiple logical units to enable concurrent processing of multiple client tasks.
• Network storage units can be databases, caches, and files, but they do not have to be.
• The request queue is an abstraction of how units communicate. When the I/O processing unit receives a client request, it needs to notify a logical unit in some way to process the request. Similarly, when multiple logical units access a storage unit at the same time, some mechanism is also required to coordinate and handle race conditions. Request queues are usually implemented as part of a pool. (Pool: process pool and thread pool)
  

Event handling models of two universities

• Server programs usually need to handle three types of events: I/O events, signals and timed events. There are two efficient event processing modes: Reactor and Proactor. The synchronous I/O model is usually used to implement the Reactor mode, and the asynchronous I/O model is usually used to implement the Proactor mode.
  

Reactor pattern

The main thread (I/O processing unit) is required to only monitor whether an event occurs on the file descriptor. If so, it will immediately notify the worker thread (logical unit) of the event and put the socket readable and writable events into the request queue. Hand it over to the worker thread. Apart from this, the main thread does not do any other substantive work. Reading and writing data, accepting new connections, and processing customer requests are all done in worker threads.
The workflow of the Reactor mode implemented using synchronous I/O (taking epoll_wait as an example) is:

1. The main thread registers the read ready event on the socket in the epoll kernel event table .
2. The main thread calls epoll_wait to wait for data to be read on the socket.
3. When there is data to read on the socket, epoll_wait notifies the main thread. The main thread puts the socket readable event into the request queue.
4. A worker thread sleeping on the request queue is awakened, it reads data from the socket, processes the client request, and then
   registers the write-ready event on the socket in the epoll kernel event table.
5. When the main thread calls epoll_wait, it waits for the socket to be writable.
6. When the socket is writable, epoll_wait notifies the main thread. The main thread puts the socket writable event into the request queue.
7. A worker thread sleeping on the request queue is awakened, and it writes the result of the server processing the client request to the socket.

Proactor mode

Proactor mode hands all I/O operations to the main thread and kernel for processing (reading and writing), and the worker thread is only responsible for business logic. The workflow of the Proactor mode implemented using the asynchronous I/O model (taking aio_read and aio_write as examples) is:

1. The main thread calls the aio_read function to register the read completion event on the socket with the kernel, and tells the kernel user the location of the read buffer and how to notify the application when the read operation is completed (here, take the signal as an example).
2. The main thread continues processing other logic.
3. When the data on the socket is read into the user buffer, the kernel will send a signal to the application to notify the application that the data is available.
4. The application's predefined signal handling function selects a worker thread to handle the client request. After the worker thread processes the client request, it calls the aio_write function to register the write completion event on the socket with the kernel, and tells the kernel user the location of the write buffer and how to notify the application when the write operation is completed.
5. The main thread continues processing other logic.
6. When the data in the user buffer is written to the socket, the kernel will send a signal to the application to notify the application that the data has been sent.
7. The signal processing function predefined by the application selects a worker thread to do the aftermath processing, such as deciding whether to close the socket.

• The difference is that in R mode, the main thread is only responsible for monitoring, while in P mode, everything is left to the main thread.
  

Simulate Proactor mode

Use synchronous I/O method to simulate Proactor mode. The principle is: the main thread performs data read and write operations. After the read and write is completed, the main thread notifies the working thread of this "completion event". So from the perspective of the worker threads, they directly obtain the results of data reading and writing, and the next thing to do is to logically process the results of reading and writing.
The workflow of the Proactor mode simulated using the synchronous I/O model (taking epoll_wait as an example) is as follows:

1. The main thread registers the read ready event on the socket in the epoll kernel event table.
2. The main thread calls epoll_wait to wait for data to be read on the socket.
3. When there is data to read on the socket, epoll_wait notifies the main thread. The main thread reads data from the socket in a loop until there is no more data to read, and then encapsulates the read data into a request object and inserts it into the request queue.
4. A worker thread sleeping on the request queue is awakened, it obtains the request object and processes the client request, and then registers the write-ready event on the socket in the epoll kernel event table.
5. The main thread calls epoll_wait to wait for the socket to be writable.
6. When the socket is writable, epoll_wait notifies the main thread. The main thread writes the result of the server processing the client request to the socket.

Thread Pool

• It is not good to create a thread when a client comes.
• The thread pool is a group of sub-threads pre-created by the server. The number of threads in the thread pool should be approximately the same as the number of CPUs. All child threads in the thread pool run the same code. When a new task arrives, the main thread will select a sub-thread in the thread pool to serve it in some way. Compared with dynamically creating sub-threads, the cost of selecting an existing sub-thread is obviously much smaller. As for which sub-thread the main thread chooses to serve the new task, there are many ways:
• The main thread uses some algorithm to actively select child threads. The simplest and most commonly used algorithms are the random algorithm and the RoundRobin algorithm, but better and smarter algorithms will distribute tasks more evenly among the various worker threads, thereby reducing the overall pressure on the server.
• The main thread and all child threads are synchronized through a shared work queue, and the child threads sleep on the work queue. When a new task arrives, the main thread adds the task to the work queue. This will wake up the child threads that are waiting for the task, but only one child thread will get the "takeover right" of the new task. It can take the task from the work queue and execute it, while other child threads will continue to sleep on the work queue.
• The general model of a thread pool is:

The most direct limiting factor for the number of threads in the thread pool is the number N of processors/cores of the central processing unit (CPU)
: if your CPU is 4-cores, for CPU-intensive tasks (such as video editing) For tasks that consume CPU computing resources), it is best to set the number of threads in the thread pool to 4 (or +1 to prevent thread blocking caused by other factors); for IO-intensive tasks, it is generally more than the number of CPU Number of cores, because threads compete not for CPU computing resources but for IO, and IO processing is generally slower. Threads with more cores will compete for more tasks for the CPU, preventing the CPU from being idle while the threads are processing IO. Lead to waste of resources.

• Space is exchanged for time, and server hardware resources are wasted in exchange for operating efficiency.
• A pool is a collection of resources that are completely created and initialized when the server starts. This is called a static resource. When the server enters the official operation stage and starts processing customer requests, if it needs related resources, it can obtain them directly from the pool without dynamic allocation.
• After the server has processed a client connection, it can put the related resources back into the pool without executing a system call to release the resources.
  

Finite State Machine

• An efficient programming method inside a logic unit: finite state machine. Some application layer protocol headers contain a packet type field. Each type can be mapped to an execution state of a logical unit, and the server can write corresponding processing logic based on it. The following is a state-independent finite state machine:

STATE_MACHINE( Package _pack )
{
    
    
	PackageType _type = _pack.GetType(); switch( _type )
	{
    
    
		case type_A:
    	process_package_A( _pack ); break;
    	case type_B:
    	process_package_B( _pack ); break;
	}
}

• This is a simple finite state machine, except that each state of the state machine is independent of each other, that is, there is no transfer between states. The transition between states requires the internal driver of the state machine, as shown in the following code:

STATE_MACHINE()
{
    
    
    State cur_State = type_A;
    while( cur_State != type_C )
    {
    
    
        Package _pack = getNewPackage(); 
        switch( cur_State )
        {
    
    
            case type_A:
            	process_package_state_A( _pack );
                cur_State = type_B;
            	break;
            case type_B:
            	process_package_state_B( _pack ); 
                cur_State = type_C;
            break;
    	}
	}
}

• The state machine contains three states: type_A, type_B and type_C, where type_A is the start state of the state machine and type_C is the end state of the state machine. The current state of the state machine is recorded in the cur_State variable. During a loop, the state machine first obtains a new data packet through the getNewPackage method, and then determines how to process the data packet based on the value of the cur_State variable. After the data packet is processed, the state machine implements state transition by passing the target state value to the cur_State variable. Then when the state machine enters the next cycle, it will execute the logic corresponding to the new state

EPOLLONESHOT EVENT

• Even if ET mode can be used, an event on a socket may still be triggered multiple times. This can cause a problem in concurrent programs. For example, a thread starts processing the data after reading the data on a certain socket. During the data processing process, new data is readable on the socket (EPOLLIN is triggered again). At this time, another thread is awakened. Read this new data. So there is a situation where two threads operate a socket at the same time. A socket connection is processed by only one thread at any time, which can be achieved using the EPOLLONESHOT event of epoll.
• For a file descriptor registered with the EPOLLONESHOT event, the operating system triggers at most one readable, writable or exception event registered on it, and only triggers it once, unless we use the epoll_ctl function to reset the EPOLLONESHOT event registered on the file descriptor. In this way, when one thread is processing a certain socket, other threads cannot have the opportunity to operate the socket. But thinking about it the other way around, once the socket registered with the EPOLLONESHOT event is processed by a thread, the thread should immediately reset the EPOLLONESHOT event on the socket to ensure that the EPOLLIN event can be triggered the next time the socket is readable. This gives other worker threads the opportunity to continue processing this socket.
  

accomplish

• Thread pool is used together with mutex lock
• So first create a mutually exclusive lock header file (the code is simple and puts the .h file and the operation file together)

#ifndef LOCKER_H
#define LOCKER_H

#include <exception>..异常
#include <pthread.h>
#include <semaphore.h>

// 线程同步机制封装类

// 互斥锁类
class locker {
    
    
public:
    locker() {
    
    //构造函数
        if(pthread_mutex_init(&m_mutex, NULL) != 0) {
    
     //返回值不等于0就是出错了
            throw std::exception();//抛出异常
        }
    }

    ~locker() {
    
     //析构函数
        pthread_mutex_destroy(&m_mutex);
    }

    bool lock() {
    
    
        return pthread_mutex_lock(&m_mutex) == 0;
    }

    bool unlock() {
    
    
        return pthread_mutex_unlock(&m_mutex) == 0;
    }

    pthread_mutex_t *get()//获取互斥量
    {
    
    
        return &m_mutex;
    }

private:
    pthread_mutex_t m_mutex;
};


// 条件变量类，判断队列当中是否有数据
class cond {
    
    
public:
    cond(){
    
    
        if (pthread_cond_init(&m_cond, NULL) != 0) {
    
    
            throw std::exception();
        }
    }
    ~cond() {
    
    
        pthread_cond_destroy(&m_cond);
    }

    bool wait(pthread_mutex_t *m_mutex) {
    
    
        int ret = 0;
        ret = pthread_cond_wait(&m_cond, m_mutex);
        return ret == 0;
    }
    bool timewait(pthread_mutex_t *m_mutex, struct timespec t) {
    
    
        int ret = 0;
        ret = pthread_cond_timedwait(&m_cond, m_mutex, &t);
        return ret == 0;
    }
    bool signal() {
    
    
        return pthread_cond_signal(&m_cond) == 0;
    }
    bool broadcast() {
    
    
        return pthread_cond_broadcast(&m_cond) == 0;
    }

private:
    pthread_cond_t m_cond;
};


// 信号量类
class sem {
    
    
public:
    sem() {
    
    
        if( sem_init( &m_sem, 0, 0 ) != 0 ) {
    
    
            throw std::exception();
        }
    }
    sem(int num) {
    
    
        if( sem_init( &m_sem, 0, num ) != 0 ) {
    
    
            throw std::exception();
        }
    }
    ~sem() {
    
    
        sem_destroy( &m_sem );
    }
    // 等待信号量
    bool wait() {
    
    
        return sem_wait( &m_sem ) == 0;
    }
    // 增加信号量
    bool post() {
    
    
        return sem_post( &m_sem ) == 0;
    }
private:
    sem_t m_sem;
};

#endif

• Then create the thread pool class

#ifndef THREADPOOL_H
#define THREADPOOL_H

#include <list>
#include <cstdio>
#include <exception>
#include <pthread.h>
#include "locker.h"

// 线程池类，将它定义为模板类是为了代码复用，模板参数T是任务类
template<typename T>
class threadpool {
    
    
public:
    /*thread_number是线程池中线程的数量，max_requests是请求队列中最多允许的、等待处理的请求的数量*/
    threadpool(int thread_number = 8, int max_requests = 10000);//构造
    ~threadpool();//析构
    bool append(T* request);//添加任务

private:
    /*工作线程运行的函数，它不断从工作队列中取出任务并执行之*/
    static void* worker(void* arg);
    void run();

private:
    // 线程的数量
    int m_thread_number;  
    
    // 描述线程池的数组，大小为m_thread_number    
    pthread_t * m_threads;//动态创建数组

    // 请求队列中最多允许的、等待处理的请求的数量  
    int m_max_requests; 
    
    // 请求队列
    std::list< T* > m_workqueue;  

    // 保护请求队列的互斥锁
    locker m_queuelocker;   

    // 是否有任务需要处理
    sem m_queuestat;//状态

    // 是否结束线程          
    bool m_stop;                    
};

template< typename T >//实现构造函数
threadpool< T >::threadpool(int thread_number, int max_requests) : //冒号后可以对成员进行初始化
        m_thread_number(thread_number), m_max_requests(max_requests), 
        m_stop(false), m_threads(NULL) {
    
    

    if((thread_number <= 0) || (max_requests <= 0) ) {
    
     //小于0就是错误值，抛出异常
        throw std::exception();
    }

    m_threads = new pthread_t[m_thread_number]; //创建数组，new动态创建
    if(!m_threads) {
    
    
        throw std::exception();
    }

    // 创建 thread_number 个线程，并将他们设置为脱离线程。自己释放资源
    for ( int i = 0; i < thread_number; ++i ) {
    
    
        printf( "create the %dth thread\n", i);
        if(pthread_create(m_threads + i, NULL, worker, this ) != 0) {
    
     //worker必须是静态函数，C中是全局函数
            //第一个参数为地址
            //不等于0代表出错
            delete [] m_threads;
            throw std::exception();
        }
        
        if( pthread_detach( m_threads[i] ) ) {
    
     //出错了线程分离
            delete [] m_threads;
            throw std::exception();
        }
    }
}

template< typename T >
threadpool< T >::~threadpool() {
    
     //析构函数
    delete [] m_threads;
    m_stop = true;
}

template< typename T > //添加事件
bool threadpool< T >::append( T* request )
{
    
    
    // 操作工作队列时一定要加锁，因为它被所有线程共享。
    m_queuelocker.lock();
    if ( m_workqueue.size() > m_max_requests ) {
    
     //不能超过最大的请求
        m_queuelocker.unlock();
        return false;
    }
    m_workqueue.push_back(request);//追加
    m_queuelocker.unlock();//解锁
    m_queuestat.post();//信号量增加
    return true;
}

template< typename T >
void* threadpool< T >::worker( void* arg ) //实现worker，但是可以使用this将参数传递过来
{
    
    
    threadpool* pool = ( threadpool* )arg;
    pool->run();//线程创建出来就执行
    return pool;
}

template< typename T >
void threadpool< T >::run() {
    
     //实现run

    while (!m_stop) {
    
     //一直循环
        m_queuestat.wait();
        m_queuelocker.lock();
        if ( m_workqueue.empty() ) {
    
    
            m_queuelocker.unlock();
            continue;
        }
        T* request = m_workqueue.front();//取出第一个任务
        m_workqueue.pop_front();//取了就删掉，删掉第一个
        m_queuelocker.unlock();
        if ( !request ) {
    
    
            continue;
        }
        request->process();
    }

}

#endif

• Write socket-related main files

#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <stdio.h>
#include <unistd.h>
#include <errno.h>
#include <string.h>
#include <fcntl.h>
#include <stdlib.h>
#include <sys/epoll.h>
#include "locker.h"
#include "threadpool.h"
#include "http_conn.h"

#define MAX_FD 65536   // 最大的文件描述符个数
#define MAX_EVENT_NUMBER 10000  // 监听的最大的事件数量

// 添加文件描述符
extern void addfd( int epollfd, int fd, bool one_shot );//加入exteren这个就是为了在别的文件中也能用
extern void removefd( int epollfd, int fd );

void addsig(int sig, void( handler )(int)){
    
     //添加信号捕捉
    struct sigaction sa; //注册信号
    memset( &sa, '\0', sizeof( sa ) );
    sa.sa_handler = handler;
    sigfillset( &sa.sa_mask );//设置临时阻塞信号集
    assert( sigaction( sig, &sa, NULL ) != -1 );
}

int main( int argc, char* argv[] ) {
    
    
    
    if( argc <= 1 ) {
    
    
        printf( "usage: %s port_number\n", basename(argv[0]));
        return 1;
    }

    int port = atoi( argv[1] ); //转换成整数
    addsig( SIGPIPE, SIG_IGN );//信号处理，捕捉到了信号就忽略他

    threadpool< http_conn >* pool = NULL; //任务是http连接的任务
    try {
    
    
        pool = new threadpool<http_conn>;//创建
    } catch( ... ) {
    
    
        return 1;
    }

    http_conn* users = new http_conn[ MAX_FD ];//最大用户数

    int listenfd = socket( PF_INET, SOCK_STREAM, 0 );//监听套接字

    int ret = 0;
    struct sockaddr_in address;
    address.sin_addr.s_addr = INADDR_ANY;
    address.sin_family = AF_INET;
    address.sin_port = htons( port );//网络字节序

    // 端口复用，在绑定之前进行设置
    int reuse = 1;//为1就是复用
    //SOL_SOCKET是级别，SO_REUSEADDR代表端口复用
    setsockopt( listenfd, SOL_SOCKET, SO_REUSEADDR, &reuse, sizeof( reuse ) );
    ret = bind( listenfd, ( struct sockaddr* )&address, sizeof( address ) );//绑定
    ret = listen( listenfd, 5 );//监听

    // 创建epoll对象，和事件数组，添加，多路复用
    epoll_event events[ MAX_EVENT_NUMBER ];//检测到了把事件写入数组
    int epollfd = epoll_create( 5 );
    // 添加到epoll对象中
    addfd( epollfd, listenfd, false );//添加文件描述符
    http_conn::m_epollfd = epollfd;//静态成员

    while(true) {
    
     //主线程不断循环检测事件发生
        
        int number = epoll_wait( epollfd, events, MAX_EVENT_NUMBER, -1 ); //检测到了几个事件
        
        if ( ( number < 0 ) && ( errno != EINTR ) ) {
    
    
            printf( "epoll failure\n" );
            break;
        }

        for ( int i = 0; i < number; i++ ) {
    
     //循环遍历事件数组
            
            int sockfd = events[i].data.fd;//获取监听的文件描述符
            
            if( sockfd == listenfd ) {
    
    //有客户端连接进来
                
                struct sockaddr_in client_address;
                socklen_t client_addrlength = sizeof( client_address );
                int connfd = accept( listenfd, ( struct sockaddr* )&client_address, &client_addrlength );
                //连接客户端
                
                if ( connfd < 0 ) {
    
    
                    printf( "errno is: %d\n", errno );
                    continue;
                } 

                if( http_conn::m_user_count >= MAX_FD ) {
    
     //最大用户数不能超
                    close(connfd);
                    continue;
                }
                users[connfd].init( connfd, client_address);//直接把sockfd作为索引去操作

            } else if( events[i].events & ( EPOLLRDHUP | EPOLLHUP | EPOLLERR ) ) {
    
    
            	//对方异常断开
                users[sockfd].close_conn();//非监听fd的判断，即为通信fd

            } else if(events[i].events & EPOLLIN) {
    
     //读的事件发生，模拟P模式一次性要把数据都读出来

                if(users[sockfd].read()) {
    
     //一次性把所有数据读完
                    pool->append(users + sockfd);//交给工作线程处理，地址直接相加
                } else {
    
    
                    users[sockfd].close_conn();//读失败了，关闭连接
                }

            }  else if( events[i].events & EPOLLOUT ) {
    
     //检测写事件

                if( !users[sockfd].write() ) {
    
     
                    users[sockfd].close_conn(); 
                }

            }
        }
    }
    //程序结束
    close( epollfd );
    close( listenfd );
    delete [] users;
    delete pool;
    return 0;
}

• Define http header file

#ifndef HTTPCONNECTION_H
#define HTTPCONNECTION_H

#include <unistd.h>
#include <signal.h>
#include <sys/types.h>
#include <sys/epoll.h>
#include <fcntl.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <assert.h>
#include <sys/stat.h>
#include <string.h>
#include <pthread.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/mman.h>
#include <stdarg.h>
#include <errno.h>
#include "locker.h"
#include <sys/uio.h>

class http_conn
{
    
    
public:
    static const int FILENAME_LEN = 200;        // 文件名的最大长度
    static const int READ_BUFFER_SIZE = 2048;   // 读缓冲区的大小
    static const int WRITE_BUFFER_SIZE = 1024;  // 写缓冲区的大小
    
    // HTTP请求方法，这里只支持GET
    enum METHOD {
    
    GET = 0, POST, HEAD, PUT, DELETE, TRACE, OPTIONS, CONNECT};
    
    /*
        解析客户端请求时，主状态机的状态
        CHECK_STATE_REQUESTLINE:当前正在分析请求行
        CHECK_STATE_HEADER:当前正在分析头部字段
        CHECK_STATE_CONTENT:当前正在解析请求体
    */
    enum CHECK_STATE {
    
     CHECK_STATE_REQUESTLINE = 0, CHECK_STATE_HEADER, CHECK_STATE_CONTENT };
    
    /*
        服务器处理HTTP请求的可能结果，报文解析的结果
        NO_REQUEST          :   请求不完整，需要继续读取客户数据
        GET_REQUEST         :   表示获得了一个完成的客户请求
        BAD_REQUEST         :   表示客户请求语法错误
        NO_RESOURCE         :   表示服务器没有资源
        FORBIDDEN_REQUEST   :   表示客户对资源没有足够的访问权限
        FILE_REQUEST        :   文件请求,获取文件成功
        INTERNAL_ERROR      :   表示服务器内部错误
        CLOSED_CONNECTION   :   表示客户端已经关闭连接了
    */
    enum HTTP_CODE {
    
     NO_REQUEST, GET_REQUEST, BAD_REQUEST, NO_RESOURCE, FORBIDDEN_REQUEST, FILE_REQUEST, INTERNAL_ERROR, CLOSED_CONNECTION };
    
    // 从状态机的三种可能状态，即行的读取状态，分别表示
    // 1.读取到一个完整的行 2.行出错 3.行数据尚且不完整
    enum LINE_STATUS {
    
     LINE_OK = 0, LINE_BAD, LINE_OPEN };
public:
    http_conn(){
    
    }//构造与析构
    ~http_conn(){
    
    }
public:
    void init(int sockfd, const sockaddr_in& addr); // 初始化新接受的连接
    void close_conn();  // 关闭连接
    void process(); // 处理客户端请求，响应也在这里处理
    bool read();// 非阻塞读
    bool write();// 非阻塞写
private:
    void init();    // 初始化连接
    HTTP_CODE process_read();    // 解析HTTP请求
    bool process_write( HTTP_CODE ret );    // 填充HTTP应答

    // 下面这一组函数被process_read调用以分析HTTP请求
    HTTP_CODE parse_request_line( char* text );
    HTTP_CODE parse_headers( char* text );
    HTTP_CODE parse_content( char* text );
    HTTP_CODE do_request();
    char* get_line() {
    
     return m_read_buf + m_start_line; }
    LINE_STATUS parse_line();

    // 这一组函数被process_write调用以填充HTTP应答。
    void unmap();
    bool add_response( const char* format, ... );
    bool add_content( const char* content );
    bool add_content_type();
    bool add_status_line( int status, const char* title );
    bool add_headers( int content_length );
    bool add_content_length( int content_length );
    bool add_linger();
    bool add_blank_line();

public:
    static int m_epollfd;       // 所有socket上的事件都被注册到同一个epoll内核事件中，所以设置成静态的
    static int m_user_count;    // 统计用户的数量

private:
    int m_sockfd;           // 该HTTP连接的socket和对方的socket地址
    sockaddr_in m_address; //通信socket地址
    
    char m_read_buf[ READ_BUFFER_SIZE ];    // 读缓冲区
    int m_read_idx;                         // 标识读缓冲区中已经读入的客户端数据的最后一个字节的下一个位置
    int m_checked_idx;                      // 当前正在分析的字符在读缓冲区中的位置
    int m_start_line;                       // 当前正在解析的行的起始位置

    CHECK_STATE m_check_state;              // 主状态机当前所处的状态
    METHOD m_method;                        // 请求方法

    char m_real_file[ FILENAME_LEN ];       // 客户请求的目标文件的完整路径，其内容等于 doc_root + m_url, doc_root是网站根目录
    char* m_url;                            // 客户请求的目标文件的文件名
    char* m_version;                        // HTTP协议版本号，我们仅支持HTTP1.1
    char* m_host;                           // 主机名
    int m_content_length;                   // HTTP请求的消息总长度
    bool m_linger;                          // HTTP请求是否要求保持连接

    char m_write_buf[ WRITE_BUFFER_SIZE ];  // 写缓冲区
    int m_write_idx;                        // 写缓冲区中待发送的字节数
    char* m_file_address;                   // 客户请求的目标文件被mmap到内存中的起始位置
    struct stat m_file_stat;                // 目标文件的状态。通过它我们可以判断文件是否存在、是否为目录、是否可读，并获取文件大小等信息
    struct iovec m_iv[2];                   // 我们将采用writev来执行写操作，所以定义下面两个成员，其中m_iv_count表示被写内存块的数量。
    int m_iv_count;
};

#endif

• Source File

#include "http_conn.h"

// 定义HTTP响应的一些状态信息
const char* ok_200_title = "OK";
const char* error_400_title = "Bad Request";
const char* error_400_form = "Your request has bad syntax or is inherently impossible to satisfy.\n";
const char* error_403_title = "Forbidden";
const char* error_403_form = "You do not have permission to get file from this server.\n";
const char* error_404_title = "Not Found";
const char* error_404_form = "The requested file was not found on this server.\n";
const char* error_500_title = "Internal Error";
const char* error_500_form = "There was an unusual problem serving the requested file.\n";

// 网站的根目录
const char* doc_root = "/home/kagome/webserver/resources";

int setnonblocking( int fd ) {
    
     //设置文件描述符非阻塞
    int old_option = fcntl( fd, F_GETFL );
    int new_option = old_option | O_NONBLOCK;
    fcntl( fd, F_SETFL, new_option );
    return old_option;
}

// 向epoll中添加需要监听的文件描述符
void addfd( int epollfd, int fd, bool one_shot ) {
    
    
    epoll_event event;
    event.data.fd = fd;
    event.events = EPOLLIN | EPOLLRDHUP;//水平触发/边沿触发
    if(one_shot) 
    {
    
    
        // 防止同一个通信被不同的线程处理
        event.events |= EPOLLONESHOT;
    }
    epoll_ctl(epollfd, EPOLL_CTL_ADD, fd, &event);
    // 设置文件描述符非阻塞
    setnonblocking(fd);  //ET模式要全给，非阻塞
}

// 从epoll中移除监听的文件描述符
void removefd( int epollfd, int fd ) {
    
    
    epoll_ctl( epollfd, EPOLL_CTL_DEL, fd, 0 );
    close(fd);
}

// 修改文件描述符，重置socket上的EPOLLONESHOT事件，以确保下一次可读时，EPOLLIN事件能被触发
void modfd(int epollfd, int fd, int ev) {
    
    
    epoll_event event;
    event.data.fd = fd;
    event.events = ev | EPOLLET | EPOLLONESHOT | EPOLLRDHUP;
    epoll_ctl( epollfd, EPOLL_CTL_MOD, fd, &event );
}

// 所有的客户数
int http_conn::m_user_count = 0;
// 所有socket上的事件都被注册到同一个epoll内核事件中，所以设置成静态的
int http_conn::m_epollfd = -1;

// 关闭连接
void http_conn::close_conn() {
    
    
    if(m_sockfd != -1) {
    
    
        removefd(m_epollfd, m_sockfd);//删除
        m_sockfd = -1;
        m_user_count--; // 关闭一个连接，将客户总数量-1
    }
}

// 初始化连接,外部调用初始化套接字地址
void http_conn::init(int sockfd, const sockaddr_in& addr){
    
    
    m_sockfd = sockfd;
    m_address = addr;
    
    // 端口复用
    int reuse = 1;
    setsockopt( m_sockfd, SOL_SOCKET, SO_REUSEADDR, &reuse, sizeof( reuse ) );
    addfd( m_epollfd, sockfd, true);//添加到epoll，主线程只负责监听，其余交给http_conn
    m_user_count++;
    init();
}

void http_conn::init()
{
    
    
    m_check_state = CHECK_STATE_REQUESTLINE;    // 初始状态为检查请求行
    m_linger = false;       // 默认不保持链接  Connection : keep-alive保持连接

    m_method = GET;         // 默认请求方式为GET
    m_url = 0;              
    m_version = 0;
    m_content_length = 0;
    m_host = 0;
    m_start_line = 0;
    m_checked_idx = 0;
    m_read_idx = 0;
    m_write_idx = 0;
    bzero(m_read_buf, READ_BUFFER_SIZE);
    bzero(m_write_buf, READ_BUFFER_SIZE);
    bzero(m_real_file, FILENAME_LEN);
}

// 循环读取客户数据，直到无数据可读或者对方关闭连接
bool http_conn::read() {
    
    
    if( m_read_idx >= READ_BUFFER_SIZE ) {
    
    
        return false;
    }
    int bytes_read = 0;
    while(true) {
    
    
        // 从m_read_buf + m_read_idx索引出开始保存数据，大小是READ_BUFFER_SIZE - m_read_idx
        bytes_read = recv(m_sockfd, m_read_buf + m_read_idx, 
        READ_BUFFER_SIZE - m_read_idx, 0 );
        if (bytes_read == -1) {
    
    
            if( errno == EAGAIN || errno == EWOULDBLOCK ) {
    
    
                // 没有数据
                break;
            }
            return false;   
        } else if (bytes_read == 0) {
    
       // 对方关闭连接
            return false;
        }
        m_read_idx += bytes_read;
    }
    return true;
}

// 解析一行，判断依据\r\n
http_conn::LINE_STATUS http_conn::parse_line() {
    
    
    char temp;
    for ( ; m_checked_idx < m_read_idx; ++m_checked_idx ) {
    
    
        temp = m_read_buf[ m_checked_idx ];
        if ( temp == '\r' ) {
    
    
            if ( ( m_checked_idx + 1 ) == m_read_idx ) {
    
    
                return LINE_OPEN;
            } else if ( m_read_buf[ m_checked_idx + 1 ] == '\n' ) {
    
    
                m_read_buf[ m_checked_idx++ ] = '\0';
                m_read_buf[ m_checked_idx++ ] = '\0';
                return LINE_OK;
            }
            return LINE_BAD;
        } else if( temp == '\n' )  {
    
    
            if( ( m_checked_idx > 1) && ( m_read_buf[ m_checked_idx - 1 ] == '\r' ) ) {
    
    
                m_read_buf[ m_checked_idx-1 ] = '\0';
                m_read_buf[ m_checked_idx++ ] = '\0';
                return LINE_OK;
            }
            return LINE_BAD;
        }
    }
    return LINE_OPEN;
}

// 解析HTTP请求行，获得请求方法，目标URL,以及HTTP版本号
http_conn::HTTP_CODE http_conn::parse_request_line(char* text) {
    
    
    // GET /index.html HTTP/1.1
    m_url = strpbrk(text, " \t"); // 判断第二个参数中的字符哪个在text中最先出现
    if (! m_url) {
    
     
        return BAD_REQUEST;
    }
    // GET\0/index.html HTTP/1.1
    *m_url++ = '\0';    // 置位空字符，字符串结束符
    char* method = text;
    if ( strcasecmp(method, "GET") == 0 ) {
    
     // 忽略大小写比较
        m_method = GET;
    } else {
    
    
        return BAD_REQUEST;
    }
    // /index.html HTTP/1.1
    // 检索字符串 str1 中第一个不在字符串 str2 中出现的字符下标。
    m_version = strpbrk( m_url, " \t" );
    if (!m_version) {
    
    
        return BAD_REQUEST;
    }
    *m_version++ = '\0';
    if (strcasecmp( m_version, "HTTP/1.1") != 0 ) {
    
    
        return BAD_REQUEST;
    }
    /**
     * http://192.168.110.129:10000/index.html
    */
    if (strncasecmp(m_url, "http://", 7) == 0 ) {
    
       
        m_url += 7;
        // 在参数 str 所指向的字符串中搜索第一次出现字符 c（一个无符号字符）的位置。
        m_url = strchr( m_url, '/' );
    }
    if ( !m_url || m_url[0] != '/' ) {
    
    
        return BAD_REQUEST;
    }
    m_check_state = CHECK_STATE_HEADER; // 检查状态变成检查头
    return NO_REQUEST;
}

// 解析HTTP请求的一个头部信息
http_conn::HTTP_CODE http_conn::parse_headers(char* text) {
    
       
    // 遇到空行，表示头部字段解析完毕
    if( text[0] == '\0' ) {
    
    
        // 如果HTTP请求有消息体，则还需要读取m_content_length字节的消息体，
        // 状态机转移到CHECK_STATE_CONTENT状态
        if ( m_content_length != 0 ) {
    
    
            m_check_state = CHECK_STATE_CONTENT;
            return NO_REQUEST;
        }
        // 否则说明我们已经得到了一个完整的HTTP请求
        return GET_REQUEST;
    } else if ( strncasecmp( text, "Connection:", 11 ) == 0 ) {
    
    
        // 处理Connection 头部字段  Connection: keep-alive
        text += 11;
        text += strspn( text, " \t" );
        if ( strcasecmp( text, "keep-alive" ) == 0 ) {
    
    
            m_linger = true;
        }
    } else if ( strncasecmp( text, "Content-Length:", 15 ) == 0 ) {
    
    
        // 处理Content-Length头部字段
        text += 15;
        text += strspn( text, " \t" );
        m_content_length = atol(text);
    } else if ( strncasecmp( text, "Host:", 5 ) == 0 ) {
    
    
        // 处理Host头部字段
        text += 5;
        text += strspn( text, " \t" );
        m_host = text;
    } else {
    
    
        printf( "oop! unknow header %s\n", text );
    }
    return NO_REQUEST;
}

// 我们没有真正解析HTTP请求的消息体，只是判断它是否被完整的读入了
http_conn::HTTP_CODE http_conn::parse_content( char* text ) {
    
    
    if ( m_read_idx >= ( m_content_length + m_checked_idx ) )
    {
    
    
        text[ m_content_length ] = '\0';
        return GET_REQUEST;
    }
    return NO_REQUEST;
}

// 主状态机，解析请求
http_conn::HTTP_CODE http_conn::process_read() {
    
    
    LINE_STATUS line_status = LINE_OK;
    HTTP_CODE ret = NO_REQUEST;
    char* text = 0;
    while (((m_check_state == CHECK_STATE_CONTENT) && (line_status == LINE_OK))
                || ((line_status = parse_line()) == LINE_OK)) {
    
    
        // 获取一行数据
        text = get_line();
        m_start_line = m_checked_idx;
        printf( "got 1 http line: %s\n", text );

        switch ( m_check_state ) {
    
    
            case CHECK_STATE_REQUESTLINE: {
    
    
                ret = parse_request_line( text );
                if ( ret == BAD_REQUEST ) {
    
    
                    return BAD_REQUEST;
                }
                break;
            }
            case CHECK_STATE_HEADER: {
    
    
                ret = parse_headers( text );
                if ( ret == BAD_REQUEST ) {
    
    
                    return BAD_REQUEST;
                } else if ( ret == GET_REQUEST ) {
    
    
                    return do_request();
                }
                break;
            }
            case CHECK_STATE_CONTENT: {
    
    
                ret = parse_content( text );
                if ( ret == GET_REQUEST ) {
    
    
                    return do_request();
                }
                line_status = LINE_OPEN;
                break;
            }
            default: {
    
    
                return INTERNAL_ERROR;
            }
        }
    }
    return NO_REQUEST;
}

// 当得到一个完整、正确的HTTP请求时，我们就分析目标文件的属性，
// 如果目标文件存在、对所有用户可读，且不是目录，则使用mmap将其
// 映射到内存地址m_file_address处，并告诉调用者获取文件成功
http_conn::HTTP_CODE http_conn::do_request()
{
    
    
    // "/home/nowcoder/webserver/resources"
    strcpy( m_real_file, doc_root );
    int len = strlen( doc_root );
    strncpy( m_real_file + len, m_url, FILENAME_LEN - len - 1 );
    // 获取m_real_file文件的相关的状态信息，-1失败，0成功
    if ( stat( m_real_file, &m_file_stat ) < 0 ) {
    
    
        return NO_RESOURCE;
    }

    // 判断访问权限
    if ( ! ( m_file_stat.st_mode & S_IROTH ) ) {
    
    
        return FORBIDDEN_REQUEST;
    }

    // 判断是否是目录
    if ( S_ISDIR( m_file_stat.st_mode ) ) {
    
    
        return BAD_REQUEST;
    }

    // 以只读方式打开文件
    int fd = open( m_real_file, O_RDONLY );
    // 创建内存映射
    m_file_address = ( char* )mmap( 0, m_file_stat.st_size, PROT_READ, MAP_PRIVATE, fd, 0 );
    close( fd );
    return FILE_REQUEST;
}

// 对内存映射区执行munmap操作
void http_conn::unmap() {
    
    
    if( m_file_address )
    {
    
    
        munmap( m_file_address, m_file_stat.st_size );
        m_file_address = 0;
    }
}

// 写HTTP响应
bool http_conn::write()
{
    
    
    int temp = 0;
    int bytes_have_send = 0;    // 已经发送的字节
    int bytes_to_send = m_write_idx;// 将要发送的字节 （m_write_idx）写缓冲区中待发送的字节数
    
    if ( bytes_to_send == 0 ) {
    
    
        // 将要发送的字节为0，这一次响应结束。
        modfd( m_epollfd, m_sockfd, EPOLLIN ); 
        init();
        return true;
    }

    while(1) {
    
    
        // 分散写
        temp = writev(m_sockfd, m_iv, m_iv_count);
        if ( temp <= -1 ) {
    
    
            // 如果TCP写缓冲没有空间，则等待下一轮EPOLLOUT事件，虽然在此期间，
            // 服务器无法立即接收到同一客户的下一个请求，但可以保证连接的完整性。
            if( errno == EAGAIN ) {
    
    
                modfd( m_epollfd, m_sockfd, EPOLLOUT );
                return true;
            }
            unmap();
            return false;
        }
        bytes_to_send -= temp;
        bytes_have_send += temp;
        if ( bytes_to_send <= bytes_have_send ) {
    
    
            // 发送HTTP响应成功，根据HTTP请求中的Connection字段决定是否立即关闭连接
            unmap();
            if(m_linger) {
    
    
                init();
                modfd( m_epollfd, m_sockfd, EPOLLIN );
                return true;
            } else {
    
    
                modfd( m_epollfd, m_sockfd, EPOLLIN );
                return false;
            } 
        }
    }
}

// 往写缓冲中写入待发送的数据
bool http_conn::add_response( const char* format, ... ) {
    
    
    if( m_write_idx >= WRITE_BUFFER_SIZE ) {
    
    
        return false;
    }
    va_list arg_list;
    va_start( arg_list, format );
    int len = vsnprintf( m_write_buf + m_write_idx, WRITE_BUFFER_SIZE - 1 - m_write_idx, format, arg_list );
    if( len >= ( WRITE_BUFFER_SIZE - 1 - m_write_idx ) ) {
    
    
        return false;
    }
    m_write_idx += len;
    va_end( arg_list );
    return true;
}

bool http_conn::add_status_line( int status, const char* title ) {
    
    
    return add_response( "%s %d %s\r\n", "HTTP/1.1", status, title );
}

bool http_conn::add_headers(int content_len) {
    
    
    add_content_length(content_len);
    add_content_type();
    add_linger();
    add_blank_line();
}

bool http_conn::add_content_length(int content_len) {
    
    
    return add_response( "Content-Length: %d\r\n", content_len );
}

bool http_conn::add_linger()
{
    
    
    return add_response( "Connection: %s\r\n", ( m_linger == true ) ? "keep-alive" : "close" );
}

bool http_conn::add_blank_line()
{
    
    
    return add_response( "%s", "\r\n" );
}

bool http_conn::add_content( const char* content )
{
    
    
    return add_response( "%s", content );
}

bool http_conn::add_content_type() {
    
    
    return add_response("Content-Type:%s\r\n", "text/html");
}

// 根据服务器处理HTTP请求的结果，决定返回给客户端的内容
bool http_conn::process_write(HTTP_CODE ret) {
    
    
    switch (ret)
    {
    
    
        case INTERNAL_ERROR:
            add_status_line( 500, error_500_title );
            add_headers( strlen( error_500_form ) );
            if ( ! add_content( error_500_form ) ) {
    
    
                return false;
            }
            break;
        case BAD_REQUEST:
            add_status_line( 400, error_400_title );
            add_headers( strlen( error_400_form ) );
            if ( ! add_content( error_400_form ) ) {
    
    
                return false;
            }
            break;
        case NO_RESOURCE:
            add_status_line( 404, error_404_title );
            add_headers( strlen( error_404_form ) );
            if ( ! add_content( error_404_form ) ) {
    
    
                return false;
            }
            break;
        case FORBIDDEN_REQUEST:
            add_status_line( 403, error_403_title );
            add_headers(strlen( error_403_form));
            if ( ! add_content( error_403_form ) ) {
    
    
                return false;
            }
            break;
        case FILE_REQUEST:
            add_status_line(200, ok_200_title );
            add_headers(m_file_stat.st_size);
            m_iv[ 0 ].iov_base = m_write_buf;
            m_iv[ 0 ].iov_len = m_write_idx;
            m_iv[ 1 ].iov_base = m_file_address;
            m_iv[ 1 ].iov_len = m_file_stat.st_size;
            m_iv_count = 2;
            return true;
        default:
            return false;
    }

    m_iv[ 0 ].iov_base = m_write_buf;
    m_iv[ 0 ].iov_len = m_write_idx;
    m_iv_count = 1;
    return true;
}

// 由线程池中的工作线程调用，这是处理HTTP请求的入口函数
void http_conn::process() {
    
    
    // 解析HTTP请求
    HTTP_CODE read_ret = process_read();
    if ( read_ret == NO_REQUEST ) {
    
    
        modfd( m_epollfd, m_sockfd, EPOLLIN );
        return;
    }
    
    // 生成响应
    bool write_ret = process_write( read_ret );
    if ( !write_ret ) {
    
    
        close_conn();
    }
    modfd( m_epollfd, m_sockfd, EPOLLOUT);
}

• accomplish

pressure test

• Webbench is a well-known and excellent web performance stress testing tool on Linux. It is developed by Lionbridge Corporation.

The test is on the same hardware, the performance of different services, and the running status of the same service on different hardware. Displays two aspects of the server: the number of response requests per second and the amount of data transferred per second.

• Basic principle: Webbench first forks out multiple child processes, and each child process performs web access testing in a loop. The child process tells the parent process the access results through the pipe, and the parent process makes the final statistical results.
• 10,000 clients, running successfully in 5 seconds