Article directory

1. Network basics TCP/IP
2. Protocols closely related to HTTP
3. URL and encoding issues
4. Get to know the HTTP protocol first
5. HTTP methods
- 5.1 GET and POST
- - test
  - form
6. HTTP status code
- 6.1 Redirection
7. HTTP Header
- 7.1 Introduction
- - Format
  - type
- 7.2 Common Headers
- - Content-XX
  - Host
  - User-Agent
  - Refer
  - Keep-Alive
8. Session management
References
- 8.4 Testing
References

1. Network basics TCP/IP

Friendship Links: Getting Started with Network Basics

Commonly used networks (including the Internet) operate on the basis of the TCP/IP protocol suite. And HTTP is a subset of it.

That is, HTTP usually runs on top of TCP.

Protocol (Protocol) is a rule that implements the agreement. For both parties in network communication, if they want to ensure the consistency of communication methods, they must be based on the same method. Whether it is network communication or communication between hardware and operating systems, a rule constraining the communication method known to all communication participants is required.

TCP/IP is the general term for various protocol families related to the Internet

There are all sorts of things in the protocol. From the specifications of the cable to the selection method of the IP address, the method of finding remote users, the order in which the two parties establish communication, and the steps that need to be processed on the Web page display, etc.

A collection of protocols related to the Internet like this is collectively referred to as TCP/IP.

It is also said that TCP/IP refers to the two protocols of TCP and IP. There is another saying that TCP/IP is a general term for the protocol family used in the communication process of the IP protocol. – "Illustrating HTTP"

2. Protocols closely related to HTTP

2.1 IP protocol responsible for transmission

The IP protocol is different from the IP address, it is the name of a protocol (Internet Protocol). The role of the IP protocol is to transmit data (packets) to the peer host. This process requires the cooperation of the MAC address and IP address burned on the physical device. The IP address indicates the address assigned to the node, and the MAC address refers to the fixed address to which the network card belongs. IP addresses can be paired with MAC addresses. The IP address can be changed, but the MAC address basically does not change.

routing

It is very rare for two parties in the network to communicate through the same local area network (LAN), and it is usually connected to each other after being relayed by multiple computers and network devices.

In a computer network, when a data packet is sent from a source device to a target device, it may need to pass through multiple routers for forwarding. A router is a network device that forwards data packets from one network to another based on the IP address of the destination device. A router usually maintains a routing table to determine which interface a data packet should be forwarded from.

When a router receives a packet, it looks up the routing table based on the destination IP address, determines the next-hop router, and forwards the packet to the next-hop router. This process may travel through multiple routers in the network until the packet reaches the destination device. The communication between IP depends on MAC address. In the process of data transfer, each transfer device is a ring, and they cannot know the overall transmission situation. For the current transfer device, its goal is to use the MAC address of the next transfer device as the transfer target. This needs to pass the ARP protocol (Address Resolution Protocol). ARP is a protocol used to resolve addresses. According to the IP address of the communication party, the corresponding MAC address can be found out.

Routing:

It is worth noting that each transit device cannot fully grasp the transmission situation on the network, and they can only obtain a very rough transmission route. This route usually refers to the basic forwarding path from the source device to the destination device, including the origin and destination. The router only knows the destination address of the data packet and the address of the next-hop router, but for the details of the entire transmission path, such as all devices in the transit path, network topology and other information, the router cannot obtain complete information. This is the routing mechanism (routing).

Routing refers to the process by which a router forwards a data packet from a source device to a destination device by selecting the best routing path in a computer network. It's a bit like the delivery process of a courier company. People who want to send express delivery can know whether the express company is willing to accept and deliver the goods as long as they send their goods to the distribution center. The distribution center of the express company checks the delivery address of the goods and specifies which area the next stop should be sent to distribution center. Then, the distribution center in that area will judge whether it can be delivered to the other party's home.

During the route selection process, the router will select an optimal routing path according to its own routing table, and forward the data packet to the next-hop router until the data packet reaches the target device. The router selects the best routing path based on multiple factors, such as network topology, network bandwidth, and network congestion between the router and the target device.

In order to select the best routing path, a router usually maintains a routing table, which contains routing path information to different destination networks. The routing path information in the routing table may be manually configured by the network administrator, or it may be learned automatically through routing protocols.

Routing selection is a very important process in computer network, which directly affects the performance and reliability of the network. A good routing algorithm can reduce network delay, improve bandwidth utilization, reduce network congestion and other issues, thereby improving the performance and reliability of the entire network.

2.2 TCP protocol to ensure reliability

The TCP protocol provides reliable byte stream services at the transport layer.

The so-called byte stream service (Byte Stream Service) refers to, in order to facilitate transmission, a large block of data is divided into data packets in units of segments (segment) for management. And reliable transmission service refers to the ability to transmit data to the other party accurately and reliably. In a word, the TCP protocol divides the data in order to transmit large data more easily, and the TCP protocol can confirm whether the data is finally delivered to the other party.

Reliable and unreliable are neutral terms that describe the nature of the service.

Compared with the UDP protocol, it is an unreliable protocol. Because of this, its transmission speed is often very fast, because ensuring reliability requires a price.

three handshake

Both parties need to establish a connection before performing TCP communication. The process of establishing a connection is called a three-way handshake.

Briefly introduce the handshake process, and more in-depth content will be introduced in the special topic of TCP protocol.

The following is a brief introduction to the TCP three-way handshake:

The first handshake: The client sends a SYN (synchronization) message to the server, indicating that the client intends to initiate a connection to the server, and randomly selects an initial sequence number (ISN).
The second handshake: After receiving the SYN message from the client, the server sends a SYN/ACK message to the client to confirm the client's request, and also indicates that the server randomly selects an initial sequence number. At the same time, the server also sends an acknowledgment number (ACK) to the client, which is the client's ISN+1.
The third handshake: After receiving the server's SYN/ACK message, the client sends an ACK message to the server, indicating that it has received the server's confirmation, and also sends an acknowledgment number (ACK) to the server. It is the server's ISN+1.

At this point, the TCP connection is established, and the client and server can start data transmission.

The purpose of the three-way handshake is to ensure that both the client and the server are ready for data transmission, and also to ensure that the initial serial number, confirmation number and other information of both parties are correct. Through the three-way handshake, TCP can guarantee the reliability and correctness of the connection, thereby avoiding errors and losses during data transmission.

If there is an inexplicable interruption at a certain stage during the handshake process, the TCP protocol will send the same data packets in the same order again.

2.3 DNS service responsible for domain name resolution

The DNS (Domain Name System) service is a protocol at the application layer like the HTTP protocol. It provides resolution services between domain names and IP addresses.

The IP address is the identification of the computer in the network, but it does not conform to the usage habits of human beings, and it is not convenient to remember. Therefore, the DNS service is equivalent to a table, which stores the mapping relationship between alphanumeric domain names and IP addresses. When a user enters www.baidu.coma domain name such as this, the browser first obtains the IP address corresponding to the URL through the DNS service, and then accesses the server through the IP address.

The so-called DNS hijacking is to illegally modify the mapping relationship between domain names and IP addresses. Before the response from the server is actually obtained, the user is unaware.

2.4 Relationship between various protocols and HTTP protocol

Through the above brief introduction to the protocols closely related to the HTTP protocol, the HTTP protocol and the relationship between them are roughly as follows:

3. URL and encoding issues

3.1 Introduction

URL/URL came after DNS, when the Web was still in its infancy, and Web pages were usually just simple HTML text files. As Web pages become more and more complex and rich, a more flexible and general identification method is needed to identify and access various resources.

Both URL (Uniform Resource Locator, Uniform Resource Locator) and URI (Uniform Resource Identifier, Uniform Resource Identifier) are identifiers used to identify resource locations and access methods on the Internet. There is a close relationship between them, but they are There is a difference. URL is a specific implementation of URI, which is used to identifyWeb pageA standard format for where and how to access resources such as . A URI is a more general concept used to identify any type of resource location and access method, including URLs.

Uniform : A unified format can facilitate the processing of many different types of resources, without having to identify the access method specified by the resource according to the context. Also, adding new protocol schemes like http: or ftp: is easier.
Resource : The definition of a resource is "anything that can be identified". Anything but a document file, an image, or a service (such as the weather forecast for the day) that can be distinguished from the other types can be used as a resource. In addition, resources can be not only single, but also a collection of many.
Identifier : Represents an identifiable object. Also known as an identifier.

Format

URLs are usually more familiar to us, so let's use that as an example first.

A standard URL usually consists of the following parts:

URL = scheme “:” “//” authority path [ “?” query string] [ “#” fragment ]
// authority 表示资源所在的网络服务器的名称或 IP 地址，以及可选的端口
// path 表示资源在网络服务器上的位置

Protocol (Scheme): Specifies the type of protocol used to access resources, such as http, https, ftp, etc.
Host name (Host): Specify the server host name or IP address where the resource is located.
Port number (Port) [optional]: Specify the port number that the server monitors, usually 80 (HTTP protocol) or 443 (HTTPS protocol).
Path: Specifies the path of the resource on the server, which can be a relative path or an absolute path. This is similar to the file directory structure of a UNIX system.
Query String (Query String) [Optional]: Contains additional parameters in the URL in the form of key=value.
Fragment identifier (Fragment) [optional]: Specifies a specific location to access resources, such as a certain part of the page, such as a location in a document; a point in time in a video or audio document.

Here is an example of a standard URL:

https://www.example.com:443/index.html?id=123#section1

In this URL, the protocol is HTTPS, the hostname is www.example.com, the port number is 443, the path is /index.html, the query string is id=123, and the section identifier is section1.

HyperText Markup Language (HTML) is a markup language used to structure Web pages and their content . The content of a web page can be: a set of paragraphs, a list of key information, and can also contain pictures and data tables.

Notice:

When using protocol scheme names such as http: or https: to obtain access resources, the protocol type must be specified. Letters are case insensitive and appended with a colon (:) at the end.
The path part of the URL follows the Unix-style path notation, probably because most web servers were running on Unix or Unix-like systems at the time (most web servers are now running on Linux systems), and also because of the Unix-style The path notation of is more concise and general.

The above picture is an explanation of each part in a URL. It should be noted that:

user:passThe field saves the user name and password, indicating the user login authentication information. Most of the time they will be omitted and passed to the server through other schemes, because doing so is very insecure, which is equivalent to transmitting the username and password in clear text on the network.
http://Indicates the name of the protocol. In addition, the common HTTPS protocol is a security-oriented HTTP channel, which ensures the security of the transmission process through transmission encryption and identity authentication on the basis of HTTP.
The port number is often bound to the protocol, so when the client sends a request with a certain protocol, the protocol itself specifies the port number, so the port number of the server in the URL is generally omitted.
The latter two are parameters, which are optional. The URL can use an absolute path or a relative path to specify the location of the resource. An absolute path refers to a complete resource path, including all information such as host name, path, and file name, and can independently specify the location of a resource. A relative path is a path relative to the current page or the base URL, which can specify the location of the resource more concisely.

The client obtains the request and actually accesses the resources in the server through the URL, and the server responds to the request by transferring these resources to the client through the network. For example, a user accesses a webpage by sending a request to the server through the browser, and then obtains the html webpage file returned by the server, and the browser renders the html file, so that the user can obtain the content in the webpage. For example:

In addition to transmitting web page resources, the server can also return resources such as video, audio, pictures, etc., which is the origin of the name HTTP (Hyper Text Transfer Protocol, Hypertext Transfer Protocol).

3.2 Encoding issues

Characters such as ?, /, :etc. may also be used by users because they have fixed meanings, but for computers, they are part of the protocol and are identifiers for different modules. conflict, we need to escape these characters entered by the user.

URLencode and URLdecode are two PHP functions that encode and decode strings in URLs. The URLencode function willReplace special characters in a string (such as spaces, quotes, question marks, etc.) with a percent sign (%) followed by a two-digit hexadecimal number, so that these characters do not confuse or cause errors with the rest of the URL. For example:

urlencode("Hello World!") will return "Hello+World%21"

The urldecode function will restore the string encoded by the URLencode function to the original string, that is, replace the sequence of percent signs followed by two hexadecimal numbers with the corresponding characters. For example:

urldecode("Hello+World%21") will return "Hello World!"

The URLencode and urldecode functions are usually used to pass parameters or data in the URL to ensure the validity and security of the URL. In addition, PHP also provides rawurlencode and rawurldecode functions, the difference between them and URLencode and urldecode functions is that they follow the RFC 3986 standard, that is, spaces are encoded as %20 instead of + signs. For example:

rawurlencode("Hello World!") will return "Hello%20World%21"

rawurldecode("Hello%20World%21") will return "Hello World!"

Online encoding/decoding tools: https://www.urlencoder.org/

4. Get to know the HTTP protocol first

As an application layer protocol, HTTP determines the communication activities when providing application services to users. As mentioned earlier, HTTP (application layer) usually runs on top of TCP (transport layer). In network communication, the implementation details of other hierarchical structures are transparent to any layer, and the application layer is no exception. In the eyes of each layer protocol, they will think that the data is directly transmitted from the same layer protocol of the other party. like this:

For protocols at the same layer, the overall combination of payload and header is the same regardless of whether the packet is delivered or received. What this picture mainly wants to express is the red arrow, which does not really exist, but when each layer protocol of the right host receives data, it is equivalent to receiving data directly from the same layer protocol of the peer host.

The three-layer protocols other than the application layer implement specific communication details, which are completed by the operating system and will not be discussed here. They will be introduced in detail in subsequent topics.

4.1 CS mode

The CS mode, that is, an implementation of the client-server (client-server) mode, is an application layer service based on requests and responses.

The client sends a request message to the server, the server processes it according to the content of the request, and returns a response message to the client. Such services usually use an application-layer protocol to standardize the format, content, and semantics of requests and responses, such as HTTP, FTP, SMTP, and so on.

The advantages of application layer services based on requests and responses are simplicity, flexibility, and scalability, and can support various types of application requirements, such as web browsing, file transfer, and email. The disadvantage of application layer services based on requests and responses is that there may be problems such as delay, repetition, and loss, and it is necessary to provide reliability and security guarantees at the transport layer or other layers.

When using the HTTP protocol, one end must act as the client and the other end as the server . But in fact there is no absolute client and server, which is determined according to the specific needs of users. But from the point of view of only one communication route, the roles of the server and the client are determined, and the HTTP protocol can clearly distinguish which end is the client and which end is the server.

4.2 Communication through responses and requests

Since HTTP is a protocol applied in the cs mode, and it can clearly distinguish the client and the server, then the data communication is transmitted to the server through the client's request, and the server parses the request data after obtaining it, and returns the response data , thus completing the communication.

This is consistent with what was discussed above taking the client and server as examples.

In fact, what is exchanged is the communication message between the two parties, that is, the HTTP request and response information.

It is worth noting that the HTTP protocol stipulates that the request is sent from the client, and finally the server responds to the request and returns. In other words, the communication must be established from the client first, and the server will not send a response until it receives the request.

Since HTTP is built on top of TCP, the three-way handshake to establish a connection has been completed before the request message is sent.

4.3 Request and Response Formats

The HTTP request and response protocol format is a message format for quickly constructing HTTP responses and requests. Purely from the message point of view, the HTTP protocol can beline-based text protocol。

request format

The HTTP request protocol format mainly consists of the following parts:

请求行：[请求方法] [URL] [协议版本]
请求头部/请求报头/请求首部字段
空行
请求正文

Request line : consists of three parts: request method , URL and HTTP protocol version , separated by spaces. For example: GET /index.html HTTP/1.1, which means requesting access to the /index.htm page resource on an HTTP server.
- GETRepresents the type of request to access the server, called a method (method).
- /index.htmIndicates the resource object to request access to, also known as the request URI (request-URI).
- HTTP/1.1, that is, the version number of HTTP, which is used to prompt the client to use the HTTP protocol function.
Request header : consists of several header fields , each header field consists of akey value pairComposition, with a colon in the middle (:) separated. Each header field occupies one line and ends with a carriage return and line feed. For example: Host: www.example.com.
Empty line : used to indicate the end of the request header.
Request body : used to pass some additional data, usually user-related information or data, such as the content submitted by the form (sometimes when we refresh the form after filling it, relevant warnings will be prompted). The request body is allowed to be an empty string, and its length and type are specified by the header fields Content-Lengthand Content-Type. Not all requests have a request body, such as the GET method.

The three parts other than the request body are built into the HTTP protocol. If the user does not fill in the request body when requesting, the request body is an empty string.

For example, the following is a complete HTTP request message:

POST /Login/index HTTP/1.1 // 请求行
Host: www.everyonepiano.cn // 请求头部
Connection: keep-alive
Content-Length: 50
Cache-Control: max-age=0
Origin: http://www.everyonepiano.cn
Upgrade-Insecure-Requests: 1
Content-Type: application/x-www-form-urlencoded
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/63.0.3239.132 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Referer: http://www.everyonepiano.cn/Login/index.html
Accept-Encoding: gzip, deflate
Accept-Language: zh-CN,zh;q=0.9

// 空行
username=admin&password=123456 // 请求正文

From a logical point of view, the content of the protocol is displayed in lines, but in actual network communication, "line" will be replaced \r\nas a separator, and the entire message is a string, which is transmitted in the network through a byte stream.

response format

The response message of the HTTP protocol refers to the message returned by the server to the client. An HTTP response message consists of the following parts:

状态行：[协议版本] [状态码] [状态码描述]
响应头部/响应报头
空行
响应主题

A status line : used to describe whether the request is successful or failed, and contains three elements: protocol version, status code and status text. For example, HTTP/1.1 200 OKto indicate that the request was successful.
Response header [optional]: Used to provide additional information about the server or the response body, consisting of a case-insensitive string, a colon, and a value. For example, Content-Type: text/htmlindicates that the response body is an HTML document.
An empty line: used to indicate that all metadata about the response has been sent.
Response body [optional]: used to pass the data returned by the server, such asHTML page or image file. Content-LengthThe length and type of the response body are specified by the and fields in the response header Content-Type.

A typical HTTP response message is as follows:

HTTP/1.1 200 OK // 状态行
Date: Fri, 01 Nov 2013 00:00:00 GMT // 响应头部
Server: Apache/2.2.14 (Win32)
Last-Modified: Wed, 22 Jul 2009 19:15:56 GMT
Content-Type: text/html
Content-Length: 88

// 空行
<html> // 响应主体
<body>
<h1>Hello, World!</h1>
</body>
</html>

in:

HTTP/1.1 at the beginning of the start line indicates the version of HTTP that the server corresponds to.
The next 200 OK indicates the status code (status code) and reason phrase (reason-phrase) of the processing result of the request.
The next line shows the datetime when the response was created, as an attribute in the header field.
It is then separated by a blank line, and the content after that is called the entity body of the resource entity.

In this example, the final response body is an html file, which can be rendered as a web page after being obtained by the client as the return information of the server. This part can be customized later in the test.

Why do requests and responses have to exchange protocol version numbers? What's the point?

In order to ensure compatibility, because the new version of the protocol can often provide more services:

The version number of the HTTP protocol can let the client and the server know the protocol features and functions supported by each other, so as to perform appropriate processing and optimization.
The version number of the HTTP protocol allows clients and servers to check the status of requests and responses to determine success or failure and take appropriate actions, such as updating or using local caches.
The version number of the HTTP protocol allows the client and server to send corresponding error codes, such as 505 (HTTP version does not support), when encountering incompatibility or unsupported situations.

4.4 Separate the payload in the message

Payload (payload) refers to a part of the real transmission data. In the HTTP protocol, the real valid data is the final request body. As mentioned just now, logically, the message is divided by "line", but in fact it is \r\ndivided by line.

When the server gets the HTTP request (data packet), it needs to parse the request and take out the really valid data, as long as it is passed \r\nas a delimiter, becauseblank linebefore the text, then the combination \nof the blank line and the line above it is ,\r\n\r\n\nWhen the client reads 2 \n, it means that the header has been read, and the rest is the payload。

Among them, the header field in the request body specifies the length and type of the body, so that the server can take out the payload completely Content-Length.Content-Type

4.5 Introduction to HTTP

As mentioned above, the response body of the HTTP response can be an html file, which is user-defined. Let’s implement it as follows: let the browser initiate a request to the server, and then the server returns the html file so that it can be displayed on the display on display. But that's just the last link. The most important thing is how to make the server return this file to the client.

The HTTP protocol at the application layer runs on top of the TCP protocol at the transport layer, so logic such as establishing connections and communication between hosts needs to be implemented using TCP sockets.

First, encapsulate the network operations in Linux, such as creating sockets, listening, obtaining connections, connecting, etc., into functions, and then put them in classes Sock.

Then, use a class HttpServerto encapsulate the server, which has a Socktype member, so that the encapsulated network interface can be called in the member function.

// HttpServer.hpp
#include <iostream>
#include <functional>
#include <signal.h>
#include <sys/types.h>

#include "Sock.hpp"

class HttpServer
{
    
    
public:
    using func_t = std::function<void(int)>;

private:
    int _listen_sock;
    uint16_t _port;
    Sock _sock;
    func_t _func;

public:
    HttpServer(const uint16_t &port, func_t func, std::string ip = "")
        : _port(port), _func(func)
    {
    
    
        _listen_sock = _sock.Socket();
        _sock.Bind(_listen_sock, _port);
        _sock.Listen(_listen_sock);
    }
    ~HttpServer()
    {
    
    
        if (_listen_sock >= 0)
            close(_listen_sock);
    }
    void Start()
    {
    
    
        signal(SIGCHLD, SIG_IGN);
        while (1)
        {
    
    
            std::string client_ip;
            uint16_t client_port;
            int sockfd = _sock.Accept(_listen_sock, &client_ip, &client_port);
            // std::cout << client_ip << ":" << client_port << std::endl; // for DEBUG
            if (sockfd < 0)
                continue;
            if (fork() == 0) // 子进程
            {
    
    
                close(_listen_sock); // 子进程关闭不需要的监听套接字文件描述符
                _func(sockfd);       // 调用服务函数
                close(sockfd);       // 使用完毕后关闭
                exit(0);
            }
            close(sockfd); // 父进程关闭不需要的socket套接字文件描述符
        }
    }
};

It is worth noting that the Start() function uses a common method to avoid zombie processes, that is, ignores the SIGCHLD signal. In addition, the grandchild process can also be used to call service functions to respond to client requests.

// HttpServer.cc
#include "HttpServer.hpp"
#include <memory>
void Usage(const std::string &proc)
{
    
    
    std::cout << "\nUsage: " << proc << " [PORT]\n" << std::endl;
}

void HttpRequestHandler(int sockfd)
{
    
    
    char buffer[10240];
    ssize_t s = recv(sockfd, buffer, sizeof(buffer) - 1, 0);
    if (s > 0)
    {
    
    
        buffer[s] = '\0';
        std::cout << "-------------------------- http request begin ------------------------" << std::endl;
        std::cout << buffer << std::endl;
        std::cout << "-------------------------- http request end --------------------------" << std::endl;
    }
}

int main(int argc, char *argv[])
{
    
    
    if(argc != 2)
    {
    
    
        Usage(argv[0]);
        exit(1);
    }
    uint16_t port = atoi(argv[1]);
    std::unique_ptr<HttpServer> server_ptr(new HttpServer(port, HttpRequestHandler));
    server_ptr->Start();
    return 0;
}

Among them, the HttpRequestHandler() function is the server-side function. Here, the client’s request is simply printed. This request is stored in a buffer provided by the user. When using the recv() function, it is equivalent to copying the contents of the buffer to In the buffer provided by the server, in order to observe the phenomenon, print them out.

Source code: HttpServer Version 1

test

Usually, when testing, set the port to 8080 or 8081. The browser itself is a client. Type in the public IP and port number of the server, separated by ::

If the server does not display anything like this when you press Enter, then there is a high probability that the cloud server does not have an open port. You can search for methods corresponding to the operator's and system's open ports. If this does not solve the problem, it may be that the system's firewall has not opened the port.

Take Alibaba Cloud's CentOS 7 system as an example (2023/6/8):

If you can't add many, you can add them first, and then modify them in the "Edit" button on the far right.

His meowing question took me a long time.

The starting point is that I debugged and found that the encapsulated Accept() function is blocked in the accept() function, and the only possibility is that there is a problem with the first parameter, that is, the listening socket file descriptor (because the other two are output type parameter), indicating that this function is blocked and has been waiting for a connection. Combined with telnetthe test results of the tool, it is found that the connection can be successful every time using the internal network IP or 127.0.0.1, but the connection cannot be made with the public network IP (the port is 8080 or 8081 specified when the server is started), and it is suspected to be a port problem. , all kinds of searches are solved.

Harvest: Although SSH can be used to connect to the machine through this public network IP, it is not a protocol with HTTP.

Under normal circumstances, enter the IP address and port number, and after pressing Enter, the server should immediately display the following content;

This is the response message described above.

Since the logic of this client is just printing without returning any content to the client, the browser displays an error message:

It is worth noting that in many cases, you do not need to specify the protocol name when typing the URL, because the default protocol used by the browser is HTTP.

4.6 Building the response

As mentioned above, after processing the request, the server will return some information to the client, which can be an HTML file. Although the request from the server cannot be processed at present, a fixed HTML file can be returned as a demonstration.

web root directory

The website root directory (web root directory) refers to the first-level folder where the website is stored in the web server, that is, the first-level directory where website files are uploaded and stored. Visiting the homepage of the website points to this directory. The name and location of the root directory of the website may vary depending on the server environment, and the common ones are wwwroot, www, web, htdocs, public_html, etc. The root directory of the website is the installation directory of the website program system, and it is also the storage location of the website files. The root directory of the website is the first-level folder where the website is stored in the web server, that is, the installation directory of the website program system. Therefore, the root directory of the website plays an important role in the operation and management of the website.

Create a new directory under the directory of this file wwwrootas the web root directory, and then put the html file to be responded to in this directory:

// 相对路径：/wwwroot/index.html
<html>
    <h1>HELLO WORLD</h1>
</html>

Shortcut key of VSCode: !Then press Enter to generate a template.

Since the essence of using an html file is still a file stream, that is, all the fields in the html must be spliced into a string, and the server will finally send it to the client. This requires some file stream operations and string separation operations. In order to test the feasibility of the html file, we directly return a string spliced according to the rules as a test:

void HttpRequestHandler(int sockfd)
{
    
    
    // 1. 读取请求
    char buffer[10240];
    ssize_t s = recv(sockfd, buffer, sizeof(buffer) - 1, 0);
    if (s > 0)
        buffer[s] = '\0';
    // 2. 构建响应
    std::string HttpResponse = "HTTP/1.1 200 OK\r\n";
    HttpResponse += "\r\n";
    HttpResponse += "<html><h1>HELLO WORLD</h1></html>";
    send(sockfd, HttpResponse.c_str(), HttpResponse.size(), 0);
}

The results show that this is possible.

Displaying the IP and PORT of the client is a print statement written for debugging.

It is worth noting that wwwroot is the name of a common network root directory. In this article, if wwwroot is specified as the network root directory (this relative path will be defined), then it is the network root directory. Otherwise, it's just a normal directory.

In other words, the root directory of the network is only a stipulation on the name. In theory, any legal name can be used, as long as it is declared in the configuration file. For the HTTP protocol, it does not specify the name of the network root directory by default, it just specifies how to request and respond to resources.

The name of the network root directory is determined by the server software, and different server software may have different default names. If you do not declare the location of the netroot in the configuration file, the server software uses its own default value. If you want to change the location of the web root directory, you must declare it in the configuration file.

In addition, which file in the web root directory of the server is requested by the HTTP client depends on the configuration and operating system of the server. In general, there are several possible files:

index.html
index.php
default.htm
default.aspx
index.asp

This is also user-specifiable.

string split

Cut a string into multiple substrings according to the given delimiter, and store these substrings in a vector. The function takes three arguments: an input string s, a delimiter sep, and a vector pointer to store the result in out.

The function first initializes a variable startthat represents the starting position of the current search. Then enter a loop, each loop will start sfrom startthe position in the string to find the delimiter sep. If the delimiter is found, the substring preceding it is extracted and stored into the vector, and startthe value of is updated so that it points to the beginning of the next substring. If no delimiter is found, exit the loop.

Finally, if sthere is a remainder in the string, it is stored into the vector as the last substring.

#pragma once

#include <iostream>
#include <vector>

class Util
{
    
    
public:
    // example: text1\r\ntext2\r\ntext3\r\n\n
    static void cutString(std::string s, const std::string &sep, std::vector<std::string> *out)
    {
    
    
        std::size_t start = 0;
        while (start < s.size())
        {
    
    
            auto pos = s.find(sep, start);
            if (pos == std::string::npos)
                break;
            std::string sub = s.substr(start, pos - start);
            out->push_back(sub);
            start += sub.size();
            start += sep.size();
        }
        if (start < s.size())
            out->push_back(s.substr(start));
    }
};

Perfect HTTP request processing function

The perfect logic implements a simple HTTP server, which can handle the client's GET request and return the corresponding file content:

#include <fstream>
#include <vector>
#include "Util.hpp"
void HttpRequestHandler(int sockfd)
{
    
    
    // 1. 读取请求
    char buffer[10240];
    ssize_t s = recv(sockfd, buffer, sizeof(buffer) - 1, 0);
    if (s > 0)
        buffer[s] = '\0';
    // 2.0 准备响应
    std::vector<std::string> vline;
    Util::cutString(buffer, "\n", &vline);

    std::vector<std::string> vblock;
    Util::cutString(vline[0], " ", &vblock);

    std::string file = vblock[1];
    std::string target = ROOT;

    if (file == "/")
        file = "/index.html";
    target += file;
    std::cout << target << std::endl;

    std::string content;
    std::ifstream in(target);
    if (in.is_open())
    {
    
    
        std::string line;
        while (std::getline(in, line))
            content += line;
        in.close();
    }
    // 2. 构建响应
    std::string HttpResponse;
    HttpResponse = "HTTP/1.1 200 OK\r\n";
    HttpResponse += "\r\n";
    HttpResponse += content;
    // 3. 返回给客户端
    send(sockfd, HttpResponse.c_str(), HttpResponse.size(), 0);
}

First, the HandlerHttpRequest() function reads the HTTP request sent by the client from the socket and stores it in a buffer buffer. Then use Util::cutStringthe function to split the request into multiple lines according to the newline characters , and store them in the vline container. Then use it to cut the first line (that is, the request line) into multiple parts according to the space, and store them in the vblock container.

The function then extracts the requested filename and concatenates it into the full file path. If the requested filename is requested /, it is replaced with the default home page filename /index.html. The function then tries to open the file and read its contents.

Finally, the function builds an HTTP response based on the read file content. In this way, the client's request accesses the root directory of the network by default, that is, the files /under it index.html.

However, some details may be hidden from the browser:

But in fact, the client's HTTP request is: http://8.130.106.177:8081/(copy the content of the URL box, and then paste it), and one is automatically added after it /, indicating that the default request is under the root directory of the network. The file we set index.htmlis the content to be displayed on the home page.

Similarly, for example, if you press baidu.comEnter in the URL box, the actual request is https://www.baidu.com/that Baidu will return to its home page on the client.

File stream operation: Since the extracted parameter is a path, it should be the root directory of the web rather than the root directory of the system, so before performing file operations, the path must be improved by string concatenation.

Test Results:

Source code: HttpServer Version 2

5. HTTP methods

The HTTP method is the way to tell the server the intent。

According to the HTTP standard, HTTP requests can use several request methods.

HTTP1.0 defines three request methods: GET, POST and HEAD methods.
HTTP1.1 adds six new request methods: OPTIONS, PUT, PATCH, DELETE, TRACE and CONNECT methods.

serial number	method	describe	Protocol support version
1	GET	Request the specified page information and return the entity body.	1.0/1.1
2	HEAD	Similar to a GET request, except that there is no specific content in the returned response, which is used to obtain the header	1.0/1.1
3	POST	Submit data to a specified resource to process a request (such as submitting a form or uploading a file). Data is included in the request body. POST requests may result in the creation of new resources and/or the modification of existing resources.	1.0/1.1
4	PUT	The data sent from the client to the server replaces the content of the specified document.	1.0/1.1
5	DELETE	Requests the server to delete the specified page.	1.0/1.1
6	CONNECT	Reserved in the HTTP/1.1 protocol for proxy servers that can pipe connections into.	1.1
7	OPTIONS	Allows clients to view server performance.	1.1
8	TRACE	Echoes the requests received by the server, mainly for testing or diagnosis.	1.1
9	PATCH	Complementary to the PUT method for partial updates to known resources.	1.1
10	LINK	Create connections with resources	1.0
11	UNLINK	Disconnect relationship	1.0

Source: RUNOOB: HTTP Request Methods

5.1 GET and POST

These two methods are the most commonly used, and they correspond to two major behaviors of users surfing the Internet:

Get resources and data from the server
Submit data from client to server

They correspond to:

The GET method is used to request access to a resource identified by a URI/URL. The specified resource is parsed by the server and returns the response content.
The POST method is used to transfer the body of the entity.

Although the GET method can also be used to transmit the body of the entity, generally the GET method is not used for transmission, but the POST method is used. Although the function of POST is very similar to GET, the main purpose of POST is not to obtain the main content of the response. They can both upload data to the server, but the length of the URL is limited, so the GET method has limitations; the POST method takes the body as a parameter and can transmit more information.

In addition, the POST method is more private, because the URL will display the private information as a parameter, but this does not mean that it is safe to transmit the private information in the body. If the information is not encrypted, the GET and PORT methods are in the network Transmitting plaintext is not safe.

The difference between GET and POST is:

GET is a method used to request data from a specified resource, which appends a query string (key-value pair) to the URL
Features of GET:
- GET requests can be cached
- GET requests are kept in the browser history
- GET requests can be bookmarked
- GET requests should not be used to process sensitive data
- GET requests have a length limit
- GET requests are only used to request data (no modification)
POST is the method used to send data to the server, usually creating or updating a resource. The data of the POST request is stored in the body of the HTTP request, for example:
```
POST /test/demo_form.php HTTP/1.1
Host: w3schools.com
name1=value1&name2=value2
```
Features of POST:
- POST requests are not cached
- POST requests are not kept in browser history
- POST requests cannot be bookmarked
- POST request has no data length limit
- POST requests can send any type of data, including binary data
- POST requests are more secure than GET requests because the data will not be displayed in the URL

test

Use the postman tool to test the GET and POST methods and observe the results.

GET method: URL is used as a parameter, and two parameters a and b are added to the URL (in order to display specific information, the obtained client request is printed in the HttpRequestHandler() function):

It can be seen that the passed parameters are also added to the request line, which corresponds to the first column "Params" (parameters).

POST method: The body is used as a parameter, which should be set in the fourth column "Body":

In this way, the body of the HTTP request is filled with this string instead of an empty string. Because of this, the Content-Length field appears in the response header of the server, indicating the length of the response body.

The reason for printing "./wwwroot/index.html" is that another statement that prints the resource path requested by the client is not commented out.

form

The HTML form is used to collect user input information, representing an area in the document, this area contains interactive controls, and sends the information collected by the user to the Web server. In the HTTP protocol, usually used with the GET and POST methods.

Simply put, a form is a box to the user that the user can fill in with information that will be turned into part of the HTTP request. Then the form is to be submitted to the server as data, which requires specifying the submission method, and the common methods are GET and POST methods.

The method attribute of the form is used to specify which method to use, for example:

<form method="POST">
  <!-- 表单元素 -->
</form>

If the method attribute is not specified, then the GET method is used by default.
The GET method will append form data to the URL, while the POST method will store the form data in the request body.
The GET method is suitable for requesting data, while the POST method is suitable for sending data.

Let's write a simple form in an html file for users to enter information and submit, which contains some prompt information:

<html>
<body>
    <h1>HELLO WORLD</h1>
    <form name="input" method="get" action="/index.html">
        Username: <input type="text" name="user">
        <br>
        passward: <input type="password" name="pwd">
        <br>
        <input type="submit" value="登录">
    </form>
</body>
</html>

Note that the focus of this article is not the grammar of html, as long as you understand it simply.

When the user visits the server, the form in the figure will appear, and after submission, they will be inserted into the URL as parameters.

If you change the method to POST:

If the method is PORT, the two attributes submitted by the user will not be reflected in the URL, but will be transmitted to the server in the text.

It is worth noting that privacy ≠ security, the PORT method just puts the parameters in the URL in the body, but in fact they are all transmitted in the network in plain text, which is not safe, only after encryption and decryption safe.

6. HTTP status code

The responsibility of the HTTP Status Code (HTTP Status Code) is to describe the returned request result when the client sends a request to the server. With the help of the status code, the user can know whether the server has processed the request normally or an error has occurred.

When a viewer visits a web page, the viewer's browser will send a request to the server where the web page is located. Before the browser receives and displays the webpage, the server where the webpage is located will return an information header (server header) containing the HTTP status code in response to the browser's request.

Categories of status codes:

	category	reason phrase
1XX	Informational (informational status code)	The received request is being processed
2XX	Success (success status code)	The request is processed normally
3XX	Redirection (redirection status code)	Additional action is required to complete the request
4XX	Client Error (client error status code)	The server was unable to process the request
5XX	Server Error (server error status code)	An error occurred while the server was processing the request

Common HTTP status codes:

200: The request was successful
301: The resource (web page, etc.) has been permanently transferred to another URL
404: The requested resource (web page, etc.) does not exist
500: Internal Server Error

For 4xx, the standard is whether the client request is legal or not, that is to say, the request needs to be reasonable for the server, because the service scope of the server is limited. 5xx is an internal server error, which is rarely seen on the client side, because it may cause risks, and this information is generally displayed to programmers for debugging, so even if the status code of the general server is really 5xx, it may be displayed. Appear as 3xx or 4xx to clients, or redirect to other services.

From the network ( RUNOOB: HTTP status code )

What programmers want to see most: 200-OK.

What programmers don't want to see: 500-Internal-Server-Error.

What users don't want to see: 401-Unauthorized, 403-Forbidden, 408-Request-Time-out, 404-not-found.

6.1 Redirection

A 3XX response results in an indication that the browser needs to perform some special processing in order to properly handle the request.

permanent redirect

Permanent redirection (301 Moved Permanently). This status code indicates that the requested resource has been assigned a new URL and that the URL the resource now points to should be used in future. That is, the original resource has been permanently moved to the new location, and the client/browser should no longer attempt to request the original location, but use the new location.

This means that if a certain IP or domain name is permanently redirected, the browser will perform a redirection operation when visiting the website for the first time, and if you visit it later, you will directly access the redirected website.

Why does this have anything to do with browsers? Shouldn't the URL/URI have been updated?

The redirect operation performed by the browser refers to:

When a browser requests a URL, the server returns a response containing a status code and a Location header.
If the status code is 301, which means permanent redirection, then the browser will get the new URL from the Location header and initiate the request again.
The browser will cache this redirection information so that it will directly request the new URL instead of the original URL next time.
If you enter the original URL in the browser, the browser will automatically replace it with the new URL and display it in the address bar. This is what redirection does.

That is to say,The redirected address is saved in the Location header, this information is sent by the server. For example, if the server wanted to http://example.comredirect to http://example.org, it would return a response like this:

HTTP/1.1 301 Moved Permanently
Location: http://example.org

The browser will obtain the new address from the Location header and initiate the request again.

In fact, it is also easy to understand from real-life examples: when a user visits a website for the first time, he does not know whether the website is redirected. Perhaps the original domain name is easier to remember and more representative, and the redirected IP address may be set due to some needs.

The Location header will be cached by the browser by default without any expiration date. That is, it remains in the browser's cache until the user manually clears the cache, or the cache entries are cleared to make room.

The cache behavior is simply the browser's default behavior when no other cache control directives are specified. Users can use some HTTP headers to change this behavior, such as Cache-Control and Expires.

temporary redirection

Temporary redirection (Moved Temporarily 302, 307). These two status codes indicate that the requested resource has been assigned a new URL, and it is hoped that the user (this time) can use the new URL to access.

302(Found)

Similar to the 301 Moved Permanently status code, but the resource represented by the 302 status code is not permanently moved, but only temporary. In other words, URLs for resources that have been moved may change in the future. For example, the user saves the URL as a bookmark, but does not update the bookmark as when the 301 status code appears, but still retains the URL corresponding to the page that returned the 302 status code.

307(Temporary Redirect)

Temporary redirection. This status code has the same meaning as 302 Found. Although the 302 standard prohibits the conversion of POST to GET, everyone does not abide by it in actual use.

307 will follow browser standards and will not change from POST to GET. However, each browser may behave differently when handling responses.

The difference between 307 (Temporary Redirect) and 302 (Found) status codes is:

Both 307 and 302 indicate temporary redirection, that is, the original resource is temporarily located elsewhere, and the client/browser should continue to request the original location.

307 guarantees that the request method and body will not change after redirection, while 302 may cause some old clients/browsers to mistakenly change the request method to GET.

The behavior of 307 is predictable on the web, while the behavior of 302 is unpredictable on non-GET methods.

For the GET method, the behavior of 307 and 302 is the same.

redirect test

Only a temporary redirection test can be performed here, in fact, the Location field is spliced into the response string:

void HttpRequestHandler(int sockfd)
{
    
    
    // ...
    std::string HttpResponse;
    if (content.empty())
    {
    
    
        HttpResponse = "HTTP/1.1 302 Found\r\n";
        HttpResponse += "Location: https://www.bing.com/\r\n";
    }
    // ...
    send(sockfd, HttpResponse.c_str(), HttpResponse.size(), 0);
}

As long as the HTTP protocol recognizes the 302/307 status code, it will look for the Location field, and then jump to the specified domain name:

Source code: HttpServer Version 3

7. HTTP Header

7.1 Introduction

HTTP header fields (HTTP header fields) refer to the message headers in Hypertext Transfer Protocol (HTTP) request and response messages. They define the operational parameters of an HTTP protocol transaction.

Format

The fields of the protocol header are transmitted after the request (request) or response (response) line (the first line of a message). The field of the protocol header is transmitted in the string format of plain text , which is a colon-separated key name and key-value pair, ending with a sequence of carriage return (CR) and line feed (LF) symbols ( \r\n). The end of the protocol header part is marked with a blank field, and as a result, two consecutive CR+LFs are transmitted.

Historically, very long lines used to be transmitted as multiple shorter lines; at the beginning of the next line, a space (SP) or a horizontal tab (HT) was output to indicate that it was a subsequent line. Today, this newline form is deprecated. But as a learner, it is necessary to know.

type

HTTP header fields are divided into the following 4 types according to their actual usage:

Generic headers can provide information about the entire message, such as date, connection status, cache control, etc.
Request headers can provide information about the client or requested resource, such as hostname, user agent, accepted media formats, etc.
Response Headers can provide information about the server or response resource, such as server name, location, cookies, etc.
Indicates that the Header can provide information about the resource body, such as content type, content encoding, content length, etc.
The payload header can provide information unrelated to the payload data, such as transfer encoding, content length, etc.

The following mainly discusses the response Header.

7.2 Common Headers

The response header field is the field used by the server to return the response message to the client, and is used to supplement the additional information of the response, server information, and additional requirements for the client. Common response headers are:

Header	illustrate	type
User-Agent	Declare the version information of the user's operating system and browser.	ask
Refer	Which page the current page is redirected from.	ask
Content-Length	Indicates the content length.	Payload
Content-Type	Indicates what MIME type the following document belongs to.	Payload
Date	The current GMT time. You can set this header with setDateHeader to avoid the hassle of converting date formats.	universal
Host	The client tells the server which port the requested resource is on which host.	ask
Cookie	It is used to store a small amount of information on the client side, and is usually used to implement the session function.	ask
Location	Redirect the client to the specified URI	response
Connection	Hop-by-hop header and connection management. Use the keyword "Keep-Alive" to indicate that the connection should be kept open to receive future information (this was the default in HTTP 1.1, while HTTP 1.0 would by default create a new connection for each request/reply pair).	universal

RUNOOB: HTTP response header information

Content-XX

For example, two response headers are specified below, namely Content-Type and Content-Length, where "Content-Type: text/plain\r\n" means that the body of this response is in plain text format and does not contain any tags Or format, and use \r\nto indicate that the Header field has ended.

void HttpRequestHandler(int sockfd)
{
    
    
    // ...
    else
    {
    
    
        HttpResponse = "HTTP/1.1 200 OK\r\n";
        HttpResponse += "Content-Type: text/plain\r\n";
        HttpResponse += "Content-Length: " + std::to_string(content.size()) + "\r\n";
    }
    // ...
    send(sockfd, HttpResponse.c_str(), HttpResponse.size(), 0);
}

In this way, the browser will not render the HTML, and what is displayed is plain text.

If you want to access the image on the server, you can change the response header to this:Content-Type: image/png\r\n

If you want to transmit pictures, you must ensure that the binary format of the picture cannot be damaged, so you should not split the strings, so you should put the binary sequence of the picture into the content string in a way that is suitable for the picture.

In the process of testing, it is best not to make the picture too large, because the server's network speed may not be too fast.

Host

The Host field in the HTTP Header indicates:

The host and port information of the target URI in the request message enables the source server to distinguish different resources when providing services for multiple hosts.
If there is no Host field or multiple Host fields in the request message, the server will return a status code of 400 Bad Request.
The Host field is one of the request headers that must be sent in the HTTP/1.1 protocol. It can support the function of a virtual host, that is, run multiple websites on the same IP address and port.

What is the meaning of host header? The client itself wants to access the server. IP and PORT are part of the client request, and the server also needs to respond to the IP and port. Is it redundant?

The significance of the existence of the Host header is to support the function of the virtual host, that is, to run multiple websites on the same IP address and port. If there is no Host header, the server cannot determine which website should return the content based on the requested domain name. For example, suppose there are two websites www.example.comand www.example.netthey both use the same IP address and port 80, then when the client requests http://www.example.com, the server needs to know whether the client wants to visit www.example.comor not www.example.net, which requires the client to specify in the request header Send Host: www.example.comsuch information. If there is no such information, the server can only return the default website or error message.

Generally speaking, running multiple websites on the same IP address and port generally provides the following services:

Virtual hosting service, which distinguishes different websites through domain names, allows multiple customers to share the resources of the same server, reducing costs and management complexity.
Website building and hosting services, that is, by providing templates and tools to help customers create and maintain their own websites without professional technical knowledge.
Cloud computing and cloud storage services, that is, by providing scalable computing and storage resources to meet the different needs of customers, improve performance and security.

User-Agent

Version information of the operating system and browser corresponding to the client. For example, I use my mobile phone to view the picture just now:

The content of the request received by the server includes the device and software information of the client.

Refer

The HTTP Referer Header is a request type Header, which is used to identify the address of the previous web page requested, that is, from which web page the user links to the currently requested web page or resource. This Header allows servers and websites to identify the source of traffic for purposes such as analysis, logging, and caching optimization. However, this header also increases the risk of user privacy and security, because it may leak the user's browsing history or sensitive information.

Here is an Referer: URLexample

Referer: https://developer.mozilla.org/en-US/docs/Web/JavaScript

Ruan Yifeng: HTTP Referer Tutorial

Keep-Alive

When using TCP Socket to implement the client and server, we know that TCP is connection-oriented. Before the two parties communicate, the server and client must establish a connection. However, in many cases there is only one server. The common implementation of HTTP/1.0 is that after the connection is established, the client sends a request to the server, and the server processes the request and returns a response.

The problem is that if the connection is re-established every time the client interacts with the server, it will be a disaster for the server and a waste of resources. HTTP/1.1 supports long connections,That is, a client can continuously send multiple requests to the server at one time, these requests will be sent at the same time, this mode is called HTTP pipelining. However, because there is no strong specification to ensure its security, HTTP pipelining is not widely used, but is replaced by the multiplexing mechanism in HTTP/2.

If the value corresponding to the Connect field in the HTTP request or response header is Keep-Alive, it means that long connections are supported.

The Keep-Alive in the HTTP Connect Header is a general type of Header, which is used to indicate the status of the connection, and to set the timeout period and the maximum number of requests. It can also be used to allow a TCP connection to be kept open for multiple HTTP request/response multiplexing (by default HTTP connections are closed after each request).

Here is an Keep-Alive: parametersexample:

Keep-Alive: timeout=5, max=1000

timeout: An integer representing the minimum time in seconds that an idle connection will remain open. If the keep-alive TCP message is not set at the transport layer, the timeout setting greater than the TCP level will be ignored.
max: An integer indicating the maximum number of requests that can be sent on this connection before the connection is closed. On non-pipeline connections, other than 0, this value is ignored, since a new request needs to be sent in the following response. HTTP pipe connections can use it to limit the use of pipes.

8. Session management

8.1 HTTP is a protocol that does not store state

HTTP is a stateless protocol that does not save state. The HTTP protocol itself does not preserve the state of the communication between the request and the response. That is to say, at the HTTP level, the protocol does not persist the sent requests or responses.

Simply put, there is no relationship between each HTTP request and response, but this is not the case in our actual experience. For example, you use (request) the login service (response) of a certain website for the first time, even if After restarting the browser or machine, you can stay logged in for a long time.

Using the HTTP protocol, whenever a new request is sent, a corresponding new response will be generated. The protocol itself does not retain information about all previous request or response messages. This is to process a large number of transactions faster and ensure the scalability of the protocol, and the HTTP protocol is deliberately designed to be so simple.

However, with the continuous development of the Web, business processing has become more difficult due to statelessness. For example, if a user logs in to a shopping website, he needs to be able to remain logged in even after he jumps to other pages of the site. For this example, in order to be able to know who sent the request, the website needs to save the user's state.

Although HTTP/1.1 is a stateless protocol, in order to achieve the desired state-keeping function, Cookie technology is introduced. With the Cookie and then using the HTTP protocol to communicate, the state can be managed.

8.2 State management using cookies

HTTP is a stateless protocol, it does not manage the state of previous requests and responses. In other words, this request processing cannot be performed according to the previous state. Assuming that the web page that requires login authentication cannot manage the state itself (does not record the logged-in state), then each time you jump to a new page, you need to log in again, or add parameters to each request message to manage the login state.

It is undeniable that stateless protocols certainly have their advantages. Since there is no need to save the state, it can naturally reduce the CPU and memory resource consumption of the server. If the server manages all client states, it will become a burden . On the other hand, it is precisely because the HTTP protocol itself is very simple that it is applied in various scenarios.

While retaining the feature of the stateless protocol, it is necessary to solve similar contradictory problems, so the Cookie technology is introduced. Cookie technology controls the state of the client by writing cookie information in request and response messages.

The cookie will notify the client to save the cookie according to a header field information called Set-Cookie in the response message sent from the server. When the client sends a request to the server next time, the client will automatically add the Cookie value in the request message and send it out.

After the server finds the cookie sent by the client, it will check which client sent the connection request, then compare the records on the server, and finally get the previous status information.

Set-Cookie Header

When the server is ready to start managing the state of the client, it will notify various information in advance.

The table below lists the field values of Set-Cookie.

Attributes	illustrate
NAME=VALUE	The name given to the cookie and its value (required)
expires=DATE	Expiry date of the cookie (until the browser is closed by default if not specified explicitly)
path=PATH	Use the file directory on the server as the applicable object of the cookie (if not specified, it will default to the file directory where the document is located)
domain=domain name	The domain name to which the cookie applies (if not specified, it defaults to the domain name of the server that created the cookie)
Secure	Cookies are only sent for HTTPS secure communication
HttpOnly	Restrict so that cookies cannot be accessed by JavaScript scripts

Cookie flag status

Request without cookie information :

Requests after the 2nd time (with cookie information status) :

The above picture shows the scene of cookie interaction, we can understand that,A cookie is actually a data packet that saves the user's private information (such as ID and PASSWORD), and it is transmitted along with the direction of communication。

The contents of the corresponding HTTP request message and response message are as follows.

Request message (status without cookie information)

GET /reader/ HTTP/1.1
Host: hackr.jp
*首部字段内没有Cookie的相关信息

Response message (the server generates cookie information)

HTTP/1.1 200 OK
Date: Thu, 12 Jul 2012 07:12:20 GMT
Server: Apache
＜Set-Cookie: sid=1342077140226724; path=/; expires=Wed,
10-Oct-12 07:12:20 GMT＞
Content-Type: text/plain; charset=UTF-8

Request message (automatically send saved cookie information)

GET /image/ HTTP/1.1
Host: hackr.jp
Cookie: sid=1342077140226724

cookie数据可以由服务端生成，也可以由客户端生成。一般来说，服务端生成的cookie数据是用于保存用户的登录状态、购物车信息、偏好设置等，而客户端生成的cookie数据是用于保存用户在浏览器中输入的表单信息、浏览历史等。

服务端生成的cookie数据是通过在响应头中发送Set-Cookie字段来传递给客户端的，客户端收到后会将cookie数据存储在本地，并在后续的请求头中发送Cookie字段来回传给服务。
[暂不考虑]客户端生成的cookie数据是通过JavaScript中的document.cookie属性来创建和修改的，这个属性可以读取和写入当前页面的cookie数据。

例如在Chrome浏览器中，可以管理Cookie数据，只要删除了保存着用户登录信息的cookie文件，就意味着这个用户的登录状态被抹掉，需要重新登录。

安全问题

由于Cookie数据保存着用户的私密信息，如果机器被植入木马或其他安全问题而造成Cookie文件被盗取，那么对方就能使用cookie文件以用户的身份登录网站。

解决办法是改变使用cookie的方式，即使用Session ID标定Cookie文件属于哪个会话。

8.2 会话管理

Session

会话（Session）指的是客户端和服务器之间的一系列交互，从客户端第一次请求服务器，到客户端关闭浏览器或者会话超时结束。

会话可以用来保存客户端的状态信息，比如登录状态，购物车内容等。会话是一种服务器端的机制，它使用Session ID来关联客户端的请求和会话对象。

“会话超时结束”是指当客户端在一定时间内没有向服务器发送任何请求时，服务器会认为该会话已经结束，并销毁对应的会话对象。会话超时的时间可以由服务器端设置，一般默认为20分钟。如果用户设置了长连接，也就是Keep-Alive，那么只要客户端和服务器之间保持TCP连接，就不会触发会话超时。但是，如果客户端清除了Cookie，或者服务器端主动销毁了会话，那么会话也会结束，无论是否设置了Keep-Alive。

Session ID

Session ID是一种用于标识客户端和服务器之间会话的唯一标识符。

“标识性”主要取决于Cookie的domain和path属性。Cookie的domain属性指定了Cookie所属的域名，只有在该域名下的请求才会携带该Cookie。Cookie的path属性指定了Cookie所属的路径，只有在该路径下的请求才会携带该Cookie。通过这两个属性，可以限制Session ID只在特定的域名和路径下有效，从而保证了Cookie文件的归属性。

注意：

Session ID 通常是一个随机生成的字符串，存储在 Cookie 中，或者附加在 URL 后面。服务器可以根据 Session ID 检索或创建与客户端相关的会话信息，比如用户的登录状态，购物车内容等。Session ID 可以保持客户端和服务器之间的状态，因为 HTTP 协议本身是无状态的。
Session ID本身并不包含客户端的信息，它只是一个随机生成的字符串，用来标识服务器端的会话。但是，通过 Session ID，服务器可以从会话中获取客户端的相关信息，比如用户名，密码等。所以，Session ID 可以间接地标识客户端的身份。
一个 Session ID 只能标识一个会话，但是一个会话可以包含多次请求和响应。当客户端关闭浏览器或者会话超时结束时，会话就结束了，Session ID 也就失效了。下次客户端再次请求服务器时，就会生成一个新的Session ID，开始一个新的会话。除非服务器端设置了 Session ID 的持久化，否则 Session ID 是不会被重用的。

“Session ID的持久化”指的是服务器端在会话结束后，不会删除Session ID，而是将其保存到磁盘或数据库中，以便下次客户端请求时可以重新加载。这样可以避免客户端每次请求都需要重新生成Session ID，提高了效率和安全性。但是，这也需要客户端保留Cookie文件，否则无法携带Session ID。

Cookie的安全性和Session ID的持久化是两个不同的问题，Cookie的安全性主要涉及到Cookie的加密，签名，域名，路径等属性，以及客户端和服务器端的验证机制。 Session ID的持久化主要涉及到服务器端的存储和加载机制，以及客户端是否保留Cookie文件。

xx持久化，一般指的就是将内存中的数据保存到磁盘中。

Session ID的存在，使得Cookie文件不再存储私密信息，而是存储私密信息通过算法得到的唯一ID值，只要用户第一次登录，服务器就会为这个会话生成一个唯一的ID，然后通过网络传输到客户端，后续再访问时浏览器会自动携带Session ID作为报文的一部分递交给服务端，这样就相当于拿到了一个长期门禁，服务端只要验证客户端发来的Session ID和本地的Session ID的一致性就能实现保存用户（登录）状态的效果。

8.3 只有相对的安全

Session ID不是绝对安全的，它可能会被劫持，伪造，破解等。例如保存着Session ID的Cookie文件被盗取，那么中间人也能通过它来登录网站，和之前保存着用户的隐私信息的Cookie文件不同的是，中间人无法得知用户的隐私信息，因为隐私信息在服务端通过算法被映射为了一个唯一的字符串ID。

Session ID的安全性取决于多个因素，比如Cookie的设置，网络的加密，服务器端的存储和验证等。为了提高Session ID的安全性，可以采取一些措施，比如使用HTTPS，设置Cookie的HttpOnly和Secure属性，使用加密和签名算法，设置Session ID的有效期和更新机制等。

有了Session ID，用户的隐私信息就被客户端维护，而不是由浏览器维护。

以SessionID的有效性为例：

当异地登录QQ时，它会显示警告信息，当服务器发现IP地址发生更改后，很可能会立即清除之前保存的Session ID，那么用户就要重新登录以更新Session ID。盗号者的“养号”行为就相当于渡过Cookie设置的有效期。

对于某些高风险的服务，服务器可能会要求客户端再次输入密码以验证身份，这个步骤的目的不仅是给用户再次考虑的时间，更是验证用户的身份信息。因为中间人盗取了Session ID后，是有可能进行这一步骤的，但是生成Session ID的算法并不是可逆的，无法倒推出用户的隐私信息。所以即使中间人盗取了用户的Session ID，也无法进行这些被限制的操作，在一定程度上减轻了信息被盗取的风险。

“不存在绝对安全的算法或机制”：

在这个问题上，不同的人可能有不同的观点。有些人认为，只要有足够的时间和资源，任何算法或机制都可能被破解，所以不存在绝对安全的算法或机制。有些人认为，有些算法或机制是基于数学原理或物理定律的，所以它们是无法被破解的，比如一次性密码本，量子密码等。

我个人认为，不存在绝对安全的算法或机制，只有相对安全的算法或机制。理论上可以穷举所有的可能性，只不过其代价会超出想象，如果破解带来的利益远远小于破解的成本，那也就没有破解的必要，这就是相对安全的算法或机制。技术本身一直在进步，或许当下需要千年才能破译的算法，在不久的以后只要十几分钟，安全性在风险下才是有意义的。

突然想到《纸牌屋》里的黑客运用社会工程学不惜冒着风险肉身接近别人，操纵人的心理，这间接证明了直接破解的难度。

此外，Cookie分为两种类型：

会话Cookie是一种临时的Cookie，它保存在浏览器的内存中，当浏览器关闭后，它就会消失。会话Cookie用来保存用户在访问网站时的一些状态信息，比如登录状态，购物车内容等。
持久Cookie是一种长期的Cookie，它保存在用户的硬盘或数据库中，有一个过期时间，除非用户手动清除或到了过期时间，否则它不会被删除。持久Cookie用来保存用户的一些偏好设置，比如语言，主题等，或者用来实现自动登录等功能。

意义：

The type of cookie has an impact on user and website experience and security. In general, a session cookie is more secure than a persistent cookie because it is not stored on the user's device for a long period of time and is not easily hijacked or forged. However, session cookies also have some disadvantages, such as they cannot be used across browsers, and they cannot retain user personalization settings. On the contrary, persistent cookies can improve user experience and convenience, but also increase security risks. Therefore, when using cookies, you should choose the appropriate type according to different scenarios and needs, and pay attention to setting a reasonable expiration time and other attributes.

8.4 Testing

Add the set-cookie field to the response information of the server, so that the client will create a cookie file and fill in the information.

Since the current version of the Chrome browser (114.0.5735.110) cannot directly view the cookie information from the button next to the URL box like the old version. So first press F12 to enter the debug mode, and then type document.cookie in the Console (console), you can see the cookie data filled by the server.

This is just a simple code that demonstrates how the server fills information into the cookie file generated by the client. It cannot manage sessions because it lacks details such as authentication and timeliness. In fact, these user information are generally obtained from the form.

In addition, you can also use a packet capture tool fiddler to test:

Fiddler is a packet capture tool for HTTP, which can capture all HTTP requests of this machine.

Source code: HttpServer Version 4

References

Graphical HTTP.epub . Many illustrations and conceptual explanations in this article are from this book. A picture is worth a thousand words. It is highly recommended for beginners to learn, and you can use the tool caliber to read.
Wikipedia
Blog: Application Layer Protocol - HTTP Protocol

8.4 Testing

Add the set-cookie field to the response information of the server, so that the client will create a cookie file and fill in the information.

In addition, you can also use a packet capture tool fiddler to test:

Fiddler is a packet capture tool for HTTP, which can capture all HTTP requests of this machine.

Source code: HttpServer Version 4

References

Graphical HTTP.epub . Many illustrations and conceptual explanations in this article are from this book. A picture is worth a thousand words. It is highly recommended for beginners to learn, and you can use the tool caliber to read.
Wikipedia
Blog: Application Layer Protocol - HTTP Protocol

HTTP protocol [network foundation/application layer]

Article directory

1. Network basics TCP/IP

2. Protocols closely related to HTTP

2.1 IP protocol responsible for transmission

routing

2.2 TCP protocol to ensure reliability

three handshake

2.3 DNS service responsible for domain name resolution

2.4 Relationship between various protocols and HTTP protocol

3. URL and encoding issues

3.1 Introduction

Format

3.2 Encoding issues

4. Get to know the HTTP protocol first

4.1 CS mode

4.2 Communication through responses and requests

4.3 Request and Response Formats

request format

response format

4.4 Separate the payload in the message

4.5 Introduction to HTTP

test

4.6 Building the response

web root directory

string split

Perfect HTTP request processing function

5. HTTP methods

5.1 GET and POST

test

form

6. HTTP status code

6.1 Redirection

permanent redirect

temporary redirection

302(Found)

307(Temporary Redirect)

redirect test

7. HTTP Header

7.1 Introduction

Format

type

7.2 Common Headers

Content-XX

Host

User-Agent

Refer

Keep-Alive

8. Session management

8.1 HTTP is a protocol that does not store state

8.2 State management using cookies

Set-Cookie Header

Cookie flag status

安全问题

8.2 会话管理

Session

Session ID

8.3 只有相对的安全

8.4 Testing

References

8.4 Testing

References

Guess you like