Computer network (in the learning process-continuous update)

1. OSI seven-layer protocol, TCP/IP four-layer protocol, and five-layer protocol

OSI seven-layer protocol

1. Application layer: Provide services for specific applications, such as SMTP (email), HTTP, DNS, etc. for data transmission
2. Presentation layer: Convert the information processed by the application into a format suitable for network transmission
3. Session layer: responsible for establishing and disconnecting communication connections
4. Transport layer: The protocols of this layer mainly include TCP and UDP, which are used for data transmission between the host and the host.
5. Network layer: The protocol of this layer is IP protocol, address management and routing
6. Data link layer: responsible for the communication and transmission between nodes on the physical level
7. Physical layer: responsible for the interchange between 0 and 1 bit streams, voltage levels, and light flashes

TCP/IP four-layer protocol

1. Application layer: The functions of the session layer, presentation layer and application layer in the OSI reference model are all integrated into the application program.
2. Transport layer: TCP, UDP protocol, provide reliable data transmission
3. Network layer: IP protocol, responsible for address management and routing
4. Network interface layer: use the data link layer of Ethernet for communication and bit stream for physical transmission

Five layer protocol

1. Application layer
2. Transport layer
3. Network layer
4. data link layer
5. Physical layer

Chart summary

Two, TCP, UDP

TCP protocol

TCP protocol, also known as Transmission Control Protocol.
it is:

• Connection-oriented, reliable, byte stream-based transport layer communication protocol
• Divide the data stream of the application layer into segments and transmit them to the TCP layer of the target node
• The data packet has a sequence number, the other party will send an ACK to confirm if it is received, otherwise it will be retransmitted
• Use checksum to verify whether the data is incorrect during transmission

UDP protocol

it is:

• A connectionless, unreliable, message-based transport layer communication protocol
• Does not maintain the connection status and supports simultaneous transmission of the same message to multiple clients
• The packet header is only 8 bytes, with less additional overhead
• Throughput is not adjusted by congestion algorithm, only limited by data generation rate, transmission rate and machine performance
• Best effort delivery, reliable delivery is not guaranteed, no need to maintain complex link status tables
• The UDP protocol has a faster transmission speed than the TCP protocol, because it directly adds the header to the message transmitted from the application layer and then transmits it to the lower IP layer without splitting or merging.

The difference between TCP and UDP

• Connection-oriented vs connection-oriented: TCP information transmission is to establish a connection, but UDP does not need to be, it is suitable for one point to multiple point transmission
• Reliability: TCP provides reliability guarantee through handshake and retransmission mechanism, while UDP may be lost
• Orderliness: TCP uses sequence numbers to ensure the order delivery of data. Arrivals may be disordered, but TCP will eventually sort, while UDP does not have orderliness.
• Speed: TCP speed is relatively slow, because it needs to create a connection to ensure the reliability and orderliness of the data, so more additional operations need to be done, and UDP is suitable for speed-sensitive applications, such as online video media, TV broadcasting, Multiplayer online games, etc.
• Weight: TCP is heavyweight, UDP is lightweight, which is reflected in the header size of the source data, TCP is 20 bytes, and UDP is 8 bytes

TCP three-way handshake

Handshake process

• The first handshake: the client sends a SYN (seq=x) segment to the server
• The second handshake: the server sends a SYN(seq=y)+ACK(ack=x+1) segment to the client
• The third handshake: the client sends an ACK (seq=y+1) segment to the server

Why is there a three-way handshake? Isn't it two or four times?

The first handshake is to request the establishment of a transmission channel from the client to the server. The second handshake is to confirm that the transmission channel from the client to the server is normal and request the establishment of a transmission channel from the server to the client. The third handshake is to confirm the server to the client. The end transmission channel is normal. The three-way handshake just ensures that the two channels can be transmitted normally, two times less, and four times more.

The hidden danger of TCP's first handshake-SYN timeout

The cause
of the problem Since the server received the SYN from the client and did not receive the ACK confirmation when replying to the SYN-ACK, the server will continue to retry sending the SYN-ACK packet until it times out.
For SYN Flood protection measures
, after the SYN queue is full, send back SYN Cookie through the tcp_syncookies parameter.
If the connection is normal, the Client will send back the SYN Cookie.
After the connection is established directly , the client
will fail. There will be a keep-alive mechanism, which is yes. Send a keep-alive detection message to the other party. If you continue to send it to receive the response, there will be a pre-set keep-alive time interval. If there is no reply within the specified time, the connection will be terminated.

TCP waved four times

Wave process

• First wave: After the client finishes transmitting the data, it will send a FIN message to the server to close the transmission channel from the client to the server, and the client enters the FIN-WAIT-1 state
• The second wave: The server receives the FIN message sent by the client, and it will immediately reply with an ACK message to confirm that the transmission channel from the client to the server is closed, and the server enters the CLOSE-WAIT state
• The third wave: the server will send a FIN message to the client after transmitting the data, and the server will enter the LAST-ACK state
• The fourth wave: The client receives the FIN message from the server and replies with an ACK message to confirm that the transmission channel from the server to the client is closed. The client enters the TIME-WAIT (2MSL) state, and then The server enters the CLOSED state, and the client waits for the 2MSL (MaximumSegmentLifetime) time to enter the CLOSED state.

Why is there a TIME-WAIT state when waving

Make sure that there is enough time for the other party to receive the ACK packet, which is exactly 2MSL time

Why wave four times?

Because the TCP connection is a full-duplex service, both the sender and the receiver need FIN messages and ACK messages to close the channel and confirm that the channel is closed

The characteristics and purpose of TCP

The characteristic of TCP is that the transmission is reliable. If the transmission is reliable, it must solve the problems of data destruction, packet loss, duplication, and disordered fragment sequence.
TCP passes the checksum, sequence number, confirmation response, retransmission control, connection management, and window Control and other mechanisms to achieve reliable transmission

How does TCP ensure reliable transmission

Improve reliability through serial number and confirmation response

In TCP, when the sender's data arrives at the receiving host, the receiving host will return a message, which is called an acknowledgement (ACK).
TCP uses a positive acknowledgement (ACK) to ensure reliable data transmission. When the sender sends it out After the data is received, the sender will wait for the confirmation response from the receiver. If the confirmation response is received, it means that the batch of data sent has been successfully sent to the opposite end. If the confirmation response is not received after a period of time, then It will consider whether packet loss occurs during data transmission, and the sender will retransmit at this time. As a result, even if packet loss occurs, it can ensure that data can reach the opposite end and achieve reliable transmission.
There are two possibilities for packet loss at this time:

• Data packet loss: packet loss due to network congestion during the sending process. In this case, it will be re-sent to ensure that the data can reach the receiving end correctly
• Confirm that the response is lost After the receiving end receives this batch of data, and then the receiving end sends back a message to the sending end, in this process, packet loss occurs. In this case, the sender will also retransmit, but the receiver will discard the data after receiving it, and then the receiver will send back an ACK.

The above-mentioned functions such as confirmation response processing, retransmission control, and repetition control can all be realized by serial numbers. The sender sends a data packet (1~100), and when the receiver receives it, it sends back the sequence number that it should accept in the next step as a confirmation response. In this way, TCP can achieve reliable transmission through the serial number and confirmation response number.

How to determine the retransmission timeout

Retransmission timeout refers to the specific time interval to wait for the confirmation response before retransmitting the data. If the confirmation response is not received after this time, the sender will retransmit the data. So how is the specific length of time for this retransmission timeout determined?

TCP requires high performance in any network environment and must maintain this feature regardless of changes in network congestion. It calculates the round-trip time (RTT) of each packet and its deviation, and the retransmission timeout is a slightly larger value than this sum.

There are reasons why the calculation of the retransmission timeout must consider both the round trip time and the deviation. According to the network environment, the round trip time of the data packet may fluctuate greatly. This happens because the segments of the data packet arrive through different lines. The purpose of TCP/IP is even in this environment Also control, try not to waste network traffic

In BSD Unix and Windows systems, the timeout is in units of 0.5 seconds (the minimum deviation is 0.5, so the minimum retransmission time is at least 1 second). The round-trip time is not known in the initial data packet, so it is repeated The sending timeout is generally set to 6 seconds.

After the data is retransmitted, if the confirmation response is not received, it will be retransmitted again. However, the number of retransmissions cannot be unlimited. When a certain number of times is reached, the connection will be forcibly terminated because of the network reason or the peer has a problem.

TCP sends data in units of segments

While establishing a TCP connection, the unit of sending data packets can also be determined. We can also call it the "Maximum Message Length" (MSS). When TCP transmits a large amount of data, the data is divided and sent by the size of MSS. The unit of MSS is also used for retransmission.

So how is this MSS calculated? It is determined when the TCP three-way handshake establishes the connection. During the first handshake, the client recommends that the MSS be set to a certain length size1, and during the second handshake, the server recommends that the MSS be set to a certain length size2. Finally, the third handshake determines the smaller of these two sizes and sends data.

Use window control to increase speed

TCP takes 1 segment as a unit, and each segment needs to receive a confirmation response. But this transmission method has a disadvantage. That is, the longer the round-trip time of the packet, the worse the communication performance.

In response to the above problems, TCP introduces the concept of a window, which can control the degradation of network performance even in the case of a long round-trip time. At this time, the confirmation response is no longer in units of a segment, but in a larger unit. This will greatly shorten the forwarding time.

In the case of receiving a confirmation response, slide the window to the position of the serial number in the confirmation response. In this way, multiple segments can be sent simultaneously to improve communication performance. This mechanism is also called sliding window control.

Window control and retransmission control

Speaking of this window, the size of the window is what everyone cares about. Who decides the size of the window? It is determined by the ability of the receiving end to process data. Here involves a 缓冲区concept, the buffer zone represents a place for temporarily storing data sent and received. The sender sends the data to the receiver, and when the receiver replies with a confirmation response, the size of the remaining buffer is added. This size is the window size for the next transmission.

For the data in the window, after the sender sends them out, the sender needs to set up a buffer to save these data. It will be clear from the buffer until the corresponding confirmation response is received. If the confirmation response is not received, the data will be saved Resending, here is another concept involved 高速重发. As we mentioned before, if the sender does not receive the confirmation response, it will retransmit after the retransmission timeout period. But here, it will be different. TCP Set, after the data in the window is sent out, if the confirmation response sent by the receiving end is received 3 times, the retransmission will be triggered. This mechanism is more efficient than the timeout management mentioned earlier.

When the window is sending data, one packet is lost, and the subsequent ones will continue to send. But what to do if this piece is lost. In fact, when the receiving end does not receive the data of the sequence number it expects, it will confirm the data received before. Once the sender receives a certain confirmation response and receives the same confirmation response three times in a row, it is considered that the data segment has been lost and needs to be retransmitted.

Flow control

The sender sends data according to its actual situation. But under certain circumstances, an irrelevant data packet may be received and it may take some time to deal with other issues. Under high load conditions, the receiving end cannot receive any data. In this way, if the receiving end discards the data that should have been received, it will trigger the sending end's retransmission mechanism, which will cause unwarranted waste of network traffic. , There needs to be a mechanism to prevent this from happening.

TCP solves this problem through flow control. The buffer zone we mentioned earlier is related to this mechanism. The receiving end will have a buffer for storing the data to be processed. Every time the receiving end receives data, it will put this data into this buffer, and then when replying to the confirmation response, it will piggyback on the remaining buffer the size of. Next time the sender will set this size to the size of the window to send data. If the window size is 0, the sender will not send data.

After a retransmission timeout period has elapsed, if the window update notification has not been received, the sender will send a window detection packet. This data contains only one byte to obtain the latest window size information.

Congestion control

TCP adopts window control, which can send a large number of data packets continuously even if the confirmation response is no longer sent in units of a data segment between the receiving and sending hosts. However, if a large amount of data is sent at the beginning of communication, other problems may also arise. Because the computer network is a shared environment, it may also be congested due to the communication between other hosts. When the network is congested, a large amount of data is suddenly sent at this time, which will most likely lead to the paralysis of the entire network .

In order to prevent this problem from happening, TCP uses a value derived from an algorithm called slow start at the beginning of communication to control the amount of data sent.

In order to adjust the amount of data to be sent at the sender, a concept called "congestion window" is defined. During slow start, set the window size to 1 data segment (1MSS) to send data, and then the value of the congestion window will be +1 for each response. When sending a data packet, the window size is compared with the sliding window size, and a smaller value is selected to send a smaller amount of data.

If the retransmission adopts a timeout mechanism, then the initial value of the congestion window is set to 1 and then retrograde slow start correction will be made. With the above mechanism, the network congestion caused by the continuous sending of data packets at the beginning of communication can be effectively reduced, and the occurrence of network congestion can also be avoided.

However, with each round-trip of the packet, the congestion window will also increase with an exponential function such as 1, 2, 4, and the surge in congestion may even cause network congestion. To prevent this, the concept of slow-start valve is introduced. As long as the value of the congestion window exceeds this threshold, the increase will be reduced every time a confirmation response is received.

When TCP communication starts, the corresponding slow start threshold is not set. Instead, it will be set to half the size of the congestion window at the time when retransmission is timed out, and the initial default value is 1. When the notification retransmission control is performed by the repeated confirmation response, after the size of the slow start threshold is set to half of the current window, the initial default size of the window is set to the slow start threshold + the size of 3 data segments.

HTTP protocol

Introduction

When the user enters the URI of the Web page to be accessed in the address bar of the browser, HTTP processing starts immediately, and HTTP uses port 80 by default. Its working mechanism is that the client establishes a TCP connection to port 80 of the server, and then requests and answers and sends data messages on this TCP connection.

There are two commonly used versions of HTTP, one is HTTP1.0 and the other is HTTP1.1. At 1.0, it will establish a TCP connection every time it sends a command and response. Starting from version 1.1, it allows sending on a TCP connection. Multiple commands and responses (keep-alive), thereby reducing TCP connection establishment and disconnection operations, thereby improving efficiency

Main HTTP commands and response codes

The main commands of HTTP

HTTP main commands
OPTIONS	Setting Options
GET	Get the data of the specified URL
HEAD	Get only the first part of the document
POST	Request the server to receive the URI specified document as executable information
PUT	Request the server to save the data transmitted by the client to the URI specified document
DELETE	Request the server to delete the page specified by the URI
TRACE	Request message returned to the client

HTTP main status code

1xx: The server is normal and can continue to send requests

2xx: processed successfully

200 Success: indicates that the request from the client is processed normally on the server.
204 No Content: The request is processed successfully, but no data is returned.
206 Partial Content: The request is processed successfully, and the client only needs part of the data

3xx: Redirect

The 3xx response result indicates that the browser needs to perform some special processing in order to handle it correctly

301: Permanent redirection. This status code indicates that the requested resource has been assigned a new URI, and the URI that the resource currently refers to will be used in the future. In other words, if you have saved the URI corresponding to the resource as a bookmark, you will directly visit
302: Temporary redirect next time . This status code indicates that the requested resource has been assigned a new URI. I hope the user (this time) Can be accessed using the new URI

4xx: Client error

The 4xx response indicates that the client is the cause of the error

401 Unauthorized: This status code indicates that the request sent requires authentication information that has passed HTTP authentication (BASIC authentication, DIGEST authentication).
403 Forbidden: This status code indicates that access to the requested resource is denied by the server.
404 Not Found: This status code indicates that the requested resource cannot be found on the server.

5xx: server error

The 5xx response indicates that an error occurred on the server itself

500 Internal Server Error: This status code indicates that an error occurred on the server side while executing the request.
503 Service Unavailable: This status code indicates that the server is temporarily overloaded or is shutting down for maintenance, and the request cannot be processed now.

HTTPS for web security

What are the disadvantages of HTTP

The communication uses plain text (not encrypted), the content may be eavesdropped

HTTP performs data transmission with each other after making a TCP connection, but during the transmission process, these contents are in plain text, and such data packets are easily eavesdropped on the Internet.

The identity of the communicating party is not verified, so it is possible to encounter masquerading

When HTTP is connecting, it will not determine who sent it to the request it receives. It only needs to receive the request and process it, and then return the response. In such a situation, disguise may be encountered, and anyone can initiate a request. The above may cause the following hidden dangers

• It is impossible to determine whether the web server that the request is sent to the target is the one that returned the response according to the real intention. It may be a disguised web server.
• It is impossible to determine whether the response returned to the client is the one that received the response according to its true intentions. It may be a disguised client
• Cannot determine whether the communicating party has access rights. Because important information is stored on some web servers, I only want to send it to specific users with the communication authority
• There is no way to determine where the request came from or who made it.
• Even meaningless requests will be accepted in full. Unable to prevent Dos attacks (Denial of Service) under massive requests

Unable to prove the integrity of the message, so it may be tampered with

The so-called completeness refers to the accuracy of information. Failure to prove its completeness usually means that it is impossible to judge whether the information is accurate. It is also possible that the received content may be wrong.

Due to the shortcomings of HTTP above, HTTPS appeared

HTTPS=HTTP+encryption+authentication+integrity constraint
We used to be HTTP+TCP, while HTTPS is HTTP+SSL and TSL+TCP. To put it bluntly, HTTPS is HTTP with an SSL shell.

Public key encryption technology for mutual exchange of secret keys

HTTPS uses a hybrid encryption mechanism, that is, an encryption mechanism that combines shared secret key encryption and public key encryption. The HTTPS exchange of secret keys between the client and the server is divided into two steps:

• Use public key encryption to securely exchange the secret key used in later shared key encryption
• Under the premise of ensuring that the exchanged secret key is safe, use shared secret key encryption for communication
  

Certificate that proves the correctness of the public key

In fact, the public key encryption method mentioned above is also flawed, which means that the public key itself cannot prove that it is a genuine public key. For example, when preparing to establish a public key encryption communication with a certain server, how to prove that the public key received is the public key of the server originally expected. Perhaps during the transmission of the public key, the real public key has been replaced by the attacker.

In order to solve this problem, you can use the public key certificate by the digital certificate certification authority and its related agencies. The digital certificate certification authority is in the position of a third-party organization that can be trusted by both the client and the server.

Verification process: The server will send this public key certificate issued by the digital certificate authority to the client for public key encryption communication. The client receiving the certificate will be verified through SSL. If the verification is passed, it can be determined that the peer is authentic and the server's public key is trustworthy.

HTTPS establishment of SSL connection process

Three, DNS

How DNS is addressed

1. The client sends a query request and caches the search on the local computer. If it does not find it, it will send the request to the dns server.
2. Send it to the local dns server first, and the local will search in its own area. If found, according to This record will be parsed. If it is not found, it will be searched in the local cache.
3. The local server will not find the information queried by the client and will send the request to the root domain name dns server.
4. The root domain name server will resolve the client request The root domain part, which returns the address of the next-level DNS server contained in the client's DNS server address
5. The client's DNS server then accesses the next-level DNS server according to the returned information.
6. This recursive method one Close to the target of the query, and finally get the corresponding IP information on the server with the target domain name.
7. The local dns server of the client will return the query result to our client.
8. The client will access according to the obtained IP information. Target host, complete the resolution process