Network Principles - Network Protocols

transport layer protocol

​The transport
layer is responsible for data transfer from the sender to the receiver. The transport layer protocol mainly includes the TCP protocol and the UDP protocol.

TCP protocol

TCP protocol format segment

The TCP protocol format segment is:

the meaning of each part is:

Source port number (16 bits): identifies the port number of the sender.
Destination port number (16 bits): identifies the port number of the receiver.
Sequence number (32 bits): Used to identify the sequence of data sent.
Confirmation number (32 bits): used to confirm the received data.
Data offset (4 bits): Indicates the length of the TCP packet header.
Reserved (6 bits): Reserved for future use.
Flag bit (6 bits):

• URG : Send this message first, and no longer send it in sequence.
• ACK : Acknowledgment response.
• PSH : Immediately push the message without waiting for the buffer to fill up.
• RST : The other party requests to re-establish the connection; we call the message carrying the RST flag a reset message
• SYN : Request to establish a connection; we call the one carrying the SYN identifier a synchronous message
• FIN : Notify the other party that the local end is going to be closed. We call the end message carrying the FIN flag

Window Size (16 bits): Indicates the amount of data acceptable to the receiver.
Checksum (16 bits): Used to detect errors in data transmission.
Urgent pointer (16 bits): valid only when the URG flag is set, used to indicate the location of urgent data.
Options (variable length): Used to support various optional features.

TCP principle

The principle of the TCP protocol: on the premise of ensuring the security of data transmission, improve the transmission efficiency as much as possible.

connection management

The TCP protocol uses a three-way handshake to establish a connection, and four wave handshakes to disconnect.
Three-way handshake: the sender requests a connection (sends SYN, that is, a synchronous message) -> the receiver confirms the request (after receiving the request, sends ACK, that is, confirms the response) -> the receiver requests a connection (sends SYN) -> the sender confirms Request (send ACK after receiving the request):

Here, the two instructions of the receiver will be sent together because the interval is very short. Therefore, the receiver's ACK+SYN counts as a handshake.
Wave your hands four times: replace SYN in the above figure with FIN (end message). However, the ACK and FIN instructions of the receiver generally cannot be combined, so it is called four waved.
Why can't SYN and FIN be merged? The main reason is that the receiver must wait until the sender closes the resource (that is, close) before sending the FIN command. If the close is executed immediately, then the FIN can be merged with the ACK, but the close may take a while to execute (or not at all). Execution), then FIN obviously cannot be combined with ACK, and can only be sent separately. (The reason why FIN may be merged with ACK is due to the delayed response mechanism of TCP. TCP's ACK command will not be executed immediately, but will be executed after a while.)
Why do we need a three-way handshake?
The three-way handshake is to ensure that the receiving and sending capabilities of the sender and receiver are normal. If A and B make a voice call, A first speaks: "Can you hear me?", B hears what A said at this time (meaning that B knows that A's microphone is normal, and B's earphone is normal), B responds: "Can you hear me?" Arrived.", B said: "Then can you hear what I said?", A can hear (indicating that A knows that A's microphone is normal, B's microphone is normal, and A's earphone is normal), but A still needs Response: "Can hear". (B knows that B's microphone is normal).

reliable transmission

The TCP protocol uses sequence numbers and confirmation numbers to ensure the orderly transmission of data and retransmission of lost packets to achieve reliable transmission. Each transmission requires an ACK confirmation response , and the sender can only confirm that the data has been successfully transmitted after receiving the ACK, so as to ensure the reliability of data transmission. If the sender does not receive the receiver's ACK, then the sender will think that the data transmission has failed, and the sender will retransmit once - this depends on TCP's timeout retransmission mechanism : when the maximum waiting time is exceeded If the sender still does not receive the ACK, the data will be retransmitted.
Of course, there are two situations that will cause the sender to not receive the ACK: the sender loses the packet during sending (in this case, it can only timeout and retransmit); the receiver successfully receives the data, but the returned ACK is not transmitted. In the past (in this case, the retransmission cannot be timed out).
However, the timeout retransmission mechanism does not consider whether the ACK is not received due to which situation, because TCP has an automatic deduplication function, that is, the transmitted data can only be received once. no longer accepted.
In order to ensure the order, TCP numbers each byte of data, which is the sequence number. ACK is an acknowledgment of the highest sequence number of the received data, and returns the sequence number expected to be received next time, thereby ensuring the order of the data.

flow control

In order to ensure reliable transmission, TCP needs to sacrifice a lot of efficiency. However, the principle of the TCP protocol is to improve the transmission efficiency as much as possible under the premise of ensuring the security of data transmission. If you follow the question-and-answer method, if you transmit 1000 bytes at a time, you have to wait for the 1000 bytes to be transferred in place, and then transfer 1000 bytes: this kind of

transmission efficiency is obviously very low. To improve efficiency, you can Use the method of transmitting multiple pieces of data at a time:

this can effectively reduce the waiting time, but how many pieces of data are transmitted at the same time? In the TCP protocol header, there is a window size of a 16-bit field, which is the sliding window. The size of the window is actually the remaining size of the receiving data buffer. The size of the sliding window is the maximum amount of data transferred each time. After the sliding window is full, send ACK, and delete this piece of data in the buffer, then there will be remaining space in the buffer, and the sender can continue to transmit data.
The speed at which the receiving end can process data is limited. If the sending end sends too fast, the buffer at the receiving end is filled, and if the sending end continues to send at this time, packet loss will occur. Therefore, the speed at which the sender transmits data will be dynamically adjusted according to the size of the sliding window, that is, flow control .
The receiving end will put the current buffer size into the window size of the TCP header, and inform the sending end of the window size through ACK. When the sending end sees that the window is small, it will reduce the sending speed, on the contrary, it will increase the sending speed. If the buffer is full, the returned window size is 0. At this time, the sender will stop sending, but will periodically send a window detection data segment, so that the receiver will inform the buffer size, and continue sending if it is not full.
Of course, even with flow control, packet loss cannot be avoided 100%.
What should I do if there is a packet loss?
If the data has not been transmitted, if 1-1000 has not been transmitted, then the sender will always receive a reminder from the receiver: "The next one is 1001", until the data of 1-1000 is received and ACK is returned. After multiple reminders, the sender will resend the data from 1 to 1000. If there is no packet loss later, it will be received normally. When the data of 1~1000 is resent and received, the receiver will return the next ACK of which data is received, for example: 1~1000 is lost , the follow-up data is transmitted normally. After this piece of data is received, 5000 pieces have been received. At this time, the ACK returns: "The next one is 5001".
If only the ACK is lost, it can be confirmed by the subsequent ACK without resending.

congestion control

At the beginning of data transmission, even with a sliding window, a large amount of data cannot be directly transmitted, because we do not know the current network congestion, and it is likely that the current network is already relatively congested. If a large amount of data is transferred rashly at this time , will inevitably cause a large number of packet loss phenomena. Therefore, TCP introduces a slow start mechanism, which first sends a small amount of data, detects the current network situation, and then gradually increases the transmission rate. At this time, the congestion window
is introduced : the size of the congestion window changes dynamically according to the network congestion. Its changes are divided into several stages: slow start, congestion avoidance, fast retransmission, and fast recovery.

• Slow start : When the host starts to send data, it first detects the congestion of the network, that is, gradually increases the congestion window. Usually at the beginning, the window size is set to just accommodate a maximum message segment MSS (Maximum Segment Size) value, and each time an ACK is received, the congestion window is increased by at most one MSS size (each message segment will receive an acknowledgment , so the window size will double every time), so in the early stage, the window size will grow exponentially.
• Congestion avoidance : When the congestion window reaches a certain threshold (slow start threshold, ssthresh), it will enter the congestion avoidance phase. At this stage, the congestion window will grow linearly, that is, every round trip will only increase by 1 MSS size. In this way, the critical value of network congestion can be reached as much as possible.
• Fast retransmission : When the sender receives the third repeated ACK, it considers the datagram lost and retransmits it immediately without waiting for a timeout. Fast retransmission sets the threshold to half of the current congestion window, and sets the congestion window to 1, and then enters the slow start phase.
• Fast recovery : When the sender receives the third repeated ACK, it considers that the datagram is lost, and executes the fast recovery algorithm at this time: adjust the size of the threshold, set it to half of the current congestion window, and set the congestion window to the new The threshold plus 3 MSS size, and then resend the datagram, after receiving the ACK, re-enter the congestion avoidance phase.
  
  In fact, when packet loss occurs, the fast retransmission algorithm will be used first to retransmit the datagram, and then the fast recovery algorithm will be used to adjust the size of the congestion window.
  The size of each datagram sent cannot exceed the minimum value of the congestion window and the window size fed back by the receiver.

Delayed acknowledgment and piggybacking

In order to ensure that the sliding window is as large as possible, the acknowledgment response of the ACK adopts a delayed response. If we put 1k bytes into the buffer, if we ACK immediately, then the space of 1k bytes will be useless. If we wait for a while, these 1k bytes may be processed, and the returned The window size is just bigger. Based on the delayed response mechanism, four waved hands like the above-mentioned disconnection may become three waved hands. (ACK delays the response, and close is executed just during this period, so FIN can respond with ACK, that is, piggyback response )

error detection

The TCP protocol uses checksums to detect errors in the transmission of datagrams. Before sending the datagram, the sender first calculates the checksum of the datagram and stores it in the checksum field of the message. After receiving the data, the receiver recalculates the checksum and compares it with the checksum in the TCP message. If the checksums are different, the transmission has been misplaced.

TCP exception

In abnormal disconnection, process termination and machine restart will not cause TCP disconnection abnormality, and FIN will still be sent normally at the end (FIN is controlled by the transport layer and has nothing to do with the process), if there is no time after receiving FIN The ACK is turned off. At this time, the receiving end will try to resend the FIN. After resending several times, it will recharge the connection, and if it still fails, it will disconnect.
But if there is a power outage or network disconnection, the sender has no time to send the FIN message, so the receiver thinks that the connection is still there. But TCP has a heartbeat packet keep-alive mechanism , which will periodically ask whether the peer is still there, and if the peer is not, the connection will be disconnected.

UDP protocol

The UDP protocol is much simpler than TCP, and its main advantage is the transmission speed block.

UDP protocol format segment

Features of UDP protocol

no connection

The transmission based on the UDP protocol does not need to be connected, and only needs to know the IP and port number of the opposite end to transmit.

Unreliable

UDP transmission does not consider whether the other party has received it, so a large amount of data can be sent at one time. Of course, if a packet loss problem occurs, UDP has no corresponding remedial measures.

Datagram Oriented

Each transmission and reception of UDP must be received and sent in the form of a datagram. If there are 100 bytes in this datagram, only 100 bytes can be read at a time when reading, not one by one or 10 bytes. 10 reads.

size limited

The maximum size of data transmitted by UDP each time is 64k. If the transmitted data is larger than 64k, the most common method is to separate and assemble it at the application layer (split into smaller than 64k); or send it in the form of a large packet. The link layer then performs packetization and grouping.

network layer protocol

The network layer protocol is mainly the IP protocol, and its function is to determine the path of network transmission.

IP protocol

IP protocol format section :

version : used to identify the version number of IP, currently there are two versions, IPV4 and IPV6.
Header length : used to indicate the header length of the IP protocol. The size is 4 bytes, and the maximum number represented by one byte is 1111 (ie 15), so the maximum length of the header is 60 bytes. The most commonly used header length is 20 bytes.
Service type : used to obtain better services. Consists of a three-bit priority field (deprecated), four TOS bits, and 0. The 4-bit TOS respectively represent: minimum delay, maximum throughput, highest reliability, and minimum cost. These four conflict with each other, and only one of them can be chosen.

The Type of Service field is never actually used. In 1988, IETF changed the name of this field to Differentiated Services and redefined it: the first six bits were defined as DSCP (Differentiated Services Code Point), and the last two bits were reserved. Therefore this field is now commonly referred to as the DS field.

Total datagram length : the number of bytes occupied by the IP datagram as a whole.
Identifier : uniquely identifies the message sent by the host. If the length of the message to be sent is greater than the maximum length supported by the data link layer, that is, exceeding the maximum transmission unit (MTU) (1500 bytes in the case of Ethernet), the message needs to be divided into several parts before proceeding. For transmission, these divided parts all have the same identifier (id), indicating that this is a message.
Flag bit : the first bit is reserved for later use; the second bit indicates whether the packet is allowed to be fragmented (1 means not allowed at this time, if the packet length is greater than the maximum transmission unit, the packet will be discarded and an ICMP error will be returned to the sender); the second bit The three bits indicate whether it is the last fragment, and if so, it is 1.
Chip displacement : also called offset. After IP fragmentation, the transmission time of each part is not the same, and it is possible that the last one comes first, so the offset of each fragment is different, and finally the packet is reassembled according to the offset. (Similar to TCP principle)
Survival time : Indicates the maximum number of nodes (TTL, generally 64) to reach the destination. Every time a node (router) passes through, TTL-1, when TTL is 0, the destination has not yet been reached. It is considered that there is a problem with routing addressing (there may be an infinite loop), and the message will be discarded at this time, and the sender will be notified.
Protocol : used to indicate the upper layer protocol type.
Checksum : Check the IP protocol header to judge whether the transportation is normal.
Source IP : 32 bits, indicating the datagram source IP.
Destination IP : 32 bits, indicating the destination IP of the datagram.
Option : optional field, you can change some optional header formats, if you change the header length, you need to ensure that the length is a multiple of 4.

Data Link Layer Protocol

The most commonly used protocol at the data link layer is the Etnernet Ethernet protocol .

​Ethernet

Ethernet is a computer local area network technology, which stipulates some content including the physical layer, and Ethernet is the most widely used local area network technology.
Ethernet frame format :

the source mac address and the destination mac address represent the hardware address of the network card, (the physical address, which is set with the network card at the factory and cannot be changed) is used to represent the starting point and end point of each transmission.
The frame protocol type field has three values: respectively corresponding to IP, ARP, RARP;
CRC : checksum, check whether the transmission is normal.

PERSON

MTU, the Maximum Transmission Unit, means the maximum transmission unit . Different data link layer protocols have different MTUs, and the MTU of Ethernet is 1500 bytes.
The role of MTU is to limit the size of each datagram transmission, and MTU will affect the transmission of upper layer protocols.

ARP protocol

The ARP protocol establishes the mapping relationship between IP and mac. Each transmission uses the IP address to search the destination mac address in the local ARP cache table. If it is found, it can be directly transmitted to the destination mac address. If it is not found, it will broadcast to find the destination mac address.