@
Top/star public account????, hardcore articles will be delivered as soon as possible!
Link | https://juejin.im/post/6844903490595061767
1. TCP/IP model
The TCP/IP protocol model (Transmission Control Protocol/Internet Protocol) includes a series of network protocols that form the basis of the Internet and is the core protocol of the Internet.
The reference model based on TCP/IP divides the protocol into four layers, which are link layer, network layer, transport layer and application layer. The figure below shows the contrast between the layers of the TCP/IP model and the OSI model.
The TCP/IP protocol family is packaged layer by layer from top to bottom. The top layer is the application layer, which includes http, ftp and other familiar protocols. The second layer is the transport layer, and the famous TCP and UDP protocols are at this level. The third layer is the network layer, where the IP protocol is responsible for adding IP addresses and other data to the data to determine the destination of the transmission. The fourth layer is the data link layer. This layer adds an Ethernet protocol header to the data to be transmitted, and performs CRC encoding to prepare for the final data transmission.
The above figure clearly shows the role of each layer in the TCP/IP protocol, and the communication process of the TCP/IP protocol actually corresponds to the process of data stacking and popping. In the process of stacking, the data sender continuously encapsulates the header and tail at each layer, and adds some transmission information to ensure that it can be transmitted to the destination. In the process of stacking, the data receiver continuously removes the header and tail at each layer to obtain the final transmitted data.
The above figure uses the HTTP protocol as an example to illustrate it in detail.
Second, the data link layer
The physical layer is responsible for the exchange between 0 and 1 bit streams, the voltage level of physical devices, and the flickering of lights. The data link layer is responsible for dividing the sequence of 0 and 1 into data frames for transmission from one node to another adjacent node. These nodes are uniquely identified by MAC (MAC, physical address, a host will have a MAC address).
Encapsulation into a frame: add a header and a tail to the network layer datagram, and encapsulate it into a frame, and the frame header includes the source MAC address and the destination MAC address.
Transparent transmission: zero bit padding, escape characters.
Reliable transmission: It is rarely used on a link with a very low error rate, but the wireless link WLAN will ensure reliable transmission.
Error detection (CRC): The receiver detects errors and discards the frame if an error is found.
3. Network layer
1. IP protocol
The IP protocol is the core of the TCP/IP protocol. All TCP, UDP, IMCP, and IGMP data are transmitted in the IP data format. It should be noted that IP is not a reliable protocol, which means that the IP protocol does not provide a processing mechanism after the data is not communicated, which is considered to be something that the upper layer protocol: TCP or UDP has to do.
1.1 IP address
In the data link layer, we generally identify different nodes by MAC address, and in the IP layer, we also have a similar address identification, which is the IP address.
The 32-bit IP address is divided into network bits and address bits. This can reduce the number of routing table records in the router. With the network address, you can limit the terminals with the same network address to be in the same range. Then the routing table only needs to By maintaining a direction of this network address, the corresponding terminals can be found.
Class A IP address: 0.0.0.0~127.0.0.0
Class B IP address: 128.0.0.1~191.255.0.0
Class C IP address: 192.168.0.0~239.255.255.0
1.2 IP protocol header
Here we only introduce: the eight-bit TTL field. This field specifies how many routes the packet passes before being discarded. Every time an IP data packet passes through a router, the TTL value of the data packet will be reduced by 1. When the TTL of the data packet becomes zero, it will be discarded automatically.
The maximum value of this field is 255, that is to say, a protocol packet will be discarded after passing through the router 255 times. Depending on the system, this number is different, usually 32 or 64.
2. ARP and RARP protocols
ARP is a protocol for obtaining MAC addresses based on IP addresses.
The ARP (Address Resolution) protocol is a resolution protocol. Originally, the host does not know which interface of the host this IP corresponds to. When the host wants to send an IP packet, it will first check its own ARP cache (that is, An IP-MAC address correspondence table cache).
If the queried IP-MAC value pair does not exist, the host will send an ARP protocol broadcast packet to the network, which contains the IP address to be queried, and all hosts that directly receive the broadcast packet will query their own IP address, if a certain host that receives the broadcast packet finds that it meets the conditions, it will prepare an ARP packet containing its own MAC address and send it to the host that sent the ARP broadcast.
After the broadcast host receives the ARP packet, it will update its own ARP cache (that is, where the IP-MAC correspondence table is stored). The host that sends the broadcast will use the new ARP cache data to prepare the data link layer for sending data packets.
The work of the RARP protocol is opposite to this, and will not be described in detail.
3. ICMP protocol
The IP protocol is not a reliable protocol, and it does not guarantee the delivery of data, so, naturally, the work of ensuring the delivery of data should be completed by other modules. One of the important modules is the ICMP (Internet Control Message) protocol. ICMP is not a high-level protocol, but an IP layer protocol.
An error occurred while transmitting an IP packet. For example, the host is unreachable, the route is unreachable, etc., the ICMP protocol will package the error information and send it back to the host. Give the host a chance to handle errors, which is why it is possible to achieve security with protocols built above the IP layer.
Four, ping
Ping is arguably the most famous application of ICMP, which is part of the TCP/IP protocol. Use the "ping" command to check whether the network is connected, which can help us analyze and determine network failures.
For example: when one of our websites cannot be accessed. Usually ping this website. ping will echo some useful information. The general information is as follows:
The word ping comes from sonar positioning, and what this program does is, it uses ICMP protocol packets to detect whether another host is reachable. The principle is to use ICMP with type code 0 to send a request, and the host receiving the request responds with ICMP with type code 8.
5. Traceroute
Traceroute is an important tool for detecting the route between the host and the destination host, and it is also the most convenient tool.
The principle of Traceroute is very very interesting. After receiving the IP of the destination host, it first sends a UDP data packet with TTL=1 to the destination host, and after the first router passing through receives this data packet, it automatically sends The TTL is decremented by 1, and after the TTL becomes 0, the router discards the packet, and at the same time generates an ICMP datagram that the host is unreachable to the host. After the host receives this datagram, it sends a UDP datagram with TTL=2 to the destination host, and then stimulates the second router to send an ICMP datagram to the host. And so on until reaching the destination host. In this way, traceroute gets all router IPs.
6. TCP/UDP
Both TCP/UDP are transport layer protocols, but they have different characteristics and different application scenarios. The following is a comparative analysis in the form of a chart.
message-oriented
The message-oriented transmission method is that the application layer sends the message to UDP, and UDP sends it as it is, that is, sends one message at a time. Therefore, the application must choose the appropriate size of the message. If the packet is too long, the IP layer needs to be fragmented, reducing efficiency. If it is too short, it will be that the IP is too small.
stream-oriented
For byte streams, although the interaction between the application program and TCP is one data block (of different sizes) at a time, TCP regards the application program as a series of unstructured byte streams. TCP has a buffer. When the data block transmitted by the application program is too long, TCP can divide it into shorter pieces and then transmit it.
Regarding congestion control and flow control, it is the focus of TCP, which will be explained later.
Some applications of TCP and UDP protocols
When should TCP be used?
When there are requirements for the quality of network communication, for example: the entire data must be transmitted to the other party accurately, which is often used in some reliable applications, such as HTTP, HTTPS, FTP and other file transfer protocols, POP, SMTP and other mail transmission agreement.
When should UDP be used?
When the quality of network communication is not high, and the speed of network communication is required to be as fast as possible, then UDP can be used.
7. DNS
DNS (Domain Name System, Domain Name System), a distributed database on the Internet that maps domain names and IP addresses to each other, enables users to access the Internet more conveniently, without having to remember the IP number string that can be directly read by the machine. The process of finally obtaining the IP address corresponding to the host name through the host name is called domain name resolution (or host name resolution). The DNS protocol runs on top of the UDP protocol, using port number 53.
Eight, TCP connection establishment and termination
1. Three-way handshake
TCP is connection-oriented, no matter which party sends data to the other party, it must first establish a connection between the two parties. In the TCP/IP protocol, the TCP protocol provides a reliable connection service, and the connection is initialized through a three-way handshake. The purpose of the three-way handshake is to synchronize the sequence numbers and confirmation numbers of both parties and exchange TCP window size information.
The first handshake : the connection is established. The client sends a connection request segment, sets the SYN bit to 1, and the Sequence Number to x; then, the client enters the SYN_SEND state and waits for the confirmation from the server;
The second handshake : The server receives the SYN segment. The server needs to confirm the SYN message segment after receiving the SYN message segment from the client, and set the Acknowledgment Number to x+1 (Sequence Number+1); at the same time, it needs to send the SYN request message itself, and set the SYN bit to 1 , the Sequence Number is y; the server puts all the above information into a message segment (that is, the SYN+ACK message segment), and sends it to the client together, and the server enters the SYN_RECV state at this time;
The third handshake : the client receives the SYN+ACK segment from the server. Then set the Acknowledgment Number to y+1, and send an ACK message segment to the server. After the message segment is sent, both the client and the server enter the ESTABLISHED state, and complete the TCP three-way handshake.
Why the three-way handshake?
In order to prevent the invalid connection request segment from being transmitted to the server suddenly, an error is generated.
Specific example: "The invalid connection request segment" is generated in such a situation: the first connection request segment sent by the client is not lost, but stays in a certain network node for a long time , so that it is delayed until some time after the connection is released to reach the server. It turns out that this is a segment that has already expired. However, after receiving the invalid connection request segment, the server mistook it as a new connection request sent by the client again.
Then a confirmation message segment is sent to the client, agreeing to establish a connection. Assuming that the "three-way handshake" is not used, as long as the server sends a confirmation, a new connection is established. Since the client has not issued a request to establish a connection, it will ignore the server's confirmation and will not send data to the server. But the server thinks that the new transport connection has been established, and has been waiting for the client to send data. In this way, many resources of the server are wasted. The "three-way handshake" method can prevent the above phenomenon from happening. For example, in the situation just now, the client will not send a confirmation to the server's confirmation. Since the server does not receive the confirmation, it knows that the client did not request to establish a connection. "
2. Wave four times
When the client and server establish a TCP connection through a three-way handshake, when the data transmission is complete, the TCP connection must be disconnected. For the disconnection of TCP, there is a mysterious "four breakups".
The first breakup : host 1 (it can be the client or the server), set the Sequence Number, and send a FIN segment to host 2; at this time, host 1 enters the FIN_WAIT_1 state; this means that host 1 has no data to request Sent to host 2;
The second breakup : host 2 receives the FIN message segment sent by host 1, and returns an ACK message segment to host 1, and the Acknowledgment Number is the Sequence Number plus 1; host 1 enters the FIN_WAIT_2 state; host 2 tells host 1, "I" Agree to your close request;
the third breakup : host 2 sends a FIN segment to host 1, requesting to close the connection, and host 2 enters the LAST_ACK state;
The fourth breakup : host 1 receives the FIN segment sent by host 2, sends an ACK segment to host 2, and then host 1 enters the TIME_WAIT state; after host 2 receives the ACK segment from host 1, it closes the connection ; At this time, if host 1 still does not receive a reply after waiting for 2MSL, it proves that the server has been shut down normally. Well, host 1 can also close the connection.
Why break up four times?
The TCP protocol is a connection-oriented, reliable, byte stream-based transport layer communication protocol. TCP is in full-duplex mode, which means that when host 1 sends a FIN segment, it just means that host 1 has no data to send, and host 1 tells host 2 that all its data has been sent; however, At this time, host 1 can still accept data from host 2; when host 2 returns the ACK segment, it means that it already knows that host 1 has no data to send, but host 2 can still send data to host 1; when host 2 also When the FIN segment is sent, it means that host 2 has no data to send, and it will tell host 1 that I have no data to send, and then both parties will happily interrupt the TCP connection.
Why wait for 2MSL?
MSL: Maximum Segment Lifetime, which is the maximum time that any segment can be in the network before being discarded. There are two reasons:
Ensure that the full-duplex connection of the TCP protocol can be closed reliably
Ensure that duplicate data segments for this connection disappear from the network
The first point: If host 1 is directly CLOSED, then due to the unreliability of the IP protocol or other network reasons, host 2 did not receive the last ACK from host 1. Then the host 2 will continue to send the FIN after the timeout. At this time, because the host 1 is CLOSED, the connection corresponding to the resent FIN cannot be found. Therefore, host 1 does not directly enter CLOSED, but maintains TIME_WAIT. When receiving FIN again, it can ensure that the other party receives ACK, and finally closes the connection correctly.
The second point: If host 1 is directly CLOSED, and then initiates a new connection to host 2, we cannot guarantee that the port number of this new connection is different from that of the connection just closed. That is to say, it is possible that the port numbers of the new connection and the old connection are the same. Generally speaking, there will be no problems, but there are still special cases: assuming that the port number of the new connection is the same as that of the old connection that has been closed, if some data of the previous connection is still stuck in the network, these delayed data are established. After the new connection arrives at host 2, since the port numbers of the new connection and the old connection are the same, the TCP protocol thinks that the delayed data belongs to the new connection, so that it is confused with the real data packet of the new connection. Therefore, the TCP connection has to wait for 2 times the MSL in the TIME_WAIT state, which can ensure that all data of this connection disappears from the network.
Nine, TCP flow control
If the sender sends the data too fast, the receiver may not receive it in time, which will cause data loss. The so-called flow control is to make the sender's sending rate not too fast, so that the receiver has time to receive.
The flow control of the sender can be easily implemented on the TCP connection by using the sliding window mechanism .
Let A send data to B. When the connection is established, B told A: "My receive window is rwnd = 400" (rwnd here means receiver window). Therefore, the sending window of the sender cannot exceed the value of the receiving window given by the receiver. Please note that the window unit of TCP is bytes, not segments. Assume that each segment is 100 bytes long, and the initial value of the sequence number of the data segment is set to 1. Uppercase ACK indicates the confirmation bit ACK in the header, and lowercase ack indicates the value ack of the confirmation field.
It can be seen from the figure that B performs flow control three times. Reduce the window to rwnd = 300 for the first time, reduce it to rwnd = 100 for the second time, and finally reduce it to rwnd = 0, that is, the sender is not allowed to send data. This state that causes the sender to suspend sending will continue until host B reissues a new window value. The three message segments sent by B to A all set ACK = 1, and the confirmation number field is meaningful only when ACK = 1.
TCP has a persistence timer (persistence timer) for each connection. As long as one side of the TCP connection receives a zero-window notification from the other side, the duration timer is started. If the time set by the persistence timer expires, a zero-window control detection message segment (carrying 1 byte of data) is sent, and the party receiving the message segment resets the persistence timer.
Ten, TCP congestion control
The sender maintains a state variable of the congestion window cwnd ( congestion window ). The size of the congestion window depends on the degree of network congestion and is changing dynamically. The sender makes his sending window equal to the congestion window.
The principle for the sender to control the congestion window is: as long as there is no congestion in the network, the congestion window will be increased to send more packets. But as long as the network is congested, the congestion window is reduced to reduce the number of packets injected into the network.
Slow start algorithm:
When the host starts to send data, if a large number of data bytes are injected into the network immediately, it may cause network congestion, because it is not clear what the network load is. Therefore, a better method is to detect first, that is, gradually increase the sending window from small to large, that is, gradually increase the value of the congestion window from small to large.
Usually, when the message segment is just started to be sent, the congestion window cwnd is first set to a value of the maximum message segment MSS. After receiving an acknowledgment for a new message segment, the congestion window is increased by at most one MSS value. Using this method to gradually increase the congestion window cwnd of the sender can make the rate at which packets are injected into the network more reasonable.
Every time a transmission round passes, the congestion window cwnd is doubled. The time elapsed in a transmission round is actually the round-trip time RTT . However, the "transmission round" is more emphasized: the message segments allowed to be sent by the congestion window cwnd are sent continuously, and the confirmation of the last byte sent has been received.
In addition, the "slow" of slow start does not refer to the slow growth rate of cwnd, but refers to setting cwnd=1 when TCP starts to send segments, so that the sender only sends one segment at the beginning (the purpose is to test Look at the congestion of the network), and then gradually increase cwnd.
In order to prevent network congestion caused by excessive growth of the congestion window cwnd, it is also necessary to set a slow start threshold ssthresh state variable. The slow start threshold ssthresh is used as follows:
When cwnd < ssthresh, the slow start algorithm described above is used.
When cwnd > ssthresh, stop using the slow start algorithm and use the congestion avoidance algorithm instead.
When cwnd = ssthresh, either the slow start algorithm or the congestion control avoidance algorithm can be used. congestion avoidance
congestion avoidance
Let the congestion window cwnd increase slowly, that is, every time a round-trip time RTT passes, the congestion window cwnd of the sender is increased by 1 instead of doubled. In this way, the congestion window cwnd grows slowly according to a linear law, which is much slower than the congestion window growth rate of the slow start algorithm.
Regardless of whether it is in the slow start phase or the congestion avoidance phase, as long as the sender judges that the network is congested (the basis is that no acknowledgment has been received), the slow start threshold ssthresh must be set to half of the sender window value when congestion occurs (but not less than 2). Then reset the congestion window cwnd to 1 and execute the slow start algorithm.
The purpose of doing this is to quickly reduce the number of packets sent by the host to the network, so that the congested router has enough time to process the backlog of packets in the queue.
The following figure illustrates the above congestion control process with specific values. The send window is now as large as the congestion window.
2. Fast retransmission and fast recovery
fast retransmission
The fast retransmission algorithm first requires the receiver to send a repeated confirmation immediately after receiving an out-of-sequence segment (in order to let the sender know that a segment has not reached the other party as soon as possible) instead of waiting until the data is sent by itself. confirm.
After receiving M1 and M2, the receiving party sends out confirmation respectively. Now assume that the receiver does not receive M3 but then receives M4.
Obviously, the receiver cannot confirm M4, because M4 is an out-of-sequence segment received. According to the principle of reliable transmission, the receiver can do nothing, or send an acknowledgment to M2 at an appropriate time.
However, according to the regulations of the fast retransmission algorithm, the receiver should send a repeated confirmation of M2 in time, so that the sender can know that the message segment M3 has not reached the receiver as early as possible. The sender then sends M5 and M6. After receiving these two messages, the receiver also needs to send out a repeated confirmation of M2 again. In this way, the sender has received a total of four acknowledgments for M2 from the receiver, and the last three are repeated acknowledgments.
The fast retransmission algorithm also stipulates that as long as the sender receives three repeated acknowledgments in a row, it should immediately retransmit the message segment M3 that has not been received by the other party, without continuing to wait for the retransmission timer set by M3 to expire.
Since the sender retransmits unacknowledged segments as soon as possible, the throughput of the entire network can be increased by about 20% after fast retransmission is adopted.
fast recovery
The fast recovery algorithm is also used in conjunction with fast retransmission. The process has the following two main points:
When the sender receives three repeated confirmations in a row, it executes the "multiplication reduction" algorithm to halve the slow start threshold ssthresh.
The difference from the slow start is that the slow start algorithm is not implemented now (that is, the congestion window cwnd is not set to 1 now), but the cwnd value is set to the value after the slow start threshold ssthresh is halved, and then the congestion avoidance algorithm is started (" Additive increase"), so that the congestion window grows slowly and linearly.