[Computer Network Notes 3] Transport Layer

port

How to mark a process on the network?

• The transport layer of the TCP/IP system uses [port number] to mark different application processes in the application layer.
• The port mentioned here is a logical concept, not an actual physical port.
  
  

The port number is represented by 16 bits, the value range is0 ~ 65535 , and the port number is divided into the following three categories:

• ① Well-known port number (for server)
• ② Register port number (for server)
• ③ Short-lived port number (for client)

Well-known port numbers

Well-known port numbers or system port numbers, numerically 0 ~ 1023assigned by IANA to some of the most important TCP/IP applications so that all users know them .

When a new application appears, IANA must assign it a well-known port, otherwise other application processes on the Internet will not be able to communicate with it.

Register port number

Register a port number with a value of 1024 ~ 49151. This type of port number is used by applications that do not have well-known port numbers .

The use of such port numbers must be registered with IANA in accordance with the prescribed procedures to prevent duplication, such as the default port number of Oracle1521 , the default port number of MySQL3306 , and the default port number of tomcat in Java WEB8080

ephemeral port number

The port number used by the client , the value is 49152 ~ 65535.

Because this type of port number is dynamically selected only when the client process is running , it is also called an ephemeral port number . This type of port number is reserved for client processes to choose for temporary use. In theory, these ports should not be assigned to the server, in practice, the machine is usually 1024assigned dynamic ports from the beginning.

When the server process receives the message from the client process, it knows the port number used by the client process, so it can send the data to the client process.

After the communication is completed, the client port number just used no longer exists, and this port number can be reused by other client processes .

transport layer protocol

• TCP : Transmission Control Protocol
• UDP : User Datagram Protocol ( User Datagram Protocol )
  

Each protocol is designed to solve a specific problem:

• CSMA/CD protocol: coordinates the work of computers on the bus
• ARP protocol: Obtain the MAC address of the corresponding host network card based on the IP
• IP protocol: Solving the interconnection problems of multiple heterogeneous networks
• ICMP protocol: To forward IP datagrams more efficiently and improve the chance of delivery success
• TCP protocol: Provides reliable end-to-end data transmission services
• UDP protocol: Provides unreliable but efficient transmission services

• RIP DNS TFTP SNMP DHCP at the application layer corresponds to the UDP protocol at the transport layer .
• SMTP FTP BGP HTTP HTTPS at the application layer corresponds to the TCP protocol at the transport layer
• In the IP datagram of the network layer , there is a protocol field. The value is used to indicate the transport layer protocol used by the current message . It is a number.

UDP VS TCP

UDP header:

• (1) Source port : Source port number. Use it when you need a reply from the other party. Use all 0s when not needed
• (2) Destination port : Destination port number. This must be used when delivering messages at the endpoint.
• (3) Length : The length of UDP user datagram, its minimum value is 8 (only the header).

TCP header:

The TCP header includes source port, destination port, sequence number (seq), acknowledgment number (ack), etc.

• UDP is connectionless and TCP is connection-oriented.

• UDP supports unicast, multicast, and broadcast, while TCP only supports unicast.

• UDP is oriented to application packets, and neither merges nor splits packets; TCP is oriented to byte streams, which is also the basis for TCP to achieve reliable transmission, flow control, and congestion control.

• The sender's TCP regards the data delivered by the application layer as a series of unstructured byte streams . TCP first stores the bytes in its own cache. According to the sending strategy, a part of the bytes is sent first; the receiver extracts the bytes. section and stored in its own cache; the receiving application process must be able to identify the received byte stream and restore it into meaningful application data.

• UDP provides connectionless and unreliable transmission services to the upper layer (suitable for real-time applications such as IP telephony and video conferencing), while TCP provides connection-oriented and reliable transmission services to the upper layer (suitable for applications requiring reliable transmission, such as file transfer).

Summarize:

TCP header

Implementation of TCP reliable transmission:

• TCP reliable transmission is implemented using the Selective Retransmission Protocol (SR) ( two sliding windows: sending window + receiving window )
• TCP's sliding window is measured in bytes
• The fields related to reliable transmission in the TCP header are as follows:
  

Serial number ( seq) field: occupies 4bytes

• That is to say, 32 bits are used for numbering, and the serial number size range [0, 2^32 - 1]is a total 2^32of serial numbers (that is, 4 294 967 296).

• When the sequence number increases 2^32 - 1to , the next sequence number returns 0.

• TCP is byte stream oriented , and each byte in the byte stream transmitted in a TCP connection is numbered sequentially .

• The starting sequence number of the entire byte stream to be transmitted must be set when the connection is established .

• seqThe field value in the header refers to the sequence number of the first byte of the data sent in this segment .

Confirmation number ( ack) field: occupies 4bytes

• Indicates the sequence number of the first data byte expected to be received from the other party's next message segment.
  
• If the confirmation number = N, it means that all data up to sequence number N - 1 have been received correctly.

ACK field (Acknowledgement, confirmation):

• The acknowledgment number field is valid only when ACK=1.
• When ACK = 0, the confirmation number is invalid.
• TCP stipulates that all transmitted message segments must have ACK set to 1 after the connection is established.

Window field: occupies 2bytes

• The window value is [0, 2^16 - 1]an integer between.

• The window refers to the receiving window of the party sending this segment (not its own sending window).

• The window value tells the other party: starting from the confirmation number in the header of this message segment ack, the amount of data the receiver currently allows the other party to send. The reason for this restriction is that the receiver's data cache space is limited. (Simple understanding: When I send you data, I will tell you in the data how big my receiving buffer is, and when you reply to me with data, you will not let me know that it is beyond the range)

• In short, the window value serves as the basis for the receiver to let the sender set its sending window. The window value is often dynamically changing.

TCP three-way handshake

First handshake: client → server

• SYN = 1, indicating that this is a TCP connection request message

• The sequence number field seqis set to an initial value as the initial sequence numberx selected by the TCP client process

• TCP regulations: SYN = 1A message segment cannot carry data, but a sequence number must be consumed

Second handshake: server → client

• SYN = 1，ACK = 1, indicating that this is a confirmation message for a TCP connection request
• The sequence number field seqis set to an initial value as the initial sequence numbery selected by the TCP service process (TCP is full-duplex communication, and both the client and the server can receive and receive data)
• ack = x + 1, which is a confirmation of the initial sequence number selected by the TCP client process

The third handshake: client → server

• ACK = 1, indicating that this is an ordinary confirmation message segment
• seq = x + 1, is added as the current serial number value xon the basis of the initial serial number selected last time.1
• ack = y + 1, which is the confirmation of the initial sequence number of the server.

After the three-way handshake, the TCP connection between the client and the server has been established, and data can be sent to each other:

Why do we need to send a normal confirmation message in the end?

The use of a three-packet handshake instead of a two-packet handshake to establish a TCP connection is to prevent the expired connection request segment from suddenly arriving at the TCP server. At this time, the server will send a confirmation request message to the client. , and the client cannot respond to the server because it is closed, the server will continue to send confirmation request messages to the client because it does not know that the client is closed, thus wasting server resources.

Three-way handshake summary

• The first handshake: the client sends SYN = 1 seq = x to the server.

  SYN = 1 indicates that this is a TCP connection request message

  seq = x means that the client process uses sequence number x as the initial sequence number

  TCP regulations: The message set to cannot carry data, but will consume a sequence number SYN.1

• Second handshake: The server sends SYN = 1 ACK = 1 seq = y ack = x + 1 to the client.

  SYN = 1 ACK = 1 indicates that the current message is a confirmation message from the server to the previous connection request message sent by the client.

  ack = x + 1 confirms the last starting sequence number x of the client process , and expects the starting sequence number of the next message sent by the client to be x + 1

  seq = y means that the server process selects y as the initial sequence number

  Both TCP and UDP are full-duplex communications, and both the client and the server can send and receive data.

• The third handshake: the client sends ACK = 1 seq = x + 1 ack = y + 1 to the server.

  ACK = 1 means the current message is a normal acknowledgment message for the last time

  seq = x + 1 means that the current message chooses to use the last initial sequence number xplus 1, which is the sequence number expected by the server.

  ack = y + 1 means confirming the sequence number y of the last message sent by the server , and expecting the starting sequence number of the next message sent by the server to be y + 1.

Understanding the human version:

• First time (our side): I request ( SYN = 1) to currently use the starting sequence number x( seq = x)
• The second time (the other party): I confirm receipt of your request ( ), you should start SYN = 1 ACK = 1from the serial number next time ( ), I request to use the starting serial number currently ( )x + 1ack = x + 1yseq = y
• The third time (our side): I confirm receipt of your reply ( ), you should start ACK = 1from the serial number next time ( ), I currently use the serial number you expect me to send ( )y + 1ack = y + 1x + 1seq = x + 1
  

Understanding of the saucy version:

Understanding of the mutually harmful version:

Why does a normal confirmation message need to be sent at the end/Why does TCP need a three-way handshake instead of a two-way handshake?

• TCP is a reliable transmission control protocol, and the three-way handshake is the minimum number of times to ensure reliable data transmission and improve transmission efficiency.

Why? RFC793, which is the TCP protocol, talks about the reason in RFC. This is because:

1. In order to achieve reliable data transmission, both parties communicating with the TCP protocol must maintain a sequence number to identify which of the data packets sent have been received by the other party.

2. The three-way handshake process is a necessary step for both communicating parties to inform each other of the starting value of the sequence number and confirm that the other party has received the starting value of the sequence number. If there are only two handshakes, at most only the starting sequence number of the connection initiator can be confirmed, but the sequence number selected by the other party cannot be confirmed. As for why it is not four times, it is obvious that after the three-way handshake, both parties in the communication already know the starting value of the other party's serial number, and have also confirmed that the other party knows the starting value of their own serial number. The fourth handshake is no longer necessary.

3. In order to prevent the connection request message that has expired (such as timeout retransmission) from suddenly arriving at the server, wasting server resources. If a two-way handshake is used, the server confirms that the connection has been established after the first request. At this time, if the client closes the connection and the server receives the client's previously expired TCP connection request, it will send the request to the client. Send a confirmation request message. At this time, the client cannot respond to the server because it is closed. The server will continue to send confirmation request messages to the client because it does not know that the client is closed, resulting in waste. resources .

TCP four waves

TCP release connection process

First wave:

• FIN = 1, indicating a TCP release connection request message
• TCP stipulates that even if a FIN segment does not carry data, it consumes a sequence number.

The second wave:

According to the TCP standard, the previously sent FIN segment consumes a sequence number, and the connection in the direction from A to B is released. At this time, the TCP connection is in a half- closed state, that is, A has no data requirements. Sent, but if B sends data, A still needs to receive it . In other words, the connection in the direction from B to A is not closed , and this state may last for some time.

The third wave:

The fourth wave:

• Note: Only the party that initiates the connection termination will enter the TIME_WAIT state

Is it necessary to wait for 2MSL?

• If the message sent by the client to confirm the server's connection termination request is lost and does not reach the server, it will cause the server to repeatedly send connection termination requests to the client due to delays in getting confirmation, resulting in The server has been unable to enter the CLOSED state.

What happens if the server process hangs after the connection is established?

• If the server process hangs after the connection is established, a FIN packet will be sent to the client, and the client will reply with an ACK packet. At this time, if the client continues to write data to the server, the server will reply with an RST packet and then terminate.
  
  

• If the server process hangs up due to a power outage, the client will continue to write data to the server and will wait for no response until it times out. The client process needs to set a timeout.
  
  

• In the same way, if the client process hangs, the response of the server process continuing to send data to the client is similar to the above, with some adjustments.
  
  
  

TCP connection management—keep-alive timer

1. Every time the TCP server process receives data from the TCP client process, it resets and starts the keep-alive timer (2-hour timer)

2. If no data is received from the TCP client process within the keep-alive timer period, the TCP server process will send a probe segment to the TCP client process when the keep-alive timer expires.

3. In the future 75, it will be sent every seconds. If there is still no response from the TCP client process after sending 10detection segments in succession, the TCP server process will think that the host where the TCP client process is located is faulty, and then close the connection.

To put it bluntly, the TCP keep-alive timer is a termination mechanism that waits for timeout .

Summary of four waves

• The first time: the client sends FIN = 1 ACK = 1 seq = u ack = v to the server. FIN = 1 indicates that this is a TCP request message to release the connection.

  TCP regulations: Even if a FIN segment does not carry data, it consumes a sequence number.

• The second time: the server sends ACK = 1 seq = v ack = u + 1 to the client. Indicates confirmation of the client's first release connection request message.

  At this time, the connection in the A → B direction is released, and TCP is in a semi-closed state . Although B confirms that A will no longer send data, B may still send data to A.

• The third time: The server sends FIN = 1 ACK = 1 seq = w ack = u + 1 to the client, indicating that the server releases the connection request message.

• The fourth time: the client sends ACK = 1 seq = u + 1 ack = w + 1 to the server.

  At this time, A also confirms the closed state of B. B directly enters the closed state. A enters the TIME_WAIT waiting state. The waiting time is 2MSL, which is twice the longest message life cycle. If no message from the other party is received during this period, A finally enters the closed state .

In one sentence, both parties must initiate a connection termination request, and both parties must confirm the connection termination request sent by the other party.

Both the client and the server can initiate a FIN disconnection request (TCP is full-duplex ), but only the party initiating connection termination will enter the TIME_WAIT waiting state

In practice, the second and third times are combined into one message for transmission.

Understanding of the four-time breakup version:

Why does TCP's wave take four times?

• TCP is a full-duplex connection, and both ends must close the connection at the same time for the connection to be truly closed .

  If A is ready to close writing, it can still read the data sent by B. At this time, A sends FINa message to B. After receiving it, B replies ACKthe message to A.

  When B is also ready to close writing, it sends FINa message to A, and A replies ACKto B. At this time, both ends are closed, and the TCP connection is closed normally.

  Therefore, for the full-duplex mode, in order to completely shut down, four interactions between the two ends of the communication are required to confirm each other's shutdown status .

Why should the initiator wait for 2MSL? / Why does the TIME_WAIT state exist?

1. Ensures reliable termination of TCP connections .

   FINIf the confirmation message sent by the client to the server for the fourth time is lost, the server will time out and retransmit the message to the client because it has not received the confirmation message .

   However, if the client is closed at this time, it will not be able to respond to the server, and the server will keep retransmitting FIN messages, which means that the server will never get confirmation and will eventually be unable to enter the CLOSED state .

2. Ensure that late TCP packets have enough time to be recognized and discarded .

   In Linux, a TCP port cannot be opened multiple times at the same time. When a TCP connection is in the state TIME_WAIT, we cannot use the connection's port to establish a new connection.

   Thinking about it the other way around, if there is no TIME_WAITstate, the application can immediately establish a connection similar to the one just closed (similar here means that they have the same IP address and port number). This new connection that is similar to the original connection is called the incarnation of the original connection. The new incarnation may receive TCP segments (late segments) belonging to the original connection carrying application data, which is obviously not supposed to happen. This is TIME_WAITthe second reason why states exist.

What will happen if the TCP process on one end hangs after the connection is established?

• If the server process hangs, a packet will be sent to the client FIN, and the client will reply with a ACKpacket. At this time, if the client continues to write data to the server, the server will reply with a RSTpacket and then terminate.

• If the server process hangs up due to a power outage, the client will continue to write data to the server and will wait for no response until it times out. The client process needs to set a timeout.

• In the same way, if the client process hangs, the response of the server process continuing to send data to the client is similar to the above, but it is reversed.

• How the server detects that the client is down: through the TCP keep-alive timer, similar to the timeout waiting + heartbeat mechanism.

TCP sticky/unpacking

TCP processes data in a stream manner. A complete packet may be split into multiple packets for transmission by TCP, or a small packet may be encapsulated into a large data packet for transmission.

Sticky bag

Sticky packets are multiple messages received at one time, and the application process cannot parse the data from a sticky packet.

Reasons for sticky packages:

• ① Each time the sender writes data < kernel buffer size

• ② The receiver does not read the kernel buffer in time enough

half pack

A half-packet means that a message is received multiple times, and the application process cannot parse the data from a half-packet.

Reasons for half-package :

• ① Sender writes data > kernel buffer size ,
• ② The sender’s data is larger than MTU (default 1500bytes of Ethernet large transmission unit) and must be unpacked

solution

The root cause of sticky packets and unpacking: TCP is byte stream-oriented, messages have no boundaries , and the application process cannot parse data from a sticky packet or half packet. But this is not a problem with TCP itself, but a problem that the upper application layer needs to deal with.

How does the application process interpret the byte stream? That is, how to solve the problems of sticky bags and half bags?

The fundamental solution to packet sticking and unpacking: determine the boundaries of the message

• ① Fixed length
• ② Separator
• ③ The fixed length field stores the length information of the content

The first option: fixed length

That is, fixed length bytes are sent each time. For example, each 3byte represents a message:

This solution is simple, but the disadvantage is that it wastes space. For example, it is stipulated that 10each byte represents a message, but the message sent by the client only contains 1n bytes, then the remaining 9n bytes need to be filled in or filled out 0, which is a waste.

Second option: separator

For example, use the carriage return character ( \n) as the separator:

For another example , carriage return and line feed characters are used in the HTTP header as the boundaries of the HTTP protocol:

This solution is simple and does not waste space. However, the disadvantage is that if the delimiter appears in the message content itself, it needs to be escaped, so the content needs to be scanned in advance.

The third solution: fixed length field stores the length information of the content

That is, a fixed-length field is appended to the message header to store the length of the message content . For example, a fixed-length field is used in the header of each message 4to represent the length of the message content :

When the receiver parses, it also first reads the fixed-length field to obtain the length, and then reads the data content based on the length to accurately locate the data content without escaping.

Disadvantages: The data content length is limited, and the number of bytes of the longest message needs to be known in advance.

Generally this solution is recommended, but it also depends on the scenario. Sometimes the first and second options are simpler.

Understand TCP's byte stream orientation and network byte order

Understand TCP's byte stream orientation

When the application process calls write()a method to socketsend data, the data is not actually sent from the network, but is only copied from the application to the kernel buffer. As for when it is actually sent, it depends on the sending window, congestion window and the current sending buffer. The size of the area and other conditions.

Moreover, not every write()data sent by a call will be sent completely as a whole. How many times the data will be sent out is also uncertain.

However, the order of bytes read by the TCP receiving end is guaranteed to be consistent with the order of bytes written when sending:

When TCP sends a piece of data, it may call write multiple times to send small data packets:

So when are these two data packets sent to the network? This depends on the needs of the receiving end.

• ① If the receiving end does not allow delay , then the sending end should send the data packet every time it calls write , regardless of the size of the data packet.
• ② If the receiving end allows a certain period of delay , the sending end can wait for a certain period of time and send multiple small data packets together, which can reduce the number of packets in the network.

network byte order

How does TCP decide which byte to send first and which byte to send last when sending a byte stream?

First understand what the byte order of the host is:

• Big endian (big endian): Bytes are stored in order from low memory addresses to high memory addresses .
• Little endian: Bytes are stored in order from high memory addresses to low memory addresses.
  

It can be seen that big endian order is more in line with human reading intuition (reading from left to right, storing from low to high), but for computers, little endian order is often easier to process.

In addition, different operating systems run on different CPUs, and the byte order is also different, see the table below.

processor	operating system	Byte ordering
Alpha	all	Little endian
HP-PA	NT	Little endian
HP-PA	UNIX	Big endian
Intelx86	all	Little endian <-----x86 system is a little endian byte order system
Motorola680x()	all	Big endian
MIPS	NT	Little endian
MIPS	UNIX	Big endian
PowerPC	NT	Little endian
PowerPC	Non-NT	Big endian ＜-----The PPC system is a big endian byte order system
RS/6000	UNIX	Big endian
SPARC	UNIX	Big endian
IXP1200 ARM core	all	Little endian

The network protocol uses big-endian order to send bytes.

If the sending host uses little-endian order, you need to convert from little-endian to big-endian before sending the data. Similarly, if the receiving host uses little-endian, you need to convert from big-endian to little-endian after reading the data. sequence transformation. The Linux system provides some functions that can be called directly to facilitate conversion between host byte order and network byte order:

When using these functions, we do not need to care about the byte order of the host. We only need to use the given value of the function to convert the network byte order and the host byte order.

If you send characters, you don't need to consider byte order conversion, because each character only occupies 1 byte.