Table of contents
The format of the TCP protocol
foreword
The TCP protocol is a transport layer protocol and a very important protocol in the TCP/IP protocol stack. It provides features such as connection-oriented, reliable transmission, and byte stream-oriented. The efficiency and reliability of transmission are greatly improved. Reliability is the core mechanism of the TCP protocol. It can provide various control and error recovery mechanisms during data transmission to ensure reliable data transmission in the network.
TCP protocol
TCP, that is, Transmission Control Protocol, Transmission Control Protocol. As the name suggests, it is necessary to have a detailed control over the transmission of data.
The format of the TCP protocol
It can be seen that the packet format of TCP is still more complicated than that of UDP.
- Source port: It is consistent with the source port of UDP, identifying the process from which the data comes.
- Destination port: Identify the process to which the data should go.
- 32-bit sequence number: Used to identify the byte stream that data is sent from the sender to the receiver.
- 32-bit acknowledgment number: The acknowledgment number is only useful when the receiver returns an ack to the sender. Indicates that the receiver expects the sender to send the next numbered byte data.
- 4-bit header length: Occupies 4 bits, calculated in units of 4 bytes. Indicates how many 4-bytes the TCP header has. Identifies the length of the TCP header. The fixed length is 20 bytes.
- Reserved 6 digits: no discussion.
- 6 flag bits: occupying 6 bits, including URG, ACK, PSH, RST, SYN, and FIN flag bits, respectively indicating functions such as urgent data, confirmation, push data, reset connection, synchronous serial number, and end connection.
- 16-bit window size: Indicates the size of the amount of data that the receiver can receive.
- 16-bit checksum: used to check the integrity of the data, including the header and data part of TCP.
- 16-bit urgent pointer: When URG is 1, it indicates that part of the data is urgent data.
The above is the header structure of the TCP packet.
The following is a detailed introduction.
TCP principle
TCP's control mechanism for data transmission is mainly in two aspects, one is efficiency, but security, to ensure that the transmission efficiency is improved as much as possible under the premise of data transmission security.
1. Acknowledgment
The core mechanism to achieve reliability.
The TCP sender numbers each byte of data, which is the sequence number.
After receiving the data with the sequence number, the receiving end sends an ack message. The ack message contains the confirmation sequence number from the receiving end to the sender. The confirmation sequence number is the last byte sequence number in the received data. The next sequence number is to tell the sender that I have received the data you sent, where will you start sending next time.
In the network, due to network reasons, there may be situations where the last come first come.
In the data sent by the sender, because TCP numbers each byte, during the network transmission, some bytes may be delayed due to network instability, and other bytes have already arrived in TCP. If it is in the receiving buffer, the data will be disordered. However, due to the role of the TCP sequence number, the sequence number can be sorted in the receiving buffer to make the data orderly. In this way, the upper application layer will not read wrong data.
2. Timeout retransmission
If the network is stable and there is no error in the data, then the data will be sent and received smoothly, but if the network is sending data, because of a problem with a certain node, the data cannot be sent to the receiving end, it will A packet loss problem occurs.
After the packet is lost, the receiving end must not receive the data, and cannot send an ack confirmation message.
At this time, the sender is waiting for the confirmation message from the receiver. After waiting for a period of time, if it still does not receive the confirmation message from the receiver, the sender will regard the data sent just now as packet loss, and it will to resend.
If the sent data is received by the receiving end, but the ack packet sent by the receiving end is lost, the sender will not be able to wait for the ack message.
The sender also waits for a certain period of time, if there is still no ack message, then the sender thinks that the data packet is lost, and at this time the sender will send the previously sent data again.
In the above process, client B will receive duplicate data, but because of the serial number, the TCP protocol can help us automatically identify which are duplicate data and discard the duplicate data.
If multiple data packets are lost, TCP will continue to retransmit overtime, but every time a packet is lost, the retransmission time will become longer, that is to say, the frequency of retransmission will become lower.
If there is no ack response after multiple retransmissions, then TCP will try to reset the connection. If the reset connection fails, then TCP will disconnect at this time.
The reliability of TCP is realized by confirming the connection and retransmitting after timeout.
3. Connection management
Establishing a connection : three-way handshake
Handshake: The communication parties establish a network interaction, which is equivalent to three interactions between the client and the server to establish a connection relationship with each other (both parties record each other's information).
syn is a synchronous message segment: its essence is to apply to establish a connection with the other party.
ack is a confirmation message segment.
Client A is sending a syn to client B to indicate that it wants to establish a connection with client B. At this time, client B also wants to establish a connection with client A, so client B sends a syn+ack message to client A. This text means that I agree to establish a connection with you, but you also have to establish a connection with me. At this time, after receiving it, client A readily agrees, so it sends an ack confirmation message.
So far, client A and client B have formally established a connection after the above-mentioned three-way handshake.
In our TCP header, there are these 6 special bits, which are 0 by default. When the SYN field is 1, it means a SYN message. When the ACK field is 1, it means an ACK message. When both of them are 1, it means a syn+ack message.
So why do you need a three-way handshake?
The process of the three-way handshake is actually to check whether the sending and receiving capabilities of the communication parties are normal, and it is the basis for subsequent reliable data transmission.
Disconnect : four waves
The four-way wave and the three-way handshake are very similar.
The two parties in the communication each send a FIN (end message) to the other party, and then return an ack message to the other party.
In the three-way handshake, syn and ack are combined into one data packet for sending, but in the four-way handshake, usually they cannot be combined.
Because the syn and ack of the three-way handshake are triggered at the same time (triggered by the system kernel).
Wave your hands four times. Here, the triggering timing of FIN and ACK is different. ACK is triggered by the system kernel, and the ACK message is returned as soon as the FIN message is received. However, FIN is controlled by the application code, and when the close method of the socket is called, a FIN message will be triggered.
The inability to merge is due to the logic of your code. After client A is found to be disconnected, client B immediately performs a close operation, which triggers the second FIN. If this is an ACK message, it is still If it is not sent out, then it can be merged. If the close operation is not performed immediately after the other party is found to be disconnected, if it is a long time in the middle, then the merge cannot be performed.
It should be noted that the connection is maintained by the system kernel. If a process ends, the kernel will still maintain the TCP connection until four waves are completed.
A FIN message means that when the FIN field is 1, it means that the message is a FIN message.
4. Sliding window
Sliding window, also called bulk transfer
The above-mentioned acknowledgment response mechanism is to wait for an ack confirmation message for each datagram sent, and send the next datagram after receiving the ack confirmation message, but in this case, the performance is poor.
The sliding window is to send multiple datagrams at a time, which can greatly improve performance.
The batch is not sent indefinitely, it is sent to a certain extent, just wait for ack, the datagram that can be sent without waiting for ack is limited, this limit is the window size.
And after the batch sending is completed, if an ack is returned, the next one will be sent immediately.
In the above figure, 4 pieces of data are sent in batches. After sending, wait for the ack. Note that after each ack is received, the next piece of data is sent, instead of sending the next set after receiving 4 acks.
The above batch transfer process is the sliding window.
The size of the window is the maximum number of datagrams that can be sent without waiting for an ack. The picture above is 400 bytes, 4 segments.
When sending the first 4 segments, the data can be sent directly without waiting for the ack. After receiving the first ack, the window slides backwards and continues to send the fifth data segment. In order to maintain this window, the operating system needs to create a sending buffer to record which data is currently unanswered. Only the data that has been answered can be deleted from the sending buffer.
In Figure 1, 4 pieces of data are sent in batches, and the white area in Figure 1 is equivalent to waiting for the ack response.
In Figure 2, when host B returns an ack to host A, it means that 1001-2000 has been received, and then the data of 5001-6000 will be sent.
It can be seen that the window has moved backward by one grid. If the received ack message is very fast at this time, the window will move backward quickly. This is the origin of the sliding window.
If packet loss occurs during batch sending, the sliding window processing mechanism is also very flexible.
The TCP protocol must first ensure reliability, followed by efficiency.
If packet loss occurs, there are two situations:
Data is lost during transmission
It can be seen that the data 1001-2000 is lost during the sending process, and the receiver will always ask the sender for the data of 1001-2000. When the sender receives three consecutive acks asking for 1001, the sender will respond to 1001 If -2000 is lost, the data of 1001-2000 will be resent. When the sender sends the data of 1001-2000, the confirmation ack returned by the receiver is 7001 instead of 2001, because the data before 7001 has already been sent. Sent over, all in the receiver's receive buffer.
The above-mentioned retransmission process does not have any redundancy, only data that has not been received is sent, and there will be no repeated sending. The overall speed is relatively fast. Also known as fast retransmission .
The data has arrived, but the confirmation ack is lost
In fact, there is no need to worry about this situation in TCP. Because of the confirmation sequence number, the latter ack can cover the previous ack. That is to say, when the ack of 2001 is received, the sender already knows that the data between 2001 and 2001 is successful. received.
If the last ack is lost, there is nothing wrong with it. At this time, TCP's timeout retransmission will work. After waiting for a period of time, if no ack message appears, the last data will be resent.
5. Flow control
Flow control is also a security mechanism in TCP.
The data that the receiving end can accept is limited. If the sending end sends too fast, causing the receiving buffer of the receiving end to be filled in an instant, the receiving end will directly discard the data sent later.
Flow control is to solve the above problems by controlling the sending speed of the sender.
In the message carried by the ack, there is a 16-bit window size field, and the value here is the recommended window size for the sender to send.
The receiving side calculates the window size is very simple and crude, it directly takes the size of the remaining receiving buffer as the window size.
As can be seen from the above figure, when the sender receives the receiver's ack for the first time, the receiver's window size is already carried in the ack message, that is to say, the receiver's remaining receiving buffer size It is 3000, so the sender will send in batches according to this size. When the size of the buffer sent to the receiver is 0, it will stop sending, because the receiving buffer of the receiver is full. At this time, the sender will Time to send a window detection message, if the detection will find that the remaining space of the receiver is not 0, you can continue to send at this time. When the remaining space at the receiving end is freed up, it will also send a window update notification to notify the sender to send according to the window size.
6. Congestion control
The size of the sliding window depends on flow control and congestion control.
Flow control measures the processing power of the receiving end. Congestion control measures the processing power of the transmission path.
If the network is currently congested, if a large amount of data is sent rashly at this time, it will cause simultaneous loss of data and network.
Because there are many nodes on the network, congestion control is to measure the intermediate path, how many nodes are on the intermediate path, and the current situation of each node.
TCP introduces a slow start mechanism, which sends a small amount of data first, explores the path, finds out the current network congestion state, and then decides how fast to transmit data.
At the beginning, send at a very small sending rate. If there is no packet loss, increase the sending rate (window size), and immediately reduce the rate if packet loss occurs.
Then repeat the above process.
It can be seen that the growth rate of the above-mentioned congestion window is very fast. is exponential growth.
It can be seen that the transmission is sent at a very small threshold at the beginning, and then at an exponentially increasing speed, allowing the window size to reach a relatively large value in a short time.
When the exponential growth reaches a certain threshold ssthresh, it will become a linear growth to avoid congestion.
When the growth reaches a certain level and network congestion occurs, it is considered that the window size at this time has reached the upper limit of the current network transmission, and then the window size is immediately adjusted to a smaller initial value, and then the above process is repeated.
7. Delayed response
The core of ensuring TCP reliability is the acknowledgment response.
The ack must be sent, but the ack message is not sent immediately, but will be sent after a while, because in TCP, the sending efficiency is determined by the window size, and the window size is the remaining size of the receiver's receiving buffer. If it is waiting to reply to the ack message, then the application may consume some data at this time. After the data is consumed, the consumed data will be deleted in the receiving buffer, so that the size of the receiving buffer will become larger, and the window The size will also increase. More data can be sent next time.
The effect of delayed response is to delay sending ack messages, so that the receiver's application is more likely to consume more data, and the returned window size will be larger at this time. The sender will also send a little faster. At the same time, it can also satisfy the receiver to handle it.
8. Piggybacking
Based on delayed response.
The communication mode between client and server is in the form of a question and an answer.
Based on the delayed response, we found that in many cases, the client server also "sends and receives" at the application layer. It means that the client said "How are you" to the server, and the server will also return a "Fine, thank you" to the client; then at this
time, the ACK can hitch a ride, together with the server's response of "Fine, thank you" It is more efficient to send back to the client in one package than in two.
9. Byte stream oriented
sticky bag problem
When the sending end sends multiple data to the receiving end, all these data are in the receiving buffer of the receiving end, close together. At this time, when the receiving end reads the data, it is difficult to distinguish from where to where is a complete data, it will lead to the situation of reading half a packet/one and a half packets.
At this point we can define a delimiter to distinguish packets.
package netWork.netWork02;
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.io.PrintWriter;
import java.net.ServerSocket;
import java.net.Socket;
import java.util.Scanner;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
public class TcpEchoServer {
//serverSocket 就是在外场拉客的
//ClientSocket 就是服务于拉来的客人的
//serverSocket 只有一个，但是clientSocket会给每个客户端都分配一个
private ServerSocket serverSocket = null;
public TcpEchoServer(int port) throws IOException {
serverSocket = new ServerSocket(port); //进行端口号绑定
}
public void start() throws IOException {
//还可以使用线程池的方式
ExecutorService executorService = Executors.newCachedThreadPool();
System.out.println("服务器启动!!!");
while (true) {
//accept() 接收一个连接 clientSocket 针对具体的客户端进行服务 通过这个clientSocket和客户端进行通信
Socket clientSocket = serverSocket.accept(); //accept是会阻塞的，如果没有客户端连接的话，就会阻塞
//频繁的创建和销毁线程对系统资源的消耗也是很大的
/*如果直接调用这个方法，会影响这个循环的二次执行。到时accept就不及时了，
采用创建新的线程来调用processConnection这个方法
每次来一个新的客户端都创建一个新的线程
主线程就是while循环，只做两件事，accept 和创建线程，当线程创建好了之后，就下一次调用accept，
刚刚创建好的线程，去处理请求
处理连接和处理请求之间是并发的，没有关联关系*/
/* Thread t = new Thread(()->{
try {
processConnection(clientSocket);
} catch (IOException e) {
e.printStackTrace();
}
});
t.start();*///这个循环和线程中的任务就会并发的执行
//使用线程池的方式
//一个连接的所有请求处理完，这个线程不会立即销毁，而是放在线程池里面，下次直接使用。
executorService.submit(new Runnable() {
@Override
public void run() {
try {
processConnection(clientSocket);
} catch (IOException e) {
e.printStackTrace();
}
}
});
}
}
//通过这个方法来处理一个连接
//读取请求
//根据请求计算响应
//把响应返回给客户端
private void processConnection(Socket clientSocket) throws IOException {
System.out.printf("[%s:%d] 客户端上线\n",clientSocket.getInetAddress().toString(),
clientSocket.getPort());
//通过clientSocket拿到一对Stream对象，inputStream 输入，从网卡读
//outputStream 输出，往网卡写
try(InputStream inputStream = clientSocket.getInputStream();
OutputStream outputStream = clientSocket.getOutputStream()){
//把inputStream和outputStream进行包装
Scanner scanner = new Scanner(inputStream);
PrintWriter printWriter = new PrintWriter(outputStream);
while (true) {
// 1 读取请求
if(!scanner.hasNext()) {
System.out.printf("[%s:%d] 客户端下线\n",clientSocket.getInetAddress().toString(),
clientSocket.getPort());
break; //读取的流到结尾了，没有数据了，（对端关闭连接了）
}
//直接使用scanner来读取一段字符串 使用换行来区分数据包
String request = scanner.next();
// 2 根据请求计算响应
String response = process(request);
// 3 把响应写回客户端 响应也是要带上换行的
printWriter.println(response);
printWriter.flush(); //刷新缓存区 冲刷 数据立即写入网卡中
System.out.printf("[%s:%d] req: %s,resp: %s\n",clientSocket.getInetAddress().toString(),
clientSocket.getPort(),request,response);
}
} catch (IOException e) {
e.printStackTrace();
} finally {
clientSocket.close(); //只给一个连接提供服务的，时刻都有新的连接，所有要释放
}
}
public String process(String request) {
return request;
}
public static void main(String[] args) throws IOException {
TcpEchoServer tcpEchoServer = new TcpEchoServer(9090);
tcpEchoServer.start();
}
}
It can be seen that in the above TCP server code, we use separators to distinguish.
You can also agree on the length to distinguish data packets.
Both of these solutions are defined and implemented in our code.
10. Abnormal situations
Process termination: process termination will release the file descriptor, and FIN can still be sent. It is no different from a normal shutdown. Connections are maintained in the operating system kernel.
Machine restart: same as process termination.
The machine is powered off/network cable is disconnected: the receiving end thinks that the connection is still there, once the receiving end has a write operation, and the receiving end finds that the connection is no longer there, it will reset. Even if there is no write operation, TCP itself has a built-in keep-alive timer, which will periodically ask whether the other party is still there. If the other party is not there, the connection will also be released.
Summarize
The reason why TCP is so complicated is that TCP wants to improve transmission reliability and ensure transmission efficiency as much as possible.
reliability:
- checksum
- Serial number (arrives sequentially)
- confirmation response
- timeout resend
- connection management
- flow control
- congestion control
Improve efficiency:
- sliding window
- fast retransmit
- delayed response
- piggybacking