Table of contents
1. Comparison of characteristics of UDP and TCP
1. With connection and without connection
2. Reliable transmission and unreliable transmission
3. Byte stream-oriented and datagram-oriented
4. Full duplex and half duplex
Implementation of translation server
Network programming is actually writing an application so that the program can use network communication
Here you need to call the API provided by the transport layer
There are two protocols provided by the transport layer:
1、UDP
2、TCP
These two protocols provide two different sets of APIs
1. Comparison of characteristics of UDP and TCP
UDP: connectionless, unreliable transmission, datagram oriented, full duplex
TCP: connection, reliable transmission, byte stream oriented, full duplex
1. With connection and without connection
When we were learning JDBC before, we had this step:
First create a DtaSourse, and then create a Connection through DataSourse
And the Connection here is what we call connection
A more intuitive understanding:
When making a call, dial a number and press the dial key. The connection establishment is completed only when the other party is connected.
When programming TCP, there is also a similar process of establishing a connection.
No connection is similar to sending WeChat/SMS, and communication can be carried out without establishing a connection.
The "connection" here is an abstract concept
Between the client and the server, memory is used to save information about the other end.
Both parties save this information, and "Connect" appears.
One client can connect to multiple servers, and one server can also correspond to the connections of multiple clients.
2. Reliable transmission and unreliable transmission
Reliable transmission does not mean that the message sent by A to B can be transmitted 100% of the time.
Instead, A tries its best to transmit the message to B, and when the transmission fails, A can sense it, or when the transmission is successful, it can also know that its transmission was successful.
TCP is reliable transmission, but the transmission efficiency is low
UDP is unreliable transmission and has higher transmission efficiency
Although TCP is reliable transmission and UDP is unreliable transmission, it cannot be said that TCP is more secure than UDP.
"Network security" refers to whether the data you transmit is easily intercepted by hackers, and whether some important information will be leaked if intercepted.
3. Byte stream-oriented and datagram-oriented
TCP is similar to file operations, both are "stream" (since the units of transmission here are bytes, it is called a byte stream)
UDP is datagram-oriented, and the basic unit of reading and writing is a UDP datagram (containing a series of data/attributes)
4. Full duplex and half duplex
Full duplex: One channel that can communicate in both directions
Half-duplex: A channel that can only communicate in one direction
The network cable is full duplex.
There are a total of 8 copper cables in the network cable, a group of 4 4, some are responsible for one direction, and some are responsible for the other direction.
2. UDP socket.api
1、DatagramSocket
is a Socket object,
The operating system uses the concept of files to manage some software and hardware resources. The operating system also uses files to manage network cards. Files representing network cards are called Socket files.
The Socket object in Java corresponds to the Socket file in the system (it will eventually fall on the network card)
For network communication, you must have a Socket object
Construction method:
The first one is often used on the client (which port the client uses is automatically assigned by the system)
The second one is often used on the server side (which port the server uses is manually specified)
A client's host has a lot of running results. God knows if the port you manually selected is occupied by other programs, so it is a wiser choice to let the system automatically allocate a port.
The server is completely controlled by the programmer. The programmer can arrange multiple programs on the server and let them use different ports.
Other methods:
2、DatagramPacket
Represents a UDP datagram, representing the binary structure of the UDP datagram set in the system.
Construction method:
The first constructor: used to accept data
DatagramPacket, as a UDP datagram, must be able to carry some data
Use manually specified byte[] as the space to store data
The second construction method: used to send data
SocketAddress address refers to ip and port number
Other methods:
getData refers to getting the UDP datagram payload part (complete application layer datagram)
Implementation of echo server
Next we start handwriting the UDP client server
The simplest UDP server: echo server, what the client sends, what the server returns
(1) Server code
When writing network programs, we often see this kind of exception, which means that the socket may fail.
The most typical situation is that the port number is occupied
The port number is used to distinguish applications on the host. One application can occupy multiple ports on the host, and one port can only be occupied by one process (there are special cases, which will not be discussed here)
In other words, when the port is already occupied by other processes, if you try to create this socket object and occupy this port, an error will be reported.
A server has to provide services to many clients. The server does not know when the clients will come. The server can only be "always ready" and provide services at any time when the clients come.
A server has three core tasks to do during its operation:
1. Read the request and parse it
2. Calculate the response based on the request
3. Write the response back to the client
For the echo server, it does not care about the second process. Whatever the request is, it will return the response.
But for a commercial-grade server, the main code is completing the second step.
In this method, the DatagramPacket of the parameter is an "output parameter"
What is passed into the receive is an empty object, and the content of the empty object will be filled in within the receive. When the receive execution ends, a DatagramPacket filled with content will be obtained.
The memory space used by this object to save data needs to be manually specified. Unlike the collection classes we have learned, it has its own internal memory management capabilities (can apply for memory, release memory, memory expansion, etc.)
The number 4096 is written casually, but it cannot be written too large, no more than 64kb.
As soon as the server program is started, it will immediately execute the loop and receive immediately.
If the client's request has not come yet, the receive method will block and wait until a client actually initiates a request.
In this code, a DatagramPacket object is constructed and the response is sent back to the client.
Note: The second parameter cannot be written as response, length()
This is because response,length() calculates the length in characters, while response.getBytes().length calculates the length in bytes.
If the string is in English, the number of bytes and the number of characters are the same, but if the string contains Chinese characters, the calculated results of the two will be different.
The socket api itself is processed in bytes
requestPacket.getSocketAddress() This part is to send the datagram to the client. You need to know the client's IP and port.
DatagramPacket This object contains the IP and port of the communicating parties.
Server part code:
import java.io.IOException;
import java.net.DatagramPacket;
import java.net.DatagramSocket;
import java.net.SocketException;
//UDP 的回显服务器
//客户端发的请求是啥，服务器的响应就是啥
public class UdpEchoServer {
private DatagramSocket socket = null;
public UdpEchoServer(int port) throws SocketException {
socket = new DatagramSocket(port);
}
//使用这个方法启动服务器
public void start() throws IOException {
System.out.println("服务器启动");
while (true){
//反复的，长期的执行针对客户端请求处理的逻辑
//一个服务器，运行过程中，要做的事情，主要是三个核心环节
//1、读取请求，并解析
DatagramPacket requestPacket = new DatagramPacket(new byte[4096],4096);
socket.receive(requestPacket);
//这样的转字符串的前提是，后续客户端发的数据就是一个文本的字符串
String request = new String(requestPacket.getData(),0,requestPacket.getLength());
//2、根据请求，计算出响应
String response = process(request);
//3、把响应写回给客户端
//此时需要告知网卡，要发的内容是啥，要发给谁
DatagramPacket responsePacket = new DatagramPacket(response.getBytes(),response.getBytes().length,
requestPacket.getSocketAddress());
socket.send(responsePacket);
//记录日志，方便观察程序执行效果
System.out.printf("[%s:%d] req: %s, resp: %s\n",responsePacket.getAddress().toString(),responsePacket.getPort(),
request,response);
}
}
//根据请求计算响应
public String process(String request){
return request;
}
public static void main(String[] args) throws IOException {
UdpEchoServer server = new UdpEchoServer(9090);
server.start();
}
}
Is this closed?
For our server program, it is not a big problem if the DatagramSocket is not closed. There is only one socket object in the entire program and it is not created frequently.
The life cycle of this object is very long and follows the entire program. At this time, the socket needs to remain open.
The socket object corresponds to the socket file in the system, and to the file descriptor ( the main purpose is to close the socket object to determine whether the file descriptor is available )
When the process ends, the pcb is recycled and the file descriptor table inside is destroyed.
But this is limited to: there is only one socket object, and the life cycle follows the process, so there is no need to release it at this time.
But if there are multiple socket objects, the socket object life cycle is shorter and needs to be created and released frequently. At this time, you must remember to close
(2) Client code
Write the client. The following parameters specify which IP and port to send to.
What is needed here is the InetAdress object, so use the static method of InetAdress, getByName to construct (factory mode/factory method)
The server will be automatically bound to 9090 when it starts.
The client will next access the window 9090
Client code:
import java.io.IOException;
import java.net.*;
import java.util.Scanner;
public class UdpEchoClient {
private DatagramSocket socket = null;
private String serverIp;
private int serverPort;
//服务器的 ip 和 服务器窗口
public UdpEchoClient(String ip ,int port) throws SocketException {
serverIp = ip;
serverPort = port;
//这个 new 操作，就不再指定端口了，让系统自动分配一个空闲端口
socket = new DatagramSocket();
}
//客户端启动，让这个客户端反复的从控制台读取用户输入的内容，把这个内容构造成 UPD 请求，发给服务器，再读取服务器返回的 UDP响应
//最终再显示在客户端的屏幕上
public void start() throws IOException {
System.out.println("客户端启动！");
Scanner scanner = new Scanner(System.in);
while (true){
//1、从控制台读取用户输入的内容
System.out.println("-> ");//命令提示符，提示用户输入字符串
String request = scanner.next();
//2、构造请求对象，并发给服务器
DatagramPacket requestPacket = new DatagramPacket(request.getBytes(),request.getBytes().length,
InetAddress.getByName(serverIp),serverPort);
socket.send(requestPacket);
//3、读取服务器的响应，并解析出响应内容
DatagramPacket responsePacket = new DatagramPacket(new byte[4096],4096);
socket.receive(responsePacket);
String response = new String(responsePacket.getData(),0,responsePacket.getLength());
//4、显示结果
System.out.println(request);
}
}
public static void main(String[] args) throws IOException {
UdpEchoClient client = new UdpEchoClient("127.0.1",9090);
client.start();
}
}At this point, run the server first, then the client, and then enter the content to see the execution results.
If multiple clients are started, multiple clients can also be handled by the server.
Implementation of translation server
The translation server requests some English words, and the response is the corresponding Chinese translation.
This server is similar to part of the code of the previous echo server, so we let it directly inherit the previous server.
Inheritance itself is for better "reuse of existing code"
If @Override is not added here, if the method name/parameter type/number of parameters/access permissions are mistaken, the rewrite will not be possible at this time, and it will be difficult to find.
import java.io.IOException;
import java.net.SocketException;
import java.util.HashMap;
import java.util.Map;
public class UdpDicSever extends UdpEchoServer{
private Map<String,String> dict = new HashMap<>();
public UdpDicSever(int port) throws SocketException {
super(port);
dict.put("cat","小猫");
dict.put("dog","小狗");
dict.put("duck","小鸭");
//可以在这里继续添加千千万万个单词，每个单词都有一个对应的翻译
}
//是要复用之前的代码，但是又要做出调整
@Override
public String process(String request){
//把请求对应单词的翻译给返回回去
return dict.getOrDefault(request,"该词没有查询到");
}
public static void main(String[] args) throws IOException {
UdpDicSever server = new UdpDicSever(9090);
//start 就不需要在写一遍了，就直接复用了之前的 start
server.start();
}
}3. TCP
TCP is a byte stream, transmitted byte by byte.
In other words, a TCP datagram is a byte array byte[]
The API provided by TCP also has two classes
1、ServerSocket
Socket used by the server
Construction method:
Other methods:
2、Socket
It is used both by the server and the client.
Construction method:
Other methods:
echo server
(1) Server code
Let’s now try to write a TCP version of the echo server
There will be some differences from UDP:
After entering the loop, the thing to do is not to read the client's request, but to process the client's "connection" first.
Although there are many connections in the kernel, they still have to be processed one by one in the application.
Connections in the kernel are like to-do items. These to-do items are in a queue data structure, and the application needs to complete these tasks one by one.
To complete the task, you need to get the task first and use serverSocker.accept()
The connection in the kernel is obtained into the application. This process is similar to the "producer-consumer model"
When the program starts, it will be executed immediately to accept
When the server executes accept, the client may not have arrived yet, and accpet will block.
Block until a client successfully connects
Accept is to take the connection that has been established in the kernel and get it to the application.
But the return value here is not an object like "Connection", but just a Socket object.
And this Socket object is like a headset, you can talk and hear the other party's voice.
Communicate with the other party over the network through the Socket object
There are two types of sockets involved in TCP server, serverSocket and clientSocket.
serverSocket can be understood as the person in the sales department responsible for soliciting customers, and clientSocket can be understood as the person in the sales department responsible for introducing details.
IO operations are relatively expensive
Compared with accessing memory, the more times IO is performed, the slower the program will be.
Use a piece of memory as a buffer. When writing data, write it to the buffer first, save a wave of data, and perform IO uniformly.
PrintWriter has a built-in buffer, which can be refreshed manually to ensure that the data here is actually sent through the network card and not left in the buffer.
It is safer to add flash here, but it will not necessarily go wrong if you don’t add it.
The buffer has a certain refresh strategy built-in, so it is recommended to add flash
In this program, two types of sockets are involved
1. ServerSocket (there is only one, the life cycle follows the program, it will be fine if it is not closed)
2、Socket
The socket here is created repeatedly
We need to ensure that the socket can be closed after the connection is disconnected, so add the code to close the socket in finally.
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.io.PrintWriter;
import java.net.ServerSocket;
import java.net.Socket;
import java.util.Scanner;
public class TcpEchoServer {
private ServerSocket serverSocket = null;
//这个操作就会绑定端口号
public TcpEchoServer(int port) throws IOException {
serverSocket = new ServerSocket(port);
}
//启动服务器
public void start() throws IOException {
System.out.println("启动服务器！");
while (true){
Socket clientSocket = serverSocket.accept();
processConnection(clientSocket);
}
}
//通过这个方法处理一个连接的逻辑
private void processConnection(Socket clientSocket) throws IOException {
System.out.printf("[%s,%d] 客户端上线!\n,", clientSocket.getInetAddress().toString(),clientSocket.getPort());
//接下来就可以读取请求，根据请求计算响应，返回响应
//Socket 对象内部，包含两个字节流对象，可以把这两个字节流对象获取到，完成后续的读写工作
try(InputStream inputStream = clientSocket.getInputStream();
OutputStream outputStream = clientSocket.getOutputStream()){
//一次连接中，可能会涉及到多次请求 / 响应
while (true){
//1、读取请求，并解析,为了读取方便，直接使用Scanner
Scanner scanner = new Scanner(inputStream);
if (!scanner.hasNext()){
//读取完毕，客户端下线
System.out.printf("[%s:%d] 客户端下线! \n",clientSocket.getInetAddress().toString(),clientSocket.getPort());
break;
}
//这个代码暗含一个约定：客户端发过来的请求，得是文本数据，同时，还得带有空白符作为分割
String request = scanner.next();
//2、根据请求，计算响应
String response = process(request);
//3、把响应写回给客户端,把 OutputStream 使用 PrinterWriter 包裹一下，方便进行发数据
PrintWriter writer = new PrintWriter(outputStream);
//使用 PrintWriter 的 println 方法，把响应返回给客户端
//此处使用 println 而不是 print 就是为了在结尾加上换行符，方便客户端读取响应，使用 Scanner.next 读取
writer.println(response);
//这里还需要加一个 “刷新缓冲区” 的操作
writer.flush();
//日志，打印一下当前的请求详情
System.out.printf("[%s:%d] req: %s,resp: %s\n",clientSocket.getInetAddress().toString(),clientSocket.getPort(),
request,response);
}
}finally {
//在 finally 中，加上 close 操作，确保当前 socket 能够被及时关闭
clientSocket.close();
}
}
public String process(String request){
return request;
}
public static void main(String[] args) throws IOException {
TcpEchoServer server = new TcpEchoServer(9090);
server.start();
}
}(2) Client code
What the client has to do:
1. Read user input from the console
2. Construct the input content into a request and send it to the server
3. Read the response from the server
4. Display the response on the console
In the current code, Scanner and PrintWriter are not closed. Will file leakage occur?
Won't! ! !
The resources held in the stream object have two parts:
1. Memory (when the object is destroyed, the memory is recycled), the while loop completes one cycle, and the memory is naturally destroyed.
2. File descriptor
Scanner and printWriter do not hold file descriptors, but hold references to inputstream and outputstream, and these two objects have been closed.
To be more precise, it is held by the socket object, so just close the socket object.
Not every stream object holds a file descriptor . To hold a file descriptor, you need to call the open method provided by the operating system.
hasNext will also block when the client does not send a request. It will block until the client actually sends data or the client exits, and hasNext will return.
There is still a big problem in the current code: what happens when we start two clients?
Phenomenon:
After the first client is connected, the second client cannot be processed correctly.
The server cannot see that the client is online, and the request from the client cannot be processed.
After the first client exits, all previous requests sent by the second client will be responded to.
When a client comes, accept will return and enter processConnection
The loop will process the client's request until the client is finished, and the method will not end until it returns to the first layer.
The key to the problem is that during the process of processing a client's request, accept cannot be called a second time. That is to say, even if the second client comes, it cannot be processed.
Here, the client's processing of processConnection itself takes a long time to execute, because we don't know when the client will end, and we don't know how many requests the client will send.
Then we hope that while executing this method, accept can also be called. At this time, multi-threading can be used.
We can be responsible for soliciting customers in the main thread. After attracting customers, we can create a new thread and let the new thread be responsible for processing various requests from the client.
After the above improvements, as long as the server resources are sufficient, it is possible to have several clients.
In fact, if the code we just wrote is not written like this, for example, each client is required to send only one request and disconnect after sending it, the above situation will be alleviated to a certain extent, but there will still be similar problems.
Processing multiple messages will naturally extend the execution time of proessConnection, making this problem more serious.
When writing TCP programs, two writing methods are involved:
1. Only one request and response are transmitted in one connection (short connection)
2. One request can transmit multiple requests and responses (long connection)
Now we are, there is a connection, there is a new thread
If there are many clients that frequently connect/disconnect, the server will involve frequent creation/release of threads.
Using a thread pool is a better solution
This is wrong to write! ! !
processConnection and the main thread are different threads. During the execution of processConnecton, the main thread try is completed. This will cause the clientSocket to be closed before it is used up.
Therefore, the clientSocket still needs to be handed over to processConnection to close, so it should be written like this:
Although using a thread pool avoids frequent creation and destruction of threads, after all, each client corresponds to one thread.
If there are many clients corresponding to the server, the server will need to create a large number of threads, which will cause a lot of overhead for the server.
When the number of clients and threads increases further, the burden on the system will become heavier and heavier, and the response speed will be greatly reduced.
Is there a way to use one thread (or at most three or four threads) to efficiently handle concurrent requests from many clients?
This is also called the C10M problem : at the same time, there are 1kw of concurrent client requests.
Many technical means have been introduced, one of which is very effective/necessary: IO multiplexing/IO multiplexing
This thing is one of the important means to solve high concurrency (C10M)
To solve the problem of high concurrency, to put it bluntly, there are only four words:
1. Open source: introduce more hardware resources
2. Throttling: Increase the number of requests that a unit’s hardware resources can handle
IO multiplexing is a throttling method. The same request consumes less hardware resources.