Article directory
- foreword
- 1. Concept
- 2. Classification (three categories)
- 3. UDP datagram socket programming
- Four, TCP datagram socket programming
- 5. About the use of input stream and output stream
- 6. Data packet-oriented VS byte stream-oriented
- Summarize
foreword
Blogger's Personal Community: Development and Algorithm Learning Community
Blogger Profile: Killing Vibe's Blog
Everyone is welcome to join and exchange and learn together~~
1. Concept
Socket is a technology provided by the system for network communication, and is the basic operating unit of network communication based on TCP/IP protocol. Socket-based network program development is network programming.
Socket is an important concept in network programming at the application layer
The transport layer, network layer, data link layer, and physical layer all provide services through OS+hardware , while the application layer needs to enjoy the service through the service window (Socket) provided by the OS to enjoy the network services provided by the OS .
Expand :
System calls provided natively by the OS (network programming on Linux):
int fd = socket();
setsocketopt(fd,TCP or UDP)
2. Classification (three categories)
Socket sockets are mainly divided into the following three categories for transport layer protocols:
2.1 Stream socket: use transport layer TCP protocol
TCP, that is, Transmission Control Protocol (Transmission Control Protocol), transport layer protocol.
The following are the characteristics of TCP:
- connected
- reliable transmission
- stream-oriented
- There are receive buffers and send buffers
- Any size
For the byte stream, it can be simply understood that the transmission data is based on the IO stream. The characteristic of the streaming data is that it is unbounded data when the IO stream is not closed. It can be sent multiple times or separated into multiple streams. received.
2.2 Datagram socket: use transport layer UDP protocol
UDP, namely User Datagram Protocol (User Datagram Protocol), transport layer protocol.
The following are the characteristics of UDP:
- no connection
- unreliable transmission
- Datagram Oriented
- With receive buffer, without send buffer
- Size limited: transfer up to 64k at a time
For datagrams, it can be simply understood that the transmission data is piece by piece, if a piece of data is sent 100 bytes, it must be sent at one time, and the reception must also receive 100 bytes at a time, instead of 100 times, each time Receive 1 byte.
2.3 Raw sockets
Raw sockets are used to customize the transport layer protocol, and are used to read and write IP protocol data that the kernel does not process.
3. UDP datagram socket programming
For the UDP protocol, it has the characteristics of connectionless and datagram-oriented, that is, no connection is established every time, and all datagrams are sent at one time, and all datagrams are received at one time.
3.1 Java datagram socket communication model
Java uses the UDP protocol to communicate, mainly based on DatagramSocketthe class to create a datagram socket, and use it
DatagramPacketas a UDP datagram to send or receive. The process of sending and receiving UDP datagrams once is as follows:
3.2 DatagramSocket API
DatagramSocketIs a UDP Socket for sending and receiving UDP datagrams.
3.2.1 DatagramSocketConstruction method:
Note :
-
UDP server (Server): A fixed port is used to facilitate the client (Client) to communicate;
useDatagramSocket(int port)it to bind to the port specified by the machine. This method may have the risk of error, indicating that the port has been occupied by other processes. -
UDP client (Client): No need to use a fixed port (you can also use a fixed port), use a random port;
useDatagramSocket(), bind to any random port on this machine
3.2.2 DatagramSocketOrdinary methods (belonging to the DatagramSocket class):
Note :
-
Once the two parties in the communication have a communication line in a logical sense, the status of the two parties is equal (anyone can be the sender and receiver)
-
The sender calls
send()the method, and the receiver callsreceive()the method -
After the communication is over, both parties should call
close()the method to recycle resources
3.3 DatagramPacket API
DatagramPacketIt is the datagram sent and received by UDP Socket.
This class is the defined packet: data abstraction in the communication process
Can be understood as: an envelope sent/received (five-tuple + letter)
3.3.1 DatagramPacketConstruction method:
Note :
- As a receiver: only need to provide the location to store the received data (byte[] buf + int length)
- As the sender: need to have the data to be sent (byte[] buf +int offset +int length), who to send to (remote ip+remote port)
3.3.2 DatagramPacketOrdinary method:
Note :
getAddress()Generally, the method and method are used for the servergetPort()to obtain the client's ip address and port number port- Generally, it is used by the receiver (it can be the server or the client)
getData()to get the "letter" (the application layer data sent by the other party's process)
3.4 NetSocketAddress API
InetSocketAddress (subclass of SocketAddress) constructor:
3.5 Code example (with request and response)
The following only shows part of the code, and the complete code can be seen in the blogger's gitee warehouse:
UDP client :
public class UserInputLoopClient {
public static void main(String[] args) throws Exception {
Scanner scanner = new Scanner(System.in);
// 1. 创建 UDP socket
Log.println("准备创建 UDP socket");
DatagramSocket socket = new DatagramSocket();
Log.println("UDP socket 创建结束");
System.out.print("请输入英文单词: ");
while (scanner.hasNextLine()) {
// 2. 发送请求
String engWord = scanner.nextLine();
Log.println("英文单词是: " + engWord);
String request = engWord;
byte[] bytes = request.getBytes("UTF-8");
// 手动构造服务器的地址
// 现在，服务器和客户端在同一台主机上，所以，使用 127.0.0.1 （环回地址 loopback address）
// 端口使用 TranslateServer.PORT（8888）
InetAddress loopbackAddress = InetAddress.getLoopbackAddress();
InetAddress remoteAddress = Inet4Address.getByName("182.254.132.183");
DatagramPacket sentPacket = new DatagramPacket(
bytes, 0, bytes.length, // 要发送的数据
remoteAddress, TranslateServer.PORT // 对方的 ip + port
);
Log.println("准备发送请求");
socket.send(sentPacket);
Log.println("请求发送结束");
// 3. 接收响应
byte[] buf = new byte[1024];
DatagramPacket receivedPacket = new DatagramPacket(buf, buf.length);
Log.println("准备接收响应");
socket.receive(receivedPacket);
Log.println("响应接收接收");
byte[] data = receivedPacket.getData();
int len = receivedPacket.getLength();
String response = new String(data, 0, len, "UTF-8");
String chiWord = response;
Log.println("翻译结果: " + chiWord);
System.out.print("请输入英文单词: ");
}
// 4. 关闭 socket
socket.close();
}
}
UDP server :
// 提供翻译的服务器
public class TranslateServer {
// 公开的 ip 地址：就看进程工作在哪个 ip 上
// 公开的 port：需要程序中指定
public static final int PORT = 8888;
// SocketException -> IOException -> Exception
public static void main(String[] args) throws Exception {
Log.println("准备进行字典的初始化");
initMap();
Log.println("完成字典的初始化");
Log.println("准备创建 UDP socket，端口是 " + PORT);
DatagramSocket socket = new DatagramSocket(PORT);
Log.println("UDP socket 创建成功");
// 作为服务器，是被动的，循环的进行请求-响应周期的处理
// 等待请求，处理并发送响应，直到永远
while (true) {
// 1. 接收请求
byte[] buf = new byte[1024]; // 1024 代表我们最大接收的数据大小（字节）
DatagramPacket receivedPacket = new DatagramPacket(buf, buf.length);
Log.println("准备好接收 DatagramPacket，最大大小为: " + buf.length);
Log.println("开始接收请求");
socket.receive(receivedPacket); // 这个方法就会阻塞（程序执行到这里就不动了，直到有客户发来请求，才能继续）
Log.println("接收到请求");
// 2. 一旦走到此处，一定是接收到请求了，拆信
// 拆出对方的 ip 地址
InetAddress address = receivedPacket.getAddress();
Log.println("对方的 IP 地址: " + address);
// 拆出对方的端口
int port = receivedPacket.getPort();
Log.println("对方的 port: " + port);
// 拆出对方的 ip 地址 + port
SocketAddress socketAddress = receivedPacket.getSocketAddress();
Log.println("对象的完整地址: " + socketAddress);
// 拆出对方发送过来的数据，其实这个 data 就是我们刚才定义的 buf 数组
byte[] data = receivedPacket.getData();
Log.println("接收到的对象的数据: " + Arrays.toString(data));
// 拆出接收到的数据的大小（字节）
int length = receivedPacket.getLength();
Log.println("接收的数据大小（字节）：" + length);
// 3. 解析请求 ：意味着我们需要定义自己的应用层协议
// 首先，做字符集解码 byte[] -> String
String request = new String(data, 0, length, "UTF-8");
// 这个按照我们的应用层协议
String engWord = request;
Log.println("请求（英文单词）：" + engWord);
// 4. 执行业务（翻译服务），不是我们本次演示的重点
String chiWord = translate(engWord);
Log.println("翻译后的结果：" + chiWord);
// 5. 按照应用层协议，封装响应
String response = chiWord;
// 进行字符集编码 String -> byte[]
byte[] sendBuf = response.getBytes("UTF-8");
// 6. 发送响应
// 作为发送方需要提供
DatagramPacket sentPacket = new DatagramPacket(
sendBuf, 0, sendBuf.length, // 要发送的数据
socketAddress // 从请求信封中拆出来的对象的地址（ip + port）
);
Log.println("准备好发送 DatagramPacket 并发送");
socket.send(sentPacket);
Log.println("发送成功");
// 7. 本次请求-响应周期完成，继续下一次请求-响应周期
}
// socket.close(); // 由于我们是死循环，这里永远不会走到
}
private static final HashMap<String, String> map = new HashMap<>();
private static void initMap() {
map.put("apple", "苹果");
map.put("pear", "梨");
map.put("orange", "橙子");
}
private static String translate(String engWord) {
String chiWord = map.getOrDefault(engWord, "查无此单词");
return chiWord;
}
}
Custom log class (remember to import this class) :
public class Log {
public static void println(Object o) {
LocalDateTime localDateTime = LocalDateTime.now(ZoneId.of("Asia/Shanghai"));
DateTimeFormatter formatter = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss");
String now = formatter.format(localDateTime);
String message = now + ": " + (o == null ? "null" : o.toString());
System.out.println(message);
}
public static void main(String[] args) {
println(1);
}
}
Four, TCP datagram socket programming
4.1 Java stream socket communication model
4.2 ServerSocket API
ServerSocketIt is an API to create a TCP server Socket.
4.2.1 ServerSocketConstruction method:
The TCP Socket object used by the server (the incoming port is the port to be exposed, generally called the listening port)
4.2. ServerSocketCommon methods
Note :
-
accept: pick up the phone (the server is the one that rings)
-
Socket object: established connection
-
close: hang up the phone (anyone can hang up)
4.3 Socket API
Socket is the client Socket, or the server Socket returned by the server after receiving the client's request to establish a connection (accept method).
Regardless of whether it is a client-side or a server-side Socket, after the two parties establish a connection, the information of the other end is saved and used to send and receive data with the other party.
4.3.1 SocketConstruction method:
Note :
- The server's Socket object is obtained from accept() , so only the client's Socket object needs to be instantiated manually . This construction method is for the client to use and pass in the server's ip+port
- Once the socket object is obtained (both parties obtain it at the same time), the two parties are equal, and only the sender and the receiver can be distinguished
4.3.2 SocketOrdinary method:
Note :
- Input stream: From the perspective of the process, the object behind it is the network card, the TCP connection abstracted by the network card, so it is used by the receiver
- Output stream: the same, so it is used by the sender
4.4 Long and short connections in TCP
When TCP sends data, it needs to establish a connection first. When the connection is closed, it is determined whether it is a short connection or a long connection:
- Short connection: Every time data is received and a response is returned, the connection is closed, which is a short connection. In other words, a short connection can only send and receive data once.
- Long-term connection: The connection is not closed, and the connection is always maintained. The two parties continue to send and receive data, which is a long-term connection. That is to say, a persistent connection can send and receive data multiple times.
Compared with the above long and short connections, the differences between the two are as follows:
- Time-consuming to establish and close a connection: for a short connection, a connection needs to be established and closed for each request and response; for a long connection, only the first connection needs to be established, and subsequent requests and responses can be transmitted directly. Relatively speaking, it is time-consuming to establish a connection and close the connection, and the long-term connection is more efficient.
- Actively sending requests are different: short connections are generally the client actively sending requests to the server; and long connections can be either the client actively sending requests or the server actively sending requests.
- The usage scenarios of the two are different: short connections are suitable for scenarios where the frequency of client requests is not high, such as browsing web pages. Persistent connections are suitable for scenarios where the client communicates frequently with the server, such as chat rooms and real-time games.
Extended understanding:
Long connections based on BIO (synchronous blocking IO) will always occupy system resources. For server systems with high concurrency requirements, such consumption is unbearable.
Since each connection needs to be continuously blocked waiting to receive data, each connection will run in a thread.
A blocking wait corresponds to a request and a response, and non-stop processing is the characteristic of long connections: the connection is not closed all the time, and requests are processed non-stop.
In practical applications, the server generally implements persistent connections based on NIO (that is, synchronous non-blocking IO), and the performance can be greatly improved.
One question remains :
If multiple persistent connection clients connect to the server at the same time, can it be processed normally?
It is necessary to configure the client in IDEA to support running multiple instances at the same time!
- Short connection client <–> short connection server supports simultaneous online
- Short connection client <-> long connection server supports simultaneous online
- Long connection client <-> long connection server does not support simultaneous online
Therefore, multi-threading can be used to solve the problem that the long connection client does not support simultaneous online:
The task is dedicated to other threads for processing, and the main thread is only responsible for accepting sockets.
4.5 Code example (short connection)
Here only demonstrates short connection, long connection and multi-threading under blogger's personal warehouse:
TCP server:
public class TranslateServerShortConnection {
public static final int PORT = 8888;
public static void main(String[] args) throws Exception {
Log.println("启动短连接版本的 TCP 服务器");
initMap();
ServerSocket serverSocket = new ServerSocket(PORT);
while (true) {
// 接电话
Log.println("等待对方来连接");
Socket socket = serverSocket.accept();
Log.println("有客户端连接上来了");
// 对方信息：
InetAddress inetAddress = socket.getInetAddress(); // ip
Log.println("对方的 ip: " + inetAddress);
int port = socket.getPort(); // port
Log.println("对方的 port: " + port);
SocketAddress remoteSocketAddress = socket.getRemoteSocketAddress(); // ip + port
Log.println("对方的 ip + port: " + remoteSocketAddress);
// 读取请求
InputStream inputStream = socket.getInputStream();
Scanner scanner = new Scanner(inputStream, "UTF-8");
String request = scanner.nextLine(); // nextLine() 就会去掉换行符
String engWord = request;
Log.println("英文: " + engWord);
// 翻译
String chiWord = translate(engWord);
Log.println("中文: " + chiWord);
// 发送响应
String response = chiWord; // TODO: 响应的单词中是没有 \r\n
OutputStream outputStream = socket.getOutputStream();
OutputStreamWriter outputStreamWriter = new OutputStreamWriter(outputStream, "UTF-8");
PrintWriter writer = new PrintWriter(outputStreamWriter);
Log.println("准备发送");
writer.printf("%s\r\n", response);
writer.flush();
Log.println("发送成功");
// 挂掉电话
socket.close();
Log.println("挂断电话");
}
// serverSocket.close();
}
private static final HashMap<String, String> map = new HashMap<>();
private static void initMap() {
map.put("apple", "苹果");
map.put("pear", "梨");
map.put("orange", "橙子");
}
private static String translate(String engWord) {
String chiWord = map.getOrDefault(engWord, "查无此单词");
return chiWord;
}
}
TCP client:
public class UserInputLoopShortConnectionClient {
public static void main(String[] args) throws Exception {
Scanner userInputScanner = new Scanner(System.in);
while (true) {
// 这里做了一个假设：1）用户肯定有输入 2）用户一行一定只输入一个单词（没有空格）
System.out.print("请输入英文单词: ");
if (!userInputScanner.hasNextLine()) {
break;
}
String engWord = userInputScanner.nextLine();
// 直接创建 Socket，使用服务器 IP + PORT
Log.println("准备创建 socket(TCP 连接)");
Socket socket = new Socket("127.0.0.1", TranslateServerShortConnection.PORT);
Log.println("socket(TCP 连接) 创建成功");
// 发送请求
Log.println("英文: " + engWord);
String request = engWord + "\r\n";
OutputStream os = socket.getOutputStream();
OutputStreamWriter osWriter = new OutputStreamWriter(os, "UTF-8");
PrintWriter writer = new PrintWriter(osWriter);
Log.println("发送请求中");
writer.print(request);
writer.flush();
Log.println("请求发送成功");
// 等待接受响应
InputStream is = socket.getInputStream();
Scanner socketScanner = new Scanner(is, "UTF-8");
// 由于我们的响应一定是一行，所以使用 nextLine() 进行读取即可
// nextLine() 返回的数据中，会自动把 \r\n 去掉
// TODO: 没有做 hasNextLine() 的判断
Log.println("准备读取响应");
String chiWord = socketScanner.nextLine();
Log.println("中文: " + chiWord);
socket.close();
}
}
}
5. About the use of input stream and output stream
5.1 About the use of input stream:
- If a direct binary read
byte[] buf = new byte[1024];
int n = inputStream.read(buf);
- If you read text data, it is recommended to use Scanner directly to encapsulate InputStream before using it
Scanner S = new Scanner(inputStream,"UTF-8");
s.nextLine() ... s.hasNextLine()
5.2 About the use of output stream:
- If direct binary output
outputStream.write(buf,offset,length) - If it is text output, it is recommended OutputStream -> OutputStreamWriter -> PrintWriter
OutputStreamWriter osWriter = new OutputStreamWriter(outputStream,"UTF-8");
PrintWriter writer = new PrintWriter(osWriter);
writer.println(...);
writer.print(...);
writer.printf(format,...);
IMPORTANT: Don't forget to flush the buffer, otherwise the data may not reach the other side!!!
outputStream.flush();
//writer.flush();
6. Data packet-oriented VS byte stream-oriented
Take a chestnut:
Summarize
Regarding the problem of port being occupied:
If a process A has bound a port, and then starts a process B to bind the port, an error will be reported. This situation is also called the port is occupied. For the java process, the common error message that the port is occupied is as follows:
At this point, you need to check which port process B is bound to, and then check which process the port is occupied by. The following is the way to check the process by port number:
-
Enter netstat -ano | findstr port number in cmd to display the pid of the corresponding process. For example, the following command shows
the pid of the 8888 process
-
In the task manager, find the process through the pid
to solve the problem of the port being occupied :
- If the process is useless, you can kill the process
- If the process is uncertain, you can use another port