Socket-based network programming
Preface
When we do network programming, we mainly write "application layer" code. If we really want to send this data, we need the upper layer protocol to call the lower layer protocol, that is, the application layer calls the transport layer. The transport layer provides a set of APIs to the application layer, collectively called Socket API
1. Socket-based network communication transmission (transport layer)
Socket Socket is a technology provided by the system for network communication, and is the basic operating unit of network communication based on the TCP/IP protocol. The development of network programs based on Socket sockets is network programming.
In this section, we mainly study two types of Sockets for transport layer protocols:
Datagram socket : uses the transport layer UDP protocol. UDP, User Datagram Protocol, transport layer protocol.
The following are the characteristics of UDP (details will be introduced later):
- No connection: Both parties using UDP communication do not need to deliberately save the relevant information of the other end.
- Unreliable transmission: don’t care about the results
- Datagram-oriented: taking a udp datagram as the basic unit
- Full duplex: two-way communication (with receiving buffer, no sending buffer)
- Size limit: transfer up to 64k at a time
Stream socket : uses the transport layer TCP protocol. TCP, Transmission Control Protocol, transport layer protocol.
The following are the characteristics of TCP (details will be introduced later):
- There is a connection: If both parties use TCP to communicate, they need to deliberately record each other's information.
- Reliable transmission: Transmit as much as possible after sending, and you will know if it fails.
- Oriented to byte stream: byte stream is the basic unit of transmission, and the reading and writing methods are very flexible.
- Full duplex: two-way communication (there is a receive buffer and a send buffer)
- No limit to size
2. UDP datagram socket programming
DatagramSocket is a UDP Socket used to send and receive UDP datagrams.
We can understand the Socket here by analogy with the File object. We know that if we cannot directly operate the hard disk, we need to use File if we want to operate the hard disk. Object indirect operations. Socket is similar. It corresponds to the hardware device of the network card. If we want to operate the network card, we need a Socket object to indirectly operate the network card. Writing data to the socket object is equivalent to sending a message through the network card. Reading data from the socket object is equivalent to receiving messages through the network card.
1. UDP socket programming API
(1) DatagramSocket
DatagramSocket construction method
| method signature | Method description |
|---|---|
| DatagramSocket() | Create a UDP datagram socket Socket and bind it to any random port on the local machine (generally used for clients) |
| DatagramSocket(int port) | Create a UDP datagram socket Socket and bind it to the port specified on the local machine (generally used on the server side) |
Note: For the server, it is generally necessary to manually specify a fixed port, but the client is not required. Similar to when I go to a canteen to eat, the window that provides me with food is a server, and I am the consumer who receives the food. It can be regarded as a client. The window needs to have a fixed window number so that I can find it easily. When enjoying food, there are no fixed seats, just sit wherever there is space.
DatagramSocket methods
| method signature | Method description |
|---|---|
| void receive(DatagramPacket p) | Receive a datagram from this socket (if no datagram is received, this method will block waiting) |
| void send(DatagramPacket p) | Send datagram packets from this socket (will not block waiting, send directly) |
| void close() | Close this datagram socket |
(2) DatagramPacket
DatagramPacket is the datagram sent and received by UDP Socket
DatagramPacket construction method
| method signature | Method description |
|---|---|
| DatagramPacket(byte[] buf, int length) | Construct a DatagramPacket to receive datagrams. The received data is stored in a byte array (the first parameter buf) and receives the specified length (the second parameter length). |
| DatagramPacket(byte[] buf, int offset, int length) | Used to receive datagrams. The received data is stored in a byte array (the first parameter buf), and is received from offset to the specified length length. |
| DatagramPacket(byte[] buf, int length, InetAddress address, int port) | Used to send datagrams. The data sent is a byte array (the first parameter buf), length is the data length, address is the address of the target host, and port is the port number of the target host. |
| DatagramPacket(byte[] buf, int offset, int length, InetAddress address, int port) | Used to send datagrams, the data sent is a byte array (the first parameter buf), from offset to length is the data length, address is the address of the target host, and port is the port number of the target host. |
| DatagramPacket(byte[] buf, int length, SocketAddress address) | Used to send datagrams. The data sent is a byte array (the first parameter buf), and length is the data length. |
| DatagramPacket(byte[] buf, int offset, int length, SocketAddress address) | Construct a DatagramPacket to send datagrams. The data sent is in the byte array (the first parameter buf), from offset to the specified length. address specifies the IP and port number of the destination host |
DatagramPacket method
| method signature | Method description |
|---|---|
| InetAddress getAddress() | Obtain the IP address of the sending host from the received datagram; or obtain the IP address of the receiving host from the sent datagram. |
| int getPort() | Obtain the port number of the sending host from the received datagram; or obtain the port number of the receiving host from the sent datagram. |
| byte[] getData() | Get the data in the datagram |
2. Use UDP Socket to achieve simple communication
Below we implement a simple client-server communication using UDP protocol in Java.
The server-client code below looks quite complicated. In fact, it is similar to JDBC in the database. It is a fixed routine. Although more complex server-client programs will be written later, they will be expanded on this basis. For example, the UDP Socket below implements network communication through the transport layer. The code is nothing more than the following steps:
1. For the server:
(1) Read the request and parse it
(2) Calculate the response according to the request
(3) Send the response result to the client
2. For the client:
(1) Construct and send the request
(2) Receive the response returned by the server Respond and parse the response
Server program:
import java.io.IOException;
import java.net.DatagramPacket;
import java.net.DatagramSocket;
import java.net.SocketException;
// Echo-回显服务器。客户端发了个请求，服务器返回一个一模一样的响应。
public class UdpEchoSever {
// 需要先定义一个 socket 对象，使用网络通信，必须要使用 socekt 对象
private DatagramSocket socket = null;
// 绑定一个端口号，不一定能成功，比如某个端口号已经被别的进程占用了，此时这里的绑定操作就会出错。
// 需要注意的是：同一个主机上，一个端口，同一时刻，只能被一个进程绑定。
public UdpEchoSever(int port) throws SocketException {
socket = new DatagramSocket(port);
}
// 启动服务器主逻辑
public void start() throws IOException {
System.out.println("服务器启动！");
while (true) {
// 每次循环，做三件事
// 1. 读取请求并解析
// 构造一个空的DatagramPacket对象，用来接收客户端请求
DatagramPacket requestPacket = new DatagramPacket(new byte[4096],4096);
// 从网卡上接收请求 此处的 requestPacket 为输出型参数
socket.receive(requestPacket);
// 这里为了方便处理这个请求，将数据包转化为 String
String request = new String(requestPacket.getData(),0,requestPacket.getLength());
// 2. 根据请求计算响应
String response = process(request);
// 3. 把响应结果写回到客户端
// 根据 response 字符串，构造一个 DatagramPacket
// 和请求 packet 不同，此处构造响应的时候，需要指定这个包要发给谁
DatagramPacket responsePacket = new DatagramPacket(response.getBytes(),response.getBytes().length,
// requestPacket 是从客户端这里收来的，getSocketAddress 会得到客户端的 ip 何为端口
requestPacket.getSocketAddress());
socket.send(responsePacket);
// 方面查看，打印一下日志
// ip 和 端口号 + 请求内容 + 响应内容
System.out.printf("[%s:%d] req: %s, resp: %s\n",requestPacket.getAddress().toString(),requestPacket.getPort(),request,response);
}
}
// process是请求处理方法，这是服务器中的一个关键环节！！！
public String process(String request) {
return request;
}
// 主函数
public static void main(String[] args) throws IOException {
UdpEchoSever udpEchoSever = new UdpEchoSever(9090);
udpEchoSever.start();
}
}
Client program:
import java.io.IOException;
import java.net.*;
import java.util.Scanner;
public class UdpEchoClient {
private DatagramSocket socket = null;
private String serverIP;
private int serverPort;
// 客户端启动, 需要知道服务器在哪里!!
public UdpEchoClient(String serverIP, int serverPort) throws SocketException {
// 对于客户端来说, 不需要显示关联端口.
// 不代表没有端口, 而是系统自动分配了个空闲的端口.
socket = new DatagramSocket();
this.serverIP = serverIP;
this.serverPort = serverPort;
}
public void start() throws IOException {
// 通过这个客户端可以多次和服务器进行交互.
Scanner scanner = new Scanner(System.in);
while (true) {
// 1. 先从控制台, 读取一个字符串过来
// 先打印一个提示符, 提示用户要输入内容
System.out.print("-> ");
String request = scanner.next();
// 2. 把字符串构造成 UDP packet, 并进行发送.
DatagramPacket requestPacket = new DatagramPacket(request.getBytes(), request.getBytes().length,
InetAddress.getByName(serverIP), serverPort);
socket.send(requestPacket);
// 3. 客户端尝试读取服务器返回的响应
DatagramPacket responsePacket = new DatagramPacket(new byte[4096], 4096);
socket.receive(responsePacket);
// 4. 把响应数据转换成 String 显示出来.
String response = new String(responsePacket.getData(), 0, responsePacket.getLength());
System.out.printf("req: %s, resp: %s\n", request, response);
}
}
public static void main(String[] args) throws IOException {
// 127.0.0.1 是一个特殊的IP地址，表示本机的回环地址。
UdpEchoClient udpEchoClient = new UdpEchoClient("127.0.0.1", 9090);
udpEchoClient.start();
}
}
For UDP Echo Sever, the life cycle of the socket object follows the entire program and does not require close. This socket object is no longer needed after the loop is released, but the end of the loop means the end of start, the end of the main method, and the end of the process. All file resources will be automatically released when the process ends.
3. TCP stream socket programming
1. TCP stream socket programming API
(1) SeverSocket
ServerSocket is an API for creating TCP server Socket.
ServerSocket construction method
| method signature | Method description |
|---|---|
| ServerSocket(int port) | Create a server-side stream socket and bind it to the specified port |
ServerSocket methods
| method signature | Method description |
|---|---|
| Socket.accept() | Start listening to the specified port (the port bound at creation). After a client connects, return a server Socket object and establish a connection with the client based on the Socket. Otherwise, it will block and wait. |
| void close() | close this socket |
(2) Socket
Socket is the client Socket, or the server Socket returned after the server receives the client's request to establish a connection (accept method).
Whether it is a client or a server Socket, the peer information is saved after the two parties establish a connection and is used to send and receive data with the other party.
Socket constructor
| method signature | Method description |
|---|---|
| Socket(String host, intport) | Create a client stream socket and establish a connection with the process on the host with the corresponding IP and the corresponding port. |
Socket method
| method signature | Method description |
|---|---|
| int getPort() | Returns the remote port number to which this socket is connected |
| InetAddress getInetAddress() | Returns the address to which the socket is connected |
| InputStream getInputStream() | Returns the input stream for this socket |
| OutputStream getOutputStream() | Returns the output stream for this socket |
| void close() | close this socket |
2. Use TCP Socket to achieve simple communication
Below we also write a simple echo server and client based on the TCP protocol. The user can send requests to the server through the client and receive responses.
Server program:
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.io.PrintWriter;
import java.net.ServerSocket;
import java.net.Socket;
import java.util.Scanner;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
public class TcpEchoSever {
// 这里有个比喻：
// severSocket 看做是外场拉客的小哥
// clientSocket 看做内场服务的小姐姐
// severSocket 只有一个，clientSocket 会给每个客户端都分配一个
private ServerSocket serverSocket = null;
public TcpEchoSever(int port) throws IOException {
serverSocket = new ServerSocket(port);
}
public void start() throws IOException {
System.out.println("服务器启动!");
ExecutorService pool = Executors.newCachedThreadPool();
while (true) {
Socket clientSocket = serverSocket.accept();
// 如果直接调用，该方法会影响这个循环的二次执行，导致 accept 不及时
// 创建新线程，用新线程调用 processConnection
// 每次来一个新的客户端都创建一个新线程
// 1.方案一：每次创建线程（每次创建销毁，开销较大）
// Thread t = new Thread(()->{
// processConnection(clientSocket);
// });
// t.start();
// 2.方案二：使用线程池
pool.submit(()->{
try {
processConnection((clientSocket));
} catch (IOException e) {
e.printStackTrace();
}
});
}
}
private void processConnection(Socket clientSocket) throws IOException {
// 打印一下日志
System.out.printf("[%s:%d] 客户端上线!\n",clientSocket.getInetAddress().toString(),
clientSocket.getPort());
// try () 这种写法，( ) 中允许写多个流对象，使用 ; 分割
try (InputStream inputStream = clientSocket.getInputStream();
OutputStream outputStream = clientSocket.getOutputStream()) {
// 为了简单，把字节流包装成了更方便的字符流
Scanner scanner = new Scanner(inputStream);
PrintWriter printWriter = new PrintWriter(outputStream);
// 一次可能发来多个请求，这里规定以 \n 为分隔符
while (true) {
// 1.读取请求
// 特殊处理一下：
if (!scanner.hasNext()) {
// 读取的流到了结尾（对端关闭了）
System.out.printf("[%s:%d] 客户端下线!\n",clientSocket.getInetAddress().toString(),
clientSocket.getPort());
break;
}
// 直接使用 scanner 读取一段字符串
String request = scanner.next();
// 2.根据请求计算响应
String response = process(request);
// 3.把响应写会给客户端，不要忘了，响应里也是要带上换行的
printWriter.println(response);
// 写网卡为全缓冲，这里使用flush刷新
printWriter.flush();
// 最后打印一下日志
System.out.printf("[%s:%d] req: %s resp: %s\n",clientSocket.getInetAddress().toString(),
clientSocket.getPort(),request,response);
}
} catch (IOException e) {
e.printStackTrace();
} finally {
// 关闭连接
clientSocket.close();
}
}
// 处理请求
public String process(String request) {
return request;
}
// 主方法
public static void main(String[] args) throws IOException {
TcpEchoSever tcpEchoSever = new TcpEchoSever(9090);
tcpEchoSever.start();
}
}
Client program:
import java.io.IOException;
import java.io.InputStream;
import java.io.OutputStream;
import java.io.PrintWriter;
import java.net.Socket;
import java.util.Scanner;
// idea 中默认一个程序只能启动一个，启动多个客户端可配置一下 IDEA。
public class TcpEchoClient {
private Socket socket = null;
// ***只有这里会建立连接，和 Udp 不同***
public TcpEchoClient(String severIp, int port) throws IOException {
// 这个操作就相当于让客户端和服务器建立 TCP 连接
// 这里的链接连上了，accept 就会返回
socket = new Socket(severIp,port);
}
public void start() {
Scanner scanner = new Scanner(System.in);
try (InputStream inputStream = socket.getInputStream();
OutputStream outputStream = socket.getOutputStream()) {
// 将字节流包装成字符流
Scanner scannerFromSocket = new Scanner(inputStream);
PrintWriter printWriter = new PrintWriter(outputStream);
while (true) {
// 1.从键盘上读取用户输入的内容
System.out.print("->");
String request = scanner.next();
// 2.把读取的内容构成请求，发给服务器
// 注意：这里的发送，是带换行的！
printWriter.println(request);
// 写网卡为全缓冲，这里使用flush刷新
printWriter.flush();
// 3.从服务器读取响应内容
String response = scannerFromSocket.next();
// 4. 把响应的结果显示到控制台上
System.out.printf("req: %s ; resq: %s\n",request,response);
}
} catch (IOException e) {
e.printStackTrace();
}
}
// 主方法
public static void main(String[] args) throws IOException {
TcpEchoClient tcpEchoClient = new TcpEchoClient("127.0.0.1",9090);
tcpEchoClient.start();
}
}
3. "Five key points" of using Tcp protocol for network transmission
(1) Customize a simple application layer protocol
. For client and server applications, requests and responses need to agree on a consistent data format. For the sake of simplicity, the following simple agreement is made:
- Each request is a string
- Use \n (line break) to separate requests from requests.
Since it is an echo server, the response and request are exactly the same, so the above rules are followed.
(2) Writing to the network card is fully buffered (writing files is also fully buffered)
为了提高IO效率，引入了缓冲区，使用缓冲区可以减少IO次数，提高整体的效率。上述 printWriter.println(“内容”) 过后，内容就被写入到了缓冲区，如果不刷新缓冲区，就要等到缓冲区满，自动刷新到网卡中，所以执行上述程序可能就会出现只请求不响应的情况，为了解决这个问题，我们可以在每次写网卡后，手动进行刷新：printWriter.flush()
（3）长连接 与 短连接
长连接和短连接是指在网络编程中不同的连接方式。
短连接指客户端与服务器建立连接后，在完成一次请求-响应操作之后就会断开连接。每次请求都需要重新建立连接，这种方式可以保证连接使用的资源较少，但也对服务器的压力较大。常用于小数据量的频繁通信场景，例如HTTP协议。
而长连接则是指客户端与服务器建立连接后，在一段时间内可以保持连接状态，多次请求-响应操作共用这一个连接。这种方式相对于短连接可以减少连接建立、关闭的次数，提高了通信效率，但是缺点是需要维护连接的状态，如果长时间没有交互，则需要进行心跳检测等机制来维持连接状态。常用于对实时性要求较高的通信场景，例如即时通讯、游戏等。
在上述TCP协议中使用到长连接。
（4）使用多线程
上述例子的服务器中使用到了多线程，如果不使用多线程，代码可能产生 BUG。因为上述 start 的 while 循环，是用来循环的接收连接，而下面的 processConnection 内部也有一个循环用来循环的处理连接。假设现在来了一个连接，start 方法接收连接后其中的 processConnection 就开始循环的处理这个连接，直到这个连接关闭，但是如果这个期间又有别的客户端进行新的连接，由于当前start中的第一次循环还没结束，就会导致一直阻塞，使其他连接处理不及时。为了解决上述问题，一个很好的办法就是使用多线程，为每个连接都分配一个线程独立处理。
（5）频繁创建，生命周期又短资源的需要 close 及时释放
- 像上述使用 UDP 协议进行网络通信这种，生命周期伴随整个程序的不需要 close。
- 在这里，使用 TCP 进行网络通信时，服务器那里的每个 Socket 对象只是给一个连接提供服务的，可能会有很多个连接。在这种情况下，服务器会为每个连接都创建一个新的 Socket 对象，作为后续通信的基础。当这个连接不再需要服务时，需要将相应的 Socket 对象关闭，以便及时释放资源。