title: Java NIO 和 AIO Result
date: 2023-05-10 13:21:26
tags:
- NIO
- AIO
categories: - Development knowledge and other
cover: https://cover.png
feature: false
1. NIO
Java NIO (New IO) is an alternative IO API for Java, meaning alternative to the standard Java IO and Java Networking API’s. Java NIO offers a different IO programming model than the traditional IO APIs. Note: Sometimes NIO is claimed to mean Non-blocking IO. However, this is not what NIO meant originally. Also, parts of the NIO APIs are actually blocking - e.g. the file APIs - so the label “Non-blocking” would be slightly misleading.
Java NIO (New IO) is an alternative IO API for Java, meaning a replacement for the standard Java IO and Java Networking API. Java NIO provides a different IO programming model than traditional IO APIs. Note: Sometimes NIO is claimed to mean non-blocking IO. However, that's not what NIO was originally meant to be. Also, some NIO APIs are actually blocking - such as the file API - so the label "non-blocking" is a bit misleading
Original link: Java NIO Tutorial (jenkov.com)
There are three major components in Java NIO : Buffer, Channel, and Selector
1.1 Buffer
A Buffer is essentially a piece of memory, we can write data into this memory, and then get data from this memory
java.nio defines the following Buffer implementations
In fact, the core is the last ByteBuffer . The previous series of classes just wrap it up. We usually use ByteBuffer the most.
We should understand Buffer as an array. IntBuffer, CharBuffer, DoubleBuffer, etc. correspond to int[], char[], double[], etc., respectively. MappedByteBuffer is used to implement memory-mapped file operations.
The following introduces several important properties and methods in Buffer
1.1.1 position、limit、capacity
Just like an array has an array capacity, every time you access an element, you need to specify a subscript. There are also several important attributes in Buffer: position, limit, capacity
The best thing to understand is of course capacity, which represents the capacity of this buffer, and once set, it cannot be changed. For example, an IntBuffer with a capacity of 1024 means that it can store 1024 int values at a time. Once the capacity of the Buffer reaches capacity, the Buffer needs to be cleared before the value can be rewritten
The position and limit are changing, let's look at how they change under the read and write operations
The initial value of position is 0, and every time a value is written into Buffer, position will automatically increase by 1, representing the next writing position. The reading operation is also similar. Every time a value is read, the position will automatically increase by 1. When switching from the write operation mode to the read operation mode (flip ) , the position will return to zero, so that you can read and write from the beginning
Limit : In write operation mode, limit represents the maximum data that can be written. At this time, limit is equal to capacity. After writing, switch to the read mode. At this time, the limit is equal to the actual data size in the Buffer, because the Buffer is not necessarily full
1.1.2 Initialize Buffer
Each Buffer implementation class provides a static method allocate(int capacity)to help us quickly instantiate a Buffer. like:
ByteBuffer byteBuf = ByteBuffer.allocate(1024);
IntBuffer intBuf = IntBuffer.allocate(1024);
LongBuffer longBuf = LongBuffer.allocate(1024);
// ...
In addition, we often use the wrap method to initialize a Buffer
public static ByteBuffer wrap(byte[] array) {
...
}
1.1.3 Filling Buffers
Each Buffer class provides some put methods for filling data into Buffer, such as several put methods in ByteBuffer:
// 填充一个 byte 值
public abstract ByteBuffer put(byte b);
// 在指定位置填充一个 int 值
public abstract ByteBuffer put(int index, byte b);
// 将一个数组中的值填充进去
public final ByteBuffer put(byte[] src) {...}
public ByteBuffer put(byte[] src, int offset, int length) {...}
The above methods need to control the size of the Buffer by themselves, and cannot exceed the capacity, otherwise java.nio.BufferOverflowException will be thrown
For Buffer, another common operation is that we need to fill the data from the Channel into the Buffer. At the system level, this operation is called a read operation, because the data is read from the outside (file or network, etc.) into the memory
int num = channel.read(buf);
The above method will return the size of the data read into the Buffer from the Channel
1.1.4 Extract the value in Buffer
The write operation was introduced earlier. Every time a value is written, the value of position needs to be increased by 1, so the position will finally point to the position behind the last written position. If the Buffer is full, then position is equal to capacity (position starts from 0)
If you want to read the value in Buffer, you need to switch modes, from write mode to read mode. Note that when talking about NIO read operations, we mean reading data from the Channel to the Buffer, which corresponds to the write operation to the Buffer.
Call the flip() method of Buffer to switch from write mode to read mode. In fact, this method is just to set the position and limit values.
public final Buffer flip() {
limit = position; // 将 limit 设置为实际写入的数据数量
position = 0; // 重置 position 为 0
mark = -1; // mark 之后再说
return this;
}
Corresponding to a series of put methods for write operations, read operations provide a series of get methods:
// 根据 position 来获取数据
public abstract byte get();
// 获取指定位置的数据
public abstract byte get(int index);
// 将 Buffer 中的数据写入到数组中
public ByteBuffer get(byte[] dst)
Attached is a frequently used method:
new String(buffer.array()).trim();
Of course, in addition to taking the data out of the Buffer for use, the more common operation is to transfer the data we write to the Channel, such as writing data to a file through a FileChannel, writing data to a network and sending it to a remote machine through a SocketChannel, etc. Correspondingly, this operation, we call it a write operation
int num = channel.write(buf);
1.1.5 mark() & reset()
In addition to the three basic attributes of position, limit, and capacity, another commonly used attribute is mark
mark is used to temporarily save the value of position, and each time mark()the method is called, the mark value will be set as the current position, which is convenient for subsequent use when needed
public final Buffer mark() {
mark = position;
return this;
}
So when do you use it? Consider the following scenario. When the position is 5, we read it first mark(), and then continue to read. When we reach the 10th position, I want to go back to the position where the position is 5 and do it again. Then just adjust the reset()method, and the position will return to 5.
public final Buffer reset() {
int m = mark;
if (m < 0)
throw new InvalidMarkException();
position = m;
return this;
}
1.1.6 rewind() & clear() & compact()
rewind(): It will reset the position to 0, usually used to re-read and write Buffer from the beginning
public final Buffer rewind() {
position = 0;
mark = -1;
return this;
}
clear(): It means to reset the Buffer, which is equivalent to re-instantiation. Usually, we will fill the Buffer first, then read data from the Buffer, and then fill it with new data. We usually call it before refillingclear()
public final Buffer clear() {
position = 0;
limit = capacity;
mark = -1;
return this;
}
compact(): Same as , they all reset several attributes clear()by calling the aforementioned method before preparing to fill the Buffer with new data , but we must see that the method will not clear the data in the Buffer, but subsequent writing will overwrite the original data, which is equivalent to clearing the dataclear()clear()
The method compact()is a little different. After calling this method, it will first process the data that has not been read, that is, the data between position and limit (data that has not been read), and first move these data to the left, and then start writing on this basis. Obviously, at this time limit is still equal to capacity, and position points to the right of the original data
1.2 Channel
All NIO operations start from a channel, which is the source of data or the destination of data writing. Mainly, we will be concerned with the following Channels implemented in the java.nio package:
- FileChannel : file channel for reading and writing files
- DatagramChannel : used for receiving and sending UDP connections
- SocketChannel : understand it as a TCP connection channel, and simply understand it as a TCP client
- ServerSocketChannel : The server corresponding to TCP, which is used to monitor incoming requests from a certain port
The most important thing to pay attention to here is also the focus of SocketChannel and ServerSocketChannel
Channel is often translated as a channel, similar to a stream in IO, for reading and writing. It deals with the Buffer introduced earlier, and fills the data in the Channel into the Buffer during the read operation, and writes the data in the Buffer into the Channel during the write operation
[External link picture transfer failed, the source site may have an anti-leeching mechanism, it is recommended to save the picture and upload it directly (img-n9ksT9gJ-1689582759656)(http://img.fan223.cn/2023/05/20230509111347.png)]
1.2.1 FileChannel
File operations should be the most familiar to everyone, but when we talk about NIO, FileChannel is not the focus of attention. And when we talk about non-blocking later, we will see that FileChannel does not support non-blocking
initialization:
FileInputStream inputStream = new FileInputStream(new File("/data.txt"));
FileChannel fileChannel = inputStream.getChannel();
Of course, we can also get FileChannel from RandomAccessFile#getChannel
Read file content:
ByteBuffer buffer = ByteBuffer.allocate(1024);
int num = fileChannel.read(buffer);
Write the contents of the file:
ByteBuffer buffer = ByteBuffer.allocate(1024);
buffer.put("随机写入一些内容到 Buffer 中".getBytes());
// Buffer 切换为读模式
buffer.flip();
while(buffer.hasRemaining()) {
// 将 Buffer 中的内容写入文件
fileChannel.write(buffer);
}
1.2.2 SocketChannel
As mentioned earlier, we can understand SocketChannel as a TCP client. Although this understanding is a bit narrow, because we will see another way to use it when we introduce ServerSocketChannel
Open a TCP connection:
SocketChannel socketChannel = SocketChannel.open(new InetSocketAddress("https://www.baidu.com", 80));
Of course, the above line of code is equivalent to the following two lines:
// 打开一个通道
SocketChannel socketChannel = SocketChannel.open();
// 发起连接
socketChannel.connect(new InetSocketAddress("https://www.baidu.com", 80));
The reading and writing of SocketChannel is no different from that of FileChannel, that is, the operation buffer
// 读取数据
socketChannel.read(buffer);
// 写入数据到网络连接中
while(buffer.hasRemaining()) {
socketChannel.write(buffer);
}
1.2.3 ServerSocketChannel
It was said before that SocketChannel is a TCP client, and the ServerSocketChannel mentioned here is the corresponding server. ServerSocketChannel is used to monitor the machine port and manage the TCP connection coming in from this port.
// 实例化
ServerSocketChannel serverSocketChannel = ServerSocketChannel.open();
// 监听 8080 端口
serverSocketChannel.socket().bind(new InetSocketAddress(8080));
while (true) {
// 一旦有一个 TCP 连接进来，就对应创建一个 SocketChannel 进行处理
SocketChannel socketChannel = serverSocketChannel.accept();
}
Here we can see the second instantiation method of SocketChannel. At this point, we should be able to understand SocketChannel. It is not just a TCP client, it represents a network channel, which is readable and writable
ServerSocketChannel does not deal with Buffer anymore, because it does not actually process data. Once it receives a request, it instantiates SocketChannel, and then it does not care about the data transfer on this connection channel, because it needs to continue to listen to the port and wait for the next connection
1.2.4 DatagramChannel
UDP is different from TCP, DatagramChannel is a class that handles the server and client
UDP is connectionless-oriented. It does not need to shake hands with the other party, and it does not need to notify the other party. It can directly throw the data packet out. As for whether it can be delivered, it does not know
Listening port:
DatagramChannel channel = DatagramChannel.open();
channel.socket().bind(new InetSocketAddress(9090));
ByteBuffer buf = ByteBuffer.allocate(48);
channel.receive(buf);
1.3 Selector
Selector is built on a non-blocking basis. The multiplexing that you often hear refers to it in the Java world. It is used to implement one thread to manage multiple Channels
1. First, we open a Selector. It can be translated into a selector or a multiplexer
Selector selector = Selector.open();
2. Register the Channel to the Selector. As we said earlier, Selector is built on non-blocking mode, so the Channel registered to Selector must support non-blocking mode, FileChannel does not support non-blocking , we discuss the most common SocketChannel and ServerSocketChannel here
// 将通道设置为非阻塞模式，因为默认都是阻塞模式的
channel.configureBlocking(false);
// 注册
SelectionKey key = channel.register(selector, SelectionKey.OP_READ);
The second int parameter of the register method (using binary flags) is used to indicate which events of interest need to be monitored, and there are four events in total:
- SelectionKey.OP_READ: corresponding to 00000001, there is data in the channel that can be read
- SelectionKey.OP_WRITE: corresponding to 00000100, data can be written to the channel
- SelectionKey.OP_CONNECT: Corresponding to 00001000, successfully established a TCP connection
- SelectionKey.OP_ACCEPT: corresponding to 00010000, accepting TCP connections
We can monitor multiple events in a Channel at the same time. For example, if we want to monitor ACCEPT and READ events, then specify the parameter as binary 000 1 000 1, which is the decimal value 17, to register. The return value of the method is a SelectionKey instance, which contains Channel and Selector information, and also includes an information called Interest Set, which is the set of events we are interested in listening to .
3. Call select()the method to obtain channel information. The Selector operation for judging whether an event we are interested in has occurred is the above 3 steps. A simple example is as follows:
Selector selector = Selector.open();
channel.configureBlocking(false);
SelectionKey key = channel.register(selector, SelectionKey.OP_READ);
while(true) {
// 判断是否有事件准备好
int readyChannels = selector.select();
if(readyChannels == 0) continue;
// 遍历
Set<SelectionKey> selectedKeys = selector.selectedKeys();
Iterator<SelectionKey> keyIterator = selectedKeys.iterator();
while(keyIterator.hasNext()) {
SelectionKey key = keyIterator.next();
if(key.isAcceptable()) {
// a connection was accepted by a ServerSocketChannel.
} else if (key.isConnectable()) {
// a connection was established with a remote server.
} else if (key.isReadable()) {
// a channel is ready for reading
} else if (key.isWritable()) {
// a channel is ready for writing
}
keyIterator.remove();
}
}
For Selector, we also need to be very familiar with the following methods:
select(): Calling this method will copy the SelectionKey corresponding to the prepared channel after the last selection to the selected set. If no channels are ready, this method blocks until at least one channel is readyselectNow(): The function is the same as select, the difference is that if there is no ready channel, then this method will return 0 immediatelyselect(long timeout): After reading the previous two, this should be easy to understand. If no channel is ready, this method will wait for a whilewakeup(): This method is used to wake up the threads waiting onselect()andselect(timeout). Ifwakeup()is called first, and no thread is blocked on select at this time, then a subsequentselect()orselect(timeout)will return immediately without blocking, of course, it will only work once
1.4 Summary
So far, the common interfaces of Buffer, Channel and Selector have been introduced
- Buffer is similar to an array, it has several important attributes such as position, limit, and capacity.
put()Click the data,flip()switch to read mode, then useget()to get the data,clear()clear the data once, and return toput()write data - Channel basically only deals with Buffer, the most important interface is
channel.read(buffer)andchannel.write(buffer) - Selector is used to implement non-blocking IO
2. Blocking mode IO
We have already introduced the ServerSocketChannel, SocketChannel and Buffer required to use the Java NIO package to form a simple client-server network communication. Let's integrate them here and give a complete and operational example:
1. The server, namely ServerSocketChannel
@Slf4j
public class ServerSocketChannelTest {
public static void main(String[] args) throws IOException {
ServerSocketChannel serverSocketChannel = ServerSocketChannel.open();
// 监听 8080 端口进来的 TCP 链接
serverSocketChannel.socket().bind(new InetSocketAddress(8080));
while (true) {
// 这里会阻塞，直到有一个请求的连接进来
SocketChannel socketChannel = serverSocketChannel.accept();
// 开启一个新的线程来处理这个请求，然后在 while 循环中继续监听 8080 端口
SocketHandler handler = new SocketHandler(socketChannel);
new Thread(handler).start();
}
}
}
2. The processor of the new thread, SocketHandler
@Slf4j
public class SocketHandler implements Runnable {
private final SocketChannel socketChannel;
public SocketHandler(SocketChannel socketChannel) {
this.socketChannel = socketChannel;
}
@Override
public void run() {
ByteBuffer buffer = ByteBuffer.allocate(1024);
try {
// 将请求数据读入 Buffer 中
int num;
while ((num = socketChannel.read(buffer)) > 0) {
// 读取 Buffer 内容之前先 flip 一下
buffer.flip();
// 提取 Buffer 中的数据
byte[] bytes = new byte[num];
buffer.get(bytes);
String re = new String(bytes, StandardCharsets.UTF_8);
log.info("收到请求: " + re);
// 回应客户端
ByteBuffer writeBuffer = ByteBuffer.wrap(("我已经收到你的请求, 你的请求内容是: " + re).getBytes());
socketChannel.write(writeBuffer);
buffer.clear();
}
} catch (IOException e) {
IOUtils.closeQuietly(socketChannel);
}
}
}
3. Client, namely SocketChannel
@Slf4j
public class SocketChannelTest {
public static void main(String[] args) throws IOException {
SocketChannel socketChannel = SocketChannel.open();
socketChannel.connect(new InetSocketAddress("localhost", 8080));
// 发送请求
ByteBuffer buffer = ByteBuffer.wrap("1234567890".getBytes());
socketChannel.write(buffer);
// 读取响应
ByteBuffer readBuffer = ByteBuffer.allocate(1024);
int num;
if ((num = socketChannel.read(readBuffer)) > 0) {
readBuffer.flip();
byte[] re = new byte[num];
readBuffer.get(re);
String result = new String(re, StandardCharsets.UTF_8);
log.info("返回值: " + result);
}
}
}
4. Running results
Start the server first, you can see that the server is listening
Then start the client again and receive the message returned by the server
At the same time, the server receives the message sent by the client
Run the client a few more times, you can see a new connection, the server will open a new thread to handle this connection, and all subsequent operations will be completed by that thread
So, where is the performance bottleneck in this mode?
- First of all, it is definitely inappropriate to open a new thread every time a connection comes. Of course, this can be done when the number of active connections is tens of hundreds of thousands, but if the number of active connections is tens of thousands or hundreds of thousands, so many threads are obviously not enough. Each thread requires a certain amount of memory, and the memory will be consumed quickly. At the same time, the overhead of thread switching is very high
- Second, blocking operations are also a problem here. First of all,
accept()it is a blocking operation. Whenaccept()returns, it means that there is a connection available. We will create a new thread to process this SocketChannel immediately, but this does not mean that the other party will transfer the data. Therefore, the SocketChannel#read method will block, waiting for data, obviously this waiting is not worth it. Similarly, the write method also needs to wait for the channel to be writable before performing the write operation, and the blocking waiting here is not worth it
3. Non-blocking I/O
After talking about the use of blocking mode and its shortcomings, we can introduce non-blocking IO here
The core of non-blocking IO is to use a Selector to manage multiple channels, which can be SocketChannel or ServerSocketChannel, register each channel to the Selector, and specify the events to monitor
After that, you can use only one thread to poll the Selector to see if there is a channel on it that is ready. When the channel is ready to be read or writable, then start the actual read and write, so the speed is very fast. There is no need for us to have a thread for each channel
Selector in NIO is an abstraction of the implementation of the underlying operating system. The status of the management channel is actually implemented by the underlying system. Here is a brief introduction to the implementation under different systems
- select : It was implemented in the 1980s. It supports the registration of FD_SETSIZE(1024) sockets. It was definitely enough in that era, but now, it must not work.
- poll : In 1997, poll appeared as a substitute for select. The biggest difference is that poll no longer limits the number of sockets
Both select and poll have a common problem, that is, they will only tell you how many channels are ready, but they will not tell you which channels . Therefore, once you know that a channel is ready, you still need to perform a scan. Obviously, this is not very good. It is okay when there are few channels. Once the number of channels is more than hundreds of thousands, the time for one scan is considerable, and the time complexity is O(n). Therefore, it was later that the following implementation was born
- epoll : Released with the Linux kernel 2.5.44 in 2002, epoll can directly return the specific prepared channel, and the time complexity is O(1)
In addition to epoll in Linux, Kqueue appeared in FreeBSD in 2000 , and there is /dev/poll in Solaris
There are so many implementations mentioned above, but there is no Windows. The non-blocking IO of the Windows platform uses select. We don’t have to think that Windows is very backward. The asynchronous IO provided by IOCP in Windows is relatively powerful.
Back to Selector. After all, JVM is such a platform that shields the underlying implementation. We can program for Selector. When introducing Selector, we have already understood its basic usage. Here is a runnable example code, as follows:
@Slf4j
public class SelectorServer {
public static void main(String[] args) throws IOException {
Selector selector = Selector.open();
ServerSocketChannel server = ServerSocketChannel.open();
server.socket().bind(new InetSocketAddress(8080));
// 将其注册到 Selector 中，监听 OP_ACCEPT 事件
server.configureBlocking(false);
server.register(selector, SelectionKey.OP_ACCEPT);
while (true) {
int readyChannels = selector.select();
if (readyChannels == 0) {
continue;
}
Set<SelectionKey> readyKeys = selector.selectedKeys();
// 遍历
Iterator<SelectionKey> iterator = readyKeys.iterator();
while (iterator.hasNext()) {
SelectionKey key = iterator.next();
iterator.remove();
if (key.isAcceptable()) {
// 有已经接受的新的到服务端的连接
SocketChannel socketChannel = server.accept();
// 有新的连接并不代表这个通道就有数据，
// 这里将这个新的 SocketChannel 注册到 Selector，监听 OP_READ 事件，等待数据
socketChannel.configureBlocking(false);
socketChannel.register(selector, SelectionKey.OP_READ);
} else if (key.isReadable()) {
// 有数据可读
// 上面一个 if 分支中注册了监听 OP_READ 事件的 SocketChannel
SocketChannel socketChannel = (SocketChannel) key.channel();
ByteBuffer readBuffer = ByteBuffer.allocate(1024);
try {
int num = socketChannel.read(readBuffer);
if (num > 0) {
// 处理进来的数据...
log.info("收到数据：" + new String(readBuffer.array()).trim());
ByteBuffer buffer = ByteBuffer.wrap("返回给客户端的数据...".getBytes());
socketChannel.write(buffer);
} else if (num == -1) {
// -1 代表连接已经关闭
socketChannel.close();
}
} catch (IOException e) {
socketChannel.close();
}
}
}
}
}
}
The client code can use the previous one, and then the running result is the same as the previous one, except that the new connection does not create a new thread to complete
4. NIO.2 Different Step IO (AIO)
More New IO, or NIO.2, was released with JDK 1.7, including the introduction of asynchronous IO interfaces and file access interfaces such as Paths
The word asynchronous, I think it is familiar to most developers, we will use asynchronous in many scenarios
Usually, we will have a thread pool for executing asynchronous tasks. The thread that submits the task submits the task to the thread pool and returns immediately without waiting until the task is actually completed. If you want to know the execution result of the task, usually by passing a callback function, call this function after the task ends
The same principle applies to asynchronous IO in Java. A thread pool is responsible for performing tasks, and then use callbacks or query results by yourself. Asynchronous IO is mainly to control the number of threads, reduce memory consumption caused by too many threads and CPU overhead on thread scheduling
In Unix/Linux and other systems, JDK uses the thread pool in the concurrent package to manage tasks . For details, you can view the source code of AsynchronousChannelGroup
In the Windows operating system, a solution called I/O Completion Ports is provided , usually referred to as IOCP . The operating system is responsible for managing the thread pool, and its performance is very good. Therefore, in Windows, the JDK directly adopts the support of IOCP , uses system support, and exposes more operating information to the operating system, which also enables the operating system to optimize our IO to a certain extent.
In fact, there is an asynchronous IO system implemented in Linux, but there are many restrictions and the performance is average, so JDK adopts the method of self-built thread pool
There are a total of three classes that need our attention, namely AsynchronousSocketChannel , AsynchronousServerSocketChannel and AsynchronousFileChannel , but a prefix Asynchronous is added to the class names of FileChannel, SocketChannel and ServerSocketChannel introduced earlier
Java asynchronous IO provides two usage methods, namely returning a Future instance and using a callback function
4.1 Return Future instance
We should be familiar with the way of returning java.util.concurrent.Future instances, which is how the JDK thread pool is used. The semantics of several methods of the Future interface are also common here, so here is a brief introduction
future.isDone();: Determine whether the operation has been completed, including normal completion, exception throwing, cancellationfuture.cancel(true);: Cancel the operation by interrupt. The parameter true says that even if the task is executing, it will be interruptedfuture.isCancelled();: Whether to be canceled, this method will return true only if the task is canceled before it ends normallyfuture.get();: Get the execution result, blockfuture.get(10, TimeUnit.SECONDS);: If you are not satisfied with the blocking of the aboveget()method, then set a timeout
4.2 Provide CompletionHandler callback function
java.nio.channels.CompletionHandler interface definition:
public interface CompletionHandler<V,A> {
void completed(V result, A attachment);
void failed(Throwable exc, A attachment);
}
Note that there is an attachment on the parameter, although it is not commonly used, we can pass this parameter value in each supported method
AsynchronousServerSocketChannel listener = AsynchronousServerSocketChannel.open().bind(null);
// accept 方法的第一个参数可以传递 attachment
listener.accept(attachment, new CompletionHandler<AsynchronousSocketChannel, Object>() {
public void completed(
AsynchronousSocketChannel client, Object attachment) {
//
}
public void failed(Throwable exc, Object attachment) {
//
}
});
4.3 AsynchronousFileChannel
Asynchronous file IO, as we said earlier, file IO does not support non-blocking mode in all operating systems, but we can use asynchronous methods for file IO to improve performance
The following are some important interfaces in AsynchronousFileChannel:
Instantiate:
AsynchronousFileChannel channel = AsynchronousFileChannel.open(Paths.get("/Users/hongjie/test.txt"));
Once the instantiation is complete, we can prepare to read the data into the Buffer :
ByteBuffer buffer = ByteBuffer.allocate(1024);
Future<Integer> result = channel.read(buffer, 0);
Both the read and write operations of the asynchronous file channel need to provide a file start position, and the file start position is 0
In addition to using the method of returning a Future instance, you can also use a callback function to operate. The interface is as follows:
public abstract <A> void read(ByteBuffer dst,
long position,
A attachment,
CompletionHandler<Integer,? super A> handler);
By the way, post the interface of the two versions of the write operation:
public abstract Future<Integer> write(ByteBuffer src, long position);
public abstract <A> void write(ByteBuffer src,
long position,
A attachment,
CompletionHandler<Integer,? super A> handler);
We can see that the reading and writing of AIO mainly deals with Buffer, which is in the same line as NIO. In addition, it also provides a method for flushing data in memory to disk:
public abstract void force(boolean metaData) throws IOException;
Because of our write operations on files, the operating system does not directly operate on the files, the system will cache them, and then periodically flush them to the disk. If you want to write data to disk in time to avoid partial data loss caused by power failure, you can call this method. If the parameter is set to true, it means that the file attribute information will also be updated to the disk
Also, the file locking function is also provided, we can lock part of the data of the file, so that exclusive operations can be performed
public abstract Future<FileLock> lock(long position, long size, boolean shared);
position is the starting position of the content to be locked, size indicates the size of the area to be locked, and shared indicates whether a shared lock or an exclusive lock is required
Of course, a version with a callback function is also available:
public abstract <A> void lock(long position,
long size,
boolean shared,
A attachment,
CompletionHandler<FileLock,? super A> handler);
The file locking function also provides a tryLock method, which will return the result quickly:
public abstract FileLock tryLock(long position, long size, boolean shared)
throws IOException;
This method is very simple, just try to acquire the lock, if the area has been locked by other threads or other applications, then return null immediately, otherwise return the FileLock object
4.4 AsynchronousServerSocketChannel
This class corresponds to the ServerSocketChannel of non-blocking IO, which can be used by analogy
@Slf4j
public class AsynchronousServer {
public static void main(String[] args) throws IOException {
// 实例化，并监听端口
AsynchronousServerSocketChannel server =
AsynchronousServerSocketChannel.open().bind(new InetSocketAddress(8080));
// 自己定义一个 Attachment 类，用于传递一些信息
Attachment att = new Attachment();
att.setServer(server);
server.accept(att, new CompletionHandler<AsynchronousSocketChannel, Attachment>() {
@Override
public void completed(AsynchronousSocketChannel client, Attachment att) {
try {
SocketAddress clientAddr = client.getRemoteAddress();
log.info("收到新的连接：" + clientAddr);
// 收到新的连接后，server 应该重新调用 accept 方法等待新的连接进来
att.getServer().accept(att, this);
Attachment newAtt = new Attachment();
newAtt.setServer(server);
newAtt.setClient(client);
newAtt.setReadMode(true);
newAtt.setBuffer(ByteBuffer.allocate(2048));
// 这里也可以继续使用匿名实现类，不过代码不好看，所以这里专门定义一个类
client.read(newAtt.getBuffer(), newAtt, new ChannelHandler());
} catch (IOException ex) {
ex.printStackTrace();
}
}
@Override
public void failed(Throwable t, Attachment att) {
log.info("accept failed");
}
});
// 为了防止 main 线程退出
try {
Thread.currentThread().join();
} catch (InterruptedException e) {
Thread.currentThread().interrupt();
}
}
}
ChannelHandler server processor
@Slf4j
public class ChannelHandler implements CompletionHandler<Integer, Attachment> {
@Override
public void completed(Integer result, Attachment att) {
if (att.isReadMode()) {
// 读取来自客户端的数据
ByteBuffer buffer = att.getBuffer();
buffer.flip();
byte[] bytes = new byte[buffer.limit()];
buffer.get(bytes);
String msg = new String(buffer.array()).trim();
log.info("收到来自客户端的数据: " + msg);
// 响应客户端请求，返回数据
buffer.clear();
buffer.put("Response from server!".getBytes(Charset.forName(StandardCharsets.UTF_8.toString())));
att.setReadMode(false);
buffer.flip();
// 写数据到客户端也是异步
att.getClient().write(buffer, att, this);
} else {
// 到这里，说明往客户端写数据也结束了，有以下两种选择:
// 1. 继续等待客户端发送新的数据过来
// att.setReadMode(true);
// att.getBuffer().clear();
// att.getClient().read(att.getBuffer(), att, this);
try {
// 2. 既然服务端已经返回数据给客户端，断开这次的连接
att.getClient().close();
} catch (IOException ignored) {
}
}
}
@Override
public void failed(Throwable exc, Attachment attachment) {
log.info("连接断开");
}
}
Custom Attachment class
@Data
public class Attachment {
private AsynchronousServerSocketChannel server;
private AsynchronousSocketChannel client;
private boolean isReadMode;
private ByteBuffer buffer;
}
4.5 AsynchronousSocketChannel
AsynchronousSocketChannel is basically similar to the previous non-blocking IO
public class AsynchronousClient {
public static void main(String[] args) throws ExecutionException, InterruptedException, IOException {
AsynchronousSocketChannel client = AsynchronousSocketChannel.open();
// 来个 Future 形式的
Future<?> future = client.connect(new InetSocketAddress("127.0.0.1", 8080));
// 阻塞一下，等待连接成功
future.get();
Attachment att = new Attachment();
att.setClient(client);
att.setReadMode(false);
att.setBuffer(ByteBuffer.allocate(2048));
byte[] data = "I am obot!".getBytes();
att.getBuffer().put(data);
att.getBuffer().flip();
// 异步发送数据到服务端
client.write(att.getBuffer(), att, new ClientChannelHandler());
// 这里休息一下再退出，给出足够的时间处理数据
Thread.sleep(2000);
}
}
ClientChannelHandler client handler
@Slf4j
public class ClientChannelHandler implements CompletionHandler<Integer, Attachment> {
@Override
public void completed(Integer result, Attachment att) {
ByteBuffer buffer = att.getBuffer();
if (att.isReadMode()) {
// 读取来自服务端的数据
buffer.flip();
byte[] bytes = new byte[buffer.limit()];
buffer.get(bytes);
String msg = new String(bytes, Charset.forName(StandardCharsets.UTF_8.toString()));
log.info("收到来自服务端的响应数据: " + msg);
// 接下来，有以下两种选择:
// 1. 向服务端发送新的数据
// att.setReadMode(false);
// buffer.clear();
// String newMsg = "new message from client";
// byte[] data = newMsg.getBytes(Charset.forName("UTF-8"));
// buffer.put(data);
// buffer.flip();
// att.getClient().write(buffer, att, this);
try {
// 2. 关闭连接
att.getClient().close();
} catch (IOException ignored) {
}
} else {
// 写操作完成后，会进到这里
att.setReadMode(true);
buffer.clear();
att.getClient().read(buffer, att, this);
}
}
@Override
public void failed(Throwable exc, Attachment attachment) {
log.info("服务器无响应");
}
}
operation result:
[External link picture transfer failed, the source site may have an anti-leeching mechanism, it is recommended to save the picture and upload it directly (img-8EP3nyQD-1689582759658)(http://img.fan223.cn/2023/05/20230509163807.png)]
4.6 Asynchronous Channel Groups
We said before that there must be a thread pool for asynchronous IO, which is responsible for receiving tasks, processing IO events, callbacks, etc. This thread pool is inside the group, once the group is closed, the corresponding thread pool will be closed
AsynchronousServerSocketChannels and AsynchronousSocketChannels belong to the group. When we call open()the method of AsynchronousServerSocketChannel or AsynchronousSocketChannel, the corresponding channel belongs to the default group, which is automatically constructed and managed by the JVM
If we want to configure this default group, we can specify the following system variables in the JVM startup parameters:
java.nio.channels.DefaultThreadPool.threadFactory: This system variable is used to set the ThreadFactory, which should be the fully qualified class name of the java.util.concurrent.ThreadFactory implementation class. Once we specify this ThreadFactory, the threads in the group will be generated using this classjava.nio.channels.DefaultThreadPool.initialSize: This system variable is also well understood and is used to set the initial size of the thread pool
Maybe you will want to use your own defined group, so that you can have more control over the threads in it, just use the following methods:
AsynchronousChannelGroup.withCachedThreadPool(ExecutorService executor, int initialSize)AsynchronousChannelGroup.withFixedThreadPool(int nThreads, ThreadFactory threadFactory)AsynchronousChannelGroup.withThreadPool(ExecutorService executor)
Readers who are familiar with the thread pool should have a good understanding of these methods. They are all static methods in AsynchronousChannelGroup. As for the use of group, it is very simple.
AsynchronousChannelGroup group = AsynchronousChannelGroup
.withFixedThreadPool(10, Executors.defaultThreadFactory());
AsynchronousServerSocketChannel server = AsynchronousServerSocketChannel.open(group);
AsynchronousSocketChannel client = AsynchronousSocketChannel.open(group);
AsynchronousFileChannels do not belong to group . But they are also associated with a thread pool. If not specified, the default thread pool of the system will be used. If you want to use the specified thread pool, you can use the following method when instantiating:
public static AsynchronousFileChannel open(Path file,
Set<? extends OpenOption> options,
ExecutorService executor,
FileAttribute<?>... attrs) {
...
}