Understanding the three cores of Java NIO (Buffer, Channel, Selector)

I amMr. Aojiaolu, a person who accumulates, learns, shares and grows.

If you think the content of the article is good, I hope you will not hesitate to "one-click three links". If there are shortcomings in the article, I hope you can add your doubts, opinions and weird questions encountered in the interview in the comment area.

memories

1. Buffer mechanism and subclasses

1. Basic introduction to Buffer

2. Basic introduction to Channel

1. FileChannel class

2. Use FileChannel to write text files

3. Use FileChannel to read text files

4. Use FileChannel to copy files

5. Use transferFrom to copy files

3. Notes on Channel and Buffer

4. Selector

1. Basic introduction to Selector

2. Selector features

3. Selector common methods

4. Selector related method description

5. Analysis of NIO non-blocking network programming process

NIO non-blocking network programming code example

operation result

1. Buffer mechanism and subclasses

1. Basic introduction to Buffer

The buffer is essentially a memory block that can read and write data. It can be understood as a 容器对象（含数组）. This object provides 一组方法 and can be changed. To easily work with memory blocks, buffer objects have built-in mechanisms to track and record buffer state changes.

Channel provides a channel for reading data from files and networks, but reading or reading must go through Buffer.

An array of corresponding type is maintained in the Buffer subclass to store data:

public abstract class IntBuffer extends Buffer implements Comparable<IntBuffer>{

    // These fields are declared here rather than in Heap-X-Buffer in order to
    // reduce the number of virtual method invocations needed to access these
    // values, which is especially costly when coding small buffers.
    //
    final int[] hb;                  // Non-null only for heap buffers
    final int offset;
    boolean isReadOnly;                 // Valid only for heap buffers

    // Creates a new buffer with the given mark, position, limit, capacity,
    // backing array, and array offset
    //
    IntBuffer(int mark, int pos, int lim, int cap,   // package-private
                 int[] hb, int offset)
    {
        super(mark, pos, lim, cap);
        this.hb = hb;
        this.offset = offset;
    }

    // Creates a new buffer with the given mark, position, limit, and capacity
    //
    IntBuffer(int mark, int pos, int lim, int cap) { // package-private
        this(mark, pos, lim, cap, null, 0);
    }
}

Buffer common subclasses	describe
ByteBuffer	Store byte data into buffer
ShortBuffer	Store string data into buffer
CharBuffer	Store character data into buffer
IntBuffer	Store integer data in a buffer
LongBuffer	Store long integer data in buffer
DoubleBuffer	Store floating point data in the buffer
FloatBuffer	Store floating point data in the buffer

Four properties are defined in Buffer to provide the contained data elements.

// Invariants: mark <= position <= limit <= capacity
private int mark = -1;
private int position = 0;
private int limit;
private int capacity;

capacity	Capacity, that is, the maximum amount of data that can be accommodated; it is specified when the buffer is created and cannot be modified.
limit	Indicates the current end point of the buffer. Read and write operations cannot be performed on the buffer beyond the limit, but the limit can be modified.
position	The current position, the next index to be read or written, this value will be changed every time the buffer data is read or written to prepare for the next read or write.
Mark	Mark the current position position, and return to the marked position after reset.

2. Basic introduction to Channel

NIO channels are similar to streams, but have the following differences:

Channels are bidirectional and can be read and written, while streams are unidirectional and can only be read or written.
Channels can read and write data asynchronously.
Channels can read data from buffers and write data to buffers.

Commonly used Channels include: FileChannel, DatagramChannel, SocketChannel, and SocketServerChannel.

1. FileChannel class

FileChannel is mainly used to perform IO operations on local files. Common methods are:

public int read(ByteBuffer dst): Read data from the channel into the buffer.
public int write(ByteBuffer src): Write the data in the buffer to the channel.
public long transferFrom(ReadableByteChannel src, long position, long count): Copy data from the target channel to the current channel.
public long transferTo(long position,long count,WriteableByteChannel target): Copy data from the current channel to the target channel.

2. Use FileChannel to write text files

public class NIOFileChannel {

    public static void main(String[] args) throws IOException {
        String str = "Hello,Java菜鸟程序员";
        //创建一个输出流
        FileOutputStream fileOutputStream = new FileOutputStream("hello.txt");
        //获取通道
        FileChannel channel = fileOutputStream.getChannel();
        //创建缓冲区
        ByteBuffer byteBuffer = ByteBuffer.allocate(100);
        //写入byteBuffer
        byteBuffer.put(str.getBytes());
        //切换模式
        byteBuffer.flip();
        //写入通道
        channel.write(byteBuffer);
        //关闭
        channel.close();
        fileOutputStream.close();
    }
}

3. Use FileChannel to read text files

public class NIOFileChannel {
    public static void main(String[] args) throws IOException {
      FileInputStream fileInputStream = new FileInputStream("hello.txt");
      FileChannel channel = fileInputStream.getChannel();
      ByteBuffer byteBuffer = ByteBuffer.allocate(100);
      channel.read(byteBuffer);
      System.out.println(new String(byteBuffer.array(), 0, byteBuffer.limit()));     
      //Hello,Java菜鸟程序员
      channel.close();
      fileInputStream.close();
    }
}

4. Use FileChannel to copy files

public class NIOFileChannel {

    public static void main(String[] args) throws IOException {
        FileInputStream fileInputStream = new FileInputStream("hello.txt");
        FileOutputStream fileOutputStream = new FileOutputStream("world.txt");
        FileChannel inChannel = fileInputStream.getChannel();
        FileChannel outChannel = fileOutputStream.getChannel();
        ByteBuffer byteBuffer = ByteBuffer.allocate(1);
        while (inChannel.read(byteBuffer) != -1) {
            byteBuffer.flip();
            outChannel.write(byteBuffer);
            //清空重置
            byteBuffer.clear();
        }
        fileOutputStream.close();
        fileInputStream.close();
    }
}

5. Use transferFrom to copy files

public class NIOFileChannel {

    public static void main(String[] args) throws IOException {
        FileInputStream fileInputStream = new FileInputStream("hello.txt");
        FileOutputStream fileOutputStream = new FileOutputStream("world.txt");
        FileChannel inChannel = fileInputStream.getChannel();
        FileChannel outChannel = fileOutputStream.getChannel();
        //从哪拷贝,从几开始到几结束 对应的还有transferTo()方法.
        outChannel.transferFrom(inChannel, 0, inChannel.size());
        outChannel.close();
        inChannel.close();
        fileOutputStream.close();
        fileInputStream.close();
    }
}

3. Notes on Channel and Buffer

ByteBuffer supports typed put and get. Whatever data type is put into put, get should use the corresponding data type to retrieve it, otherwise a ByteUnderflowException exception may occur.
You can convert a normal Buffer into a read-only Buffer: asReadOnlyBuffer() method.
NIO provides MapperByteBuffer, which allows files to be modified directly in memory (off-heap memory), and how to synchronize to files is completed by NIO.
NIO also supports reading and writing operations through multiple Buffers (i.e. Buffer arrays), namely Scattering and Gathering.
- Scattering(分散): When writing data to the buffer, you can use the Buffer array to write sequentially. After one Buffer array is full, continue writing to the next Buffer array.
- Gathering(聚集): When reading data from the buffer, you can read it in sequence. After reading one Buffer, you can read the next one in sequence.

4. Selector

1. Basic introduction to Selector

Java's NIO uses non-blocking I/O. You can use one thread to handle several client connections, and you will use a Selector.
Selector can detect whether events occur on multiple registered channels (multiple Channels are registered to the same selector in the form of events), and if an event occurs, the event is obtained Then handle each event accordingly.
Reading and writing will only occur when there are actual read and write events on the connection, which reduces system overhead and eliminates the need to create a thread for each connection and maintain multiple threads.
The overhead caused by context switching between multiple threads is avoided.

2. Selector features

Netty's I/O thread NioEventLoop aggregates Selectors (selectors/multiplexers) and can handle hundreds or thousands of client connections concurrently.

When a thread reads or writes from a client Socket channel, if no data is available, the thread can perform other tasks.

Threads typically use the idle time of non-blocking I/O to perform I/O operations on other channels, so a single thread can manage multiple input and output channels.

Since read and write operations are non-blocking, the operating efficiency of the I/O thread can be fully improved and thread suspension caused by frequent I/O blocking can be avoided.

One I/O thread can concurrently process N client connections and read and write operations, which fundamentally solves the traditional synchronous blocking I/O one-connection-one-thread model, and greatly improves architectural performance, elastic scalability, and reliability. .

3. Selector common methods

public abstract class Selector implement Closeable{

    public static Selector open(); //得到一个选择器对象

    public int select(long timeout); //监控所有注册的通道，当其中的IO操作可以进行时，将对应的selectionkey加入内部集合并返回，参数设置超时时间

    public Set<SelectionKey> selectionKeys(); //从内部集合中得到所有的SelectionKey

}

4. Selector related method description

selector.select()://If no event is detected in the registration pipeline, it will continue to block.
selector.select(1000)://Block for 1000 milliseconds and return after 1000 milliseconds
selector.wakeup()://wake up selector
selector.selectNow()： //Do not block, return immediately

5. Analysis of NIO non-blocking network programming process

When the client connects, it will get the corresponding SocketChannel through SeverSocketChannel.
The Selector listens, calls the select() method, and returns the number of channels in which events have occurred among all channels registered with the Selector.
Register socketChannel to Selector,public final SelectionKey register(Selector sel, int ops), multiple SocketChannels can be registered on one selector .
Returns a SelectionKey after registration, which will be associated with the Selector (in the form of set).
Further obtain each SelectionKey, and an event occurs.
Then obtain the SocketChannel through SelectionKey in reverse, using the channel() method.
Business processing can be completed through the obtained channel.

Four operation flags are defined in SelectionKey: OP_READ indicates that a read event occurs in the channel; OP_WRITE —Indicates that a write event occurs in the channel; OP_CONNECT —Indicates establishing a connection; OP_ACCEPT —Requests a new connection.

NIO non-blocking network programming code example

public class Server {

    public static void main(String[] args) throws IOException {
        //创建serverSocketChannel
        ServerSocketChannel serverSocketChannel = ServerSocketChannel.open();
        //绑定端口
        serverSocketChannel.socket().bind(new InetSocketAddress(6666));
        //设置为非阻塞
        serverSocketChannel.configureBlocking(false);
        //得到Selector对象
        try (Selector selector = Selector.open()) {
            //把ServerSocketChannel注册到selector，事件为OP_ACCEPT
            serverSocketChannel.register(selector, SelectionKey.OP_ACCEPT);
            //如果返回的>0，表示已经获取到关注的事件
            while (selector.select() > 0) {
                Set<SelectionKey> selectionKeys = selector.selectedKeys();
                Iterator<SelectionKey> iterator = selectionKeys.iterator();
                while (iterator.hasNext()) {
                    //获得到一个事件
                    SelectionKey next = iterator.next();
                    //如果是OP_ACCEPT，表示有新的客户端连接
                    if (next.isAcceptable()) {
                        //给该客户端生成一个SocketChannel
                        SocketChannel accept = serverSocketChannel.accept();
                        accept.configureBlocking(false);
                        //将当前的socketChannel注册到selector，关注事件为读事件，同时给socket Channel关联一个buffer
                        accept.register(selector, SelectionKey.OP_READ,ByteBuffer.allocate(1024));
                        System.out.println("获取到一个客户端连接");
                    //如果是读事件
                    } else if (next.isReadable()) {
                        //通过key 反向获取到对应的channel
                        SocketChannel channel = (SocketChannel) next.channel();
                        //获取到该channel关联的buffer
                        ByteBuffer buffer = (ByteBuffer) next.attachment();
                        while (channel.read(buffer) != -1) {
                            buffer.flip();
                            System.out.println(new String(buffer.array(), 0, buffer.limit()));
                            buffer.clear();
                        }
                    }
                    iterator.remove();
                }
            }
        }
    }

}

public class Client {

    public static void main(String[] args) throws IOException {
        //得到一个网络通道
        SocketChannel socketChannel = SocketChannel.open();
        //设置为非阻塞
        socketChannel.configureBlocking(false);
        //提供服务器端的IP和端口
        InetSocketAddress inetSocketAddress = new InetSocketAddress("127.0.0.1", 6666);
        //连接服务器
        if (!socketChannel.connect(inetSocketAddress)) {
            while (!socketChannel.finishConnect()) {
                System.out.println("连接需要时间,客户端不会阻塞...先去吃个宵夜");
            }
        }
        //连接成功,发送数据
        String str = "hello,Java菜鸟程序员";
        ByteBuffer byteBuffer = ByteBuffer.wrap(str.getBytes());
        socketChannel.write(byteBuffer);
        socketChannel.close();
        System.out.println("客户端退出");
    }

}

operation result

The series of articles continues to be updated. Search "Mr. Proud Deer " on WeChat and reply [ Interview】Prepared interview materials for first-tier manufacturers.