Netty series of notes: NIO core component Buffer

I. Overview

The essence of Buffer is actually a piece of memory, analogous to the common ByteBuffer, which can be simply understood as a Byte array. Java NIO encapsulates this memory into a Buffer object and provides a series of properties and methods to facilitate data interaction between Buffer and Channel.

Two, usage

Let's look at the FileChannel operation:

try (RandomAccessFile accessFile =
                     new RandomAccessFile("/demo.txt", "rw");) {
            // 获取 FileChannel
            FileChannel channel = accessFile.getChannel();
            // Buffer 分配空间
            ByteBuffer buffer = ByteBuffer.allocate(1024);
            // 从 channel 中读取数据到 buffer 中
            int readBytes = channel.read(buffer);
            System.out.println("读到 " + readBytes + " 字节");
            // 判断是否到文件结尾
            while (readBytes != -1) {
                buffer.flip();
                // 若 buffer 中还有数据
                while (buffer.hasRemaining()) {
                    System.out.println((char) buffer.get());
                }
                buffer.compact();
                readBytes = channel.read(buffer);
            }
        } catch (Exception e) {
            e.printStackTrace();
        }

From the above example, there are the following steps to use Buffer:

1. Allocate space

• allocate(int capacity)
  Buffer is an abstract class. Each Buffer implementation class provides allocate(int capacity) static method to help instantiate Buffer objects quickly. For example, the common ByteBuffer: public static ByteBuffer allocate (int capacity) {if (capacity <0) throw new IllegalArgumentException(); return new HeapByteBuffer(capacity, capacity); }It can also be seen from its name that the space allocated by this method is Based on heap memory.
• allocateDirect(int capacity)
  allocates space based on off-heap memory and instantiates a Buffer object. public static ByteBuffer allocateDirect (int capacity) {return new DirectByteBuffer(capacity);}
• wrap(byte[] array,int offset, int length)
  Each Buffer implementation class provides a wrap method to wrap an array into a Buffer instance.

2. Read data from Channel to Buffer

• Use channel.read(buffer) method
• All
  implementations of Buffer that use the buffer.put() method provide the put() method to put data into the Buffer.

3. Call the flip() method

4. Get data from Buffer

channel.write(buffer)
buffer.get()

5. Call the clear() method or compact() method

What is the function of the above flip(), clear(), compact() methods?

Three, source code analysis

In addition to the array used to store data, Buffer has several important properties:

public abstract class Buffer {

    // Invariants: mark <= position <= limit <= capacity
    private int mark = -1;
    private int position = 0;
    private int limit;
    private int capacity;

    // Used only by direct buffers
    // NOTE: hoisted here for speed in JNI GetDirectBufferAddress
    long address;

    Buffer(int mark, int pos, int lim, int cap) {       // package-private
        if (cap < 0)
            throw new IllegalArgumentException("Negative capacity: " + cap);
        this.capacity = cap;
        limit(lim);
        position(pos);
        if (mark >= 0) {
            if (mark > pos)
                throw new IllegalArgumentException("mark > position: ("
                                                   + mark + " > " + pos + ")");
            this.mark = mark;
        }
    }

    // ... 省略具体方法的代码
}

Based on the above four attributes, Buffer completes read and write operations by controlling their positions. We analyze from the use of Buffer:

1. Allocate space

Assuming that the ByteBuffer.allocate(10) method is called to allocate 10 bytes of space for the buffer, look at the code of allocate(int capacity):

public static ByteBuffer allocate(int capacity) {
        if (capacity < 0)
            throw new IllegalArgumentException();
        return new HeapByteBuffer(capacity, capacity);
}

It calls the construction method of HeapByteBuffer, and sets capacity and limit to 10 and position to 0.

HeapByteBuffer(int cap, int lim) {            // package-private

        super(-1, 0, lim, cap, new byte[cap], 0);
        /*
        hb = new byte[cap];
        offset = 0;
        */
    }

The schematic diagram of the buffer at this time is as follows:

In the initial state of the buffer, position is 0 , and limit and capacity point to 9 .

2. Write data to Buffer

We write three bytes of data to the buffer, and look at the HeapByteBuffer#put(byte b) method, one of the implementations of ByteBuffer:

public ByteBuffer put(byte x) {

        hb[ix(nextPutIndex())] = x;
        return this;

}

protected int ix(int i) {
        return i + offset;
}

The nextPutIndex() method in ByteBuffer is used to calculate the index of the next written data:

final int nextPutIndex() {                          // package-private
        if (position >= limit)
            throw new BufferOverflowException();
        return position++;
}

We came to the conclusion that every time a byte of data is put in, position increases by 1 .

At this time, the orientation of the three attributes changes as follows:

After writing three bytes of data, position points to the next operable position 3 , and the positions of limit and capacity remain unchanged.

We will state at this time is called a buffer write mode :

In write mode, limit and capacity are equal.

3. Read data from Buffer

Next, we read the upper three bytes of data in the Buffer, so how do we locate these three bytes of data? Recalling the use of Buffer above, you need to call the Buffer#flip() method before reading data:

public final Buffer flip() {
	// 将 limit 设置为当前数据大小的下一个坐标
        limit = position;
	// position 设置为 0
        position = 0;
	// 如果有标记还原为默认值
        mark = -1;
        return this;
}

After the above operations, we have obtained the start and end range of the data. At this time, the schematic diagram of the buffer is as follows:

We will state at this time is called the buffer read mode :

In read mode, limit is equal to the actual size of the buffer.

By comparing the read mode and write mode of the buffer, we found that by controlling the limit, the read and write of the buffer can be switched flexibly.

4. clear() and compact() methods

Through the clear() and compact() methods, you can switch from read mode to write mode.

• clear() 方法public final Buffer clear() { position = 0; limit = capacity; mark = -1; return this; }

The clear() method sets position to 0, limit to capacity, and mark to -1. That is, the Buffer is restored to the initial recognition state. At this time, the data still exists in the Buffer, but it is impossible to determine which data has been read and which data has not been read.

• The compact()
  cpmpact() method is in the implementation class of Buffer. Take ByteBuffer's HeapByteBuffer implementation as an example: public ByteBuffer compact() {// copy the original array to the beginning of the Buffer System.arraycopy(hb, ix(position()), hb, ix(0), remaining()) ; // position is set to behind the last unread element position(remaining()); // limit is set to capacitylimit(capacity()); // cancel mark discardMark(); return this; }It can be seen from the code that compact The () method copies all unread data to the beginning of the Buffer, then sets the position after the last unread element, and the limit is set to capacity. At this time Buffer has switched to write mode, but it will not overwrite unread data.

5. mark() and reset() methods

• The mark() method public final Buffer mark() {mark = position; return this;} The mark() method uses the mark attribute to store the subscript position of the current mark.
• reset() method public final Buffer reset () {int m = mark; if (m <0) throw new InvalidMarkException(); position = m; return this; }reset() method restores the position property to the value of mark. The two methods cooperate with each other to record and restore the value of position in order to read and write from the marked position.

6, rewind() method

public final Buffer rewind() {
        position = 0;
        mark = -1;
        return this;
    }

The rewind() method sets the position to 0 and clears the mark. At this time, limit remains unchanged. In read mode, all data can be re-read.

Four, conclusion

This is the end of the discussion about Buffer. There are some methods that are relatively simple, and interested friends can learn about them by themselves.