【笔记仪：itwhite horse，
视频语：Java NIO Netty configuration completed】
NIO basics
**Note:** It is recommended to complete the JavaSE and JavaWeb chapters before starting this part of the study. If you have completed the JVM chapter before then, it will seem easier.
In the study of JavaSE, we learned how to use IO for data transmission. Java IO is blocking. If the data is not ready when a read and write data call is made, or it is not currently writable, then the read and write operations will be blocked until Until the data is ready or the target is writable. Java NIO is non-blocking. Every data read and write call will return immediately, and the currently readable (or writable) content will be written to or output from the buffer. Even if there is currently no available data, the call will still return immediately. Returns without doing anything to the buffer.
The NIO framework was launched in JDK1.4. Its emergence is to solve the shortcomings of traditional IO. In this video, we will start to explain around NIO.
buffer
Everything starts with the buffer, including the source code. In fact, this is not difficult, but you need to clarify your ideas.
Buffer class and its implementation
The Buffer class is an implementation of a buffer, similar to an array in Java, and is also used to store and obtain data. But Buffer is very powerful compared to arrays in Java. It contains a series of shortcut operations for arrays.
Buffer is an abstract class, its core content:
public abstract class Buffer {
// 这四个变量的关系: mark <= position <= limit <= capacity
// 这些变量就是Buffer操作的核心了，之后我们学习的过程中可以看源码是如何操作这些变量的
private int mark = -1;
private int position = 0;
private int limit;
private int capacity;
// 直接缓冲区实现子类的数据内存地址（之后会讲解）
long address;
Let’s take a look at the subclasses of the Buffer class, including all the basic types we recognize (except for the boolean type):
- IntBuffer - Buffer of type int.
- ShortBuffer - A buffer of type short.
- LongBuffer - A buffer of type long.
- FloatBuffer - A buffer of type float.
- DoubleBuffer - A buffer of type double.
- ByteBuffer - A buffer of type byte.
- CharBuffer - A buffer of type char.
(Note that although the StringBuffer we have learned in JavaSE before is also named in this way, it does not belong to the Buffer system and will not be introduced here)
Here we take IntBuffer as an example. Let's see how to create a Buffer class:
public static void main(String[] args) {
//创建一个缓冲区不能直接new，而是需要使用静态方法去生成，有两种方式：
//1. 申请一个容量为10的int缓冲区
IntBuffer buffer = IntBuffer.allocate(10);
//2. 可以将现有的数组直接转换为缓冲区（包括数组中的数据）
int[] arr = new int[]{
1, 2, 3, 4, 5, 6};
IntBuffer buffer = IntBuffer.wrap(arr);
}
So how does it actually operate internally? Let's take a look at its source code:
public static IntBuffer allocate(int capacity) {
if (capacity < 0) //如果申请的容量小于0，那还有啥意思
throw new IllegalArgumentException();
return new HeapIntBuffer(capacity, capacity); //可以看到这里会直接创建一个新的IntBuffer实现类
//HeapIntBuffer是在堆内存中存放数据，本质上就数组，一会我们可以在深入看一下
}
public static IntBuffer wrap(int[] array, int offset, int length) {
try {
//可以看到这个也是创建了一个新的HeapIntBuffer对象，并且给了初始数组以及截取的起始位置和长度
return new HeapIntBuffer(array, offset, length);
} catch (IllegalArgumentException x) {
throw new IndexOutOfBoundsException();
}
}
public static IntBuffer wrap(int[] array) {
return wrap(array, 0, array.length); //调用的是上面的wrap方法
}
So how is this HeapIntBuffer implemented? Let’s take a look:
HeapIntBuffer(int[] buf, int off, int len) {
// 注意这个构造方法不是public，是默认的访问权限
super(-1, off, off + len, buf.length, buf, 0); //你会发现这怎么又去调父类的构造方法了，绕来绕去
//mark是标记，off是当前起始下标位置，off+len是最大下标位置，buf.length是底层维护的数组真正长度，buf就是数组，最后一个0是起始偏移位置
}
Let's take a look at how the constructor method in IntBuffer is defined:
final int[] hb; // 只有在堆缓冲区实现时才会使用
final int offset;
boolean isReadOnly; // 只有在堆缓冲区实现时才会使用
IntBuffer(int mark, int pos, int lim, int cap, // 注意这个构造方法不是public，是默认的访问权限
int[] hb, int offset)
{
super(mark, pos, lim, cap); //调用Buffer类的构造方法
this.hb = hb; //hb就是真正我们要存放数据的数组，堆缓冲区底层其实就是这么一个数组
this.offset = offset; //起始偏移位置
}
Finally, let’s take a look at the construction method in Buffer:
Buffer(int mark, int pos, int lim, int cap) {
// 注意这个构造方法不是public，是默认的访问权限
if (cap < 0) //容量不能小于0，小于0还玩个锤子
throw new IllegalArgumentException("Negative capacity: " + cap);
this.capacity = cap; //设定缓冲区容量
limit(lim); //设定最大position位置
position(pos); //设定起始位置
if (mark >= 0) {
//如果起始标记大于等于0
if (mark > pos) //并且标记位置大于起始位置，那么就抛异常（至于为啥不能大于我们后面再说）
throw new IllegalArgumentException("mark > position: ("
+ mark + " > " + pos + ")");
this.mark = mark; //否则设定mark位置（mark默认为-1）
}
}
By observing the source code, we can roughly get the following structure:
Now let's summarize the respective responsibilities of the above structures:
- Buffer: Some basic variable definitions of the buffer, such as the current position (position), capacity (capacity), maximum limit (limit), mark (mark), etc. You will definitely wonder what these variables are used for. Don’t worry, these variables It will be used in subsequent operations, and we will explain it step by step.
- Subclasses such as IntBuffer: define the array to store data (only used by heap buffer implementation subclasses), whether it is read-only, etc. In other words, the storage location of the data and related operations on the underlying array are all defined here. Okay, and the Comparable interface has been implemented.
- HeapIntBuffer heap buffer implementation subclass: the data is stored in the heap. In fact, the array of the parent class is used to save the data, and all the underlying operations defined by the parent class are implemented.
In this way, we have a general understanding of the basic structure of the Buffer class.
Buffer write operation
Earlier we learned about the basic operations of the Buffer class. Now let's take a look at how to store and obtain data in the buffer. Data storage includes the following four methods:
- public abstract IntBuffer put(int i); - Insert data at the current position (position will move backward by 1 position), implemented by a specific subclass
- public abstract IntBuffer put(int index, int i); - stores data at the specified position (position will not move backward), also implemented by specific subclasses
- public final IntBuffer put(int[] src); - directly stores the contents of all arrays (the array length cannot exceed the buffer size)
- public IntBuffer put(int[] src, int offset, int length); - Directly store the contents of the array, the same as above, but you can specify a range to store (the offset and length refer to the index of the array and how many items to take from the array )
- public IntBuffer put(IntBuffer src); - Directly store the contents of another buffer (start reading from the current position of src, write to the current buffer, until src is read)
(In the above put method, except for the specified index, everything else is 从当前position开始 and put into the buffer. If the remaining capacity cannot be accommodated, an exception will be thrown. Position represents the actual Is the current operating position)
Let's start with the simplest one, which is to insert a piece of data at the current position. So how is the current position defined? Let's take a look at the source code:
public IntBuffer put(int x) {
hb[ix(nextPutIndex())] = x; //这个ix和nextPutIndex()很灵性，我们来看看具体实现
return this;
}
protected int ix(int i) {
return i + offset; // 将i的值加上我们之前设定的offset偏移量值，但是默认是0（非0的情况后面会介绍）
}
final int nextPutIndex() {
int p = position; // 获取Buffer类中的position位置（一开始也是0）
if (p >= limit) // 位置肯定不能超过底层数组最大长度，否则越界
throw new BufferOverflowException();
position = p + 1; // 获取之后会使得Buffer类中的position+1
return p; // 返回当前的位置
}
So the put operation actually sets the data in the position position of the underlying arrayhb.
After the setting is completed, the position automatically moves back:
We can write code to see:
public static void main(String[] args) {
IntBuffer buffer = IntBuffer.allocate(10);
buffer
.put(1)
.put(2)
.put(3); //我们依次存放三个数据试试看
System.out.println(buffer);
}
Through breakpoint debugging, let's take a look at the actual operation:
You can see that as we continue to put operations, the position will keep moving backwards. Of course, if it exceeds the maximum length, an exception will be thrown directly:
Next, let's take a look at how the second put operation works. It can insert data at the specified location:
public IntBuffer put(int i, int x) {
hb[ix(checkIndex(i))] = x; //这里依然会使用ix，但是会检查位置是否合法
return this;
}
final int checkIndex(int i) {
// package-private
if ((i < 0) || (i >= limit)) // 插入的位置不能小于0并且不能大于等于底层数组最大长度
throw new IndexOutOfBoundsException();
return i; // 没有问题就把i返回
}
In fact, this is easier to understand than our previous one. Note that the value of position will not be manipulated in the whole process. You need to pay attention here.
Let's next look at the third put operation, which is implemented directly in IntBuffer and is based on the subclass implementation of the first two put methods:
public IntBuffer put(int[] src, int offset, int length) {
checkBounds(offset, length, src.length); // 检查截取范围是否合法，给offset、调用者指定长度、数组实际长度
if (length > remaining()) // 接着判断要插入的数据量在缓冲区是否容得下，装不下也不行
throw new BufferOverflowException();
int end = offset + length; // 计算出最终读取位置，下面开始for
for (int i = offset; i < end; i++)
this.put(src[i]); // 注意是直接从postion位置开始插入，直到指定范围结束
return this; // ojbk
}
public final IntBuffer put(int[] src) {
return put(src, 0, src.length); // 因为不需要指定范围，所以直接0和length，然后调上面的，多捞哦
}
public final int remaining() {
// 计算并获取当前缓冲区的剩余空间
int rem = limit - position; // 最大容量减去当前位置，就是剩余空间
return rem > 0 ? rem : 0; // 没容量就返回0
}
static void checkBounds(int off, int len, int size) {
// package-private
if ((off | len | (off + len) | (size - (off + len))) < 0) //让我猜猜，看不懂了是吧
throw new IndexOutOfBoundsException();
//实际上就是看给定的数组能不能截取出指定的这段数据，如果都不够了那肯定不行啊
}
The general process is as follows. First, an array comes in and a piece of data is taken and thrown into the buffer:
After checking that there are no problems and that the buffer has capacity, you can start inserting:
Finally, let’s take a look at the code:
public static void main(String[] args) {
IntBuffer buffer = IntBuffer.allocate(10);
int[] arr = new int[]{
1,2,3,4,5,6,7,8,9};
buffer.put(arr, 3, 4); //从下标3开始，截取4个元素
System.out.println(Arrays.toString(buffer.array())); //array方法可以直接获取到数组
}
You can see that the final result is:
Of course we can also save the contents of one buffer to another buffer:
public IntBuffer put(IntBuffer src) {
if (src == this) // 不会吧不会吧，不会有人保存自己吧
throw new IllegalArgumentException();
if (isReadOnly()) // 如果是只读的话，那么也是不允许插入操作的（我猜你们肯定会问为啥就这里会判断只读，前面四个呢）
throw new ReadOnlyBufferException();
int n = src.remaining(); // 给进来的src看看容量（注意这里不remaining的结果不是剩余容量，是转换后的，之后会说）
if (n > remaining()) // 这里判断当前剩余容量是否小于src容量
throw new BufferOverflowException();
for (int i = 0; i < n; i++)// 也是从position位置开始继续写入
put(src.get()); // 通过get方法一个一个读取数据出来，具体过程后面讲解
return this;
}
Let's take a look at the effect:
public static void main(String[] args) {
IntBuffer src = IntBuffer.wrap(new int[]{
1, 2, 3, 4, 5});
IntBuffer buffer = IntBuffer.allocate(10);
buffer.put(src); // 将src从当前position开始读取(注意这里的position是0哦), 全部读取到buffer中
// 这样读写是没有问题的
System.out.println(Arrays.toString(buffer.array()));
}
But if this is the case, a problem arises:
public static void main(String[] args) {
IntBuffer src = IntBuffer.allocate(5);
for (int i = 0; i < 5; i++) src.put(i); //手动插入数据
IntBuffer buffer = IntBuffer.allocate(10);
buffer.put(src); // 将src从当前position开始读取,全部读取到buffer中，
// 但是此时的position由于上面的put操作，现在是5了哦
// （所以，此时从src中其实啥也读不到）
System.out.println(Arrays.toString(buffer.array()));
}
We found that the result was different from the above, and the data was not successfully filled into the IntBuffer below. Why is this? In fact, it is because of the calculation problem of remaining(), because this method directly calculates the position of the postion, but after we complete the write operation, the position goes to the back, which results in The result of remaining() is finally calculated as 0.
Because this is not a write operation, but a read operation that needs to be started from the beginning, so we have to find a way to return the position to the original position so that we can read from the beginning. So how to do it?Generally, when we need to perform a read operation after writing is completed (this is the case later, not just here), we will use the flip() method to flip：
public final Buffer flip() {
limit = position; // 修改limit值，当前写到哪里，下次读的最终位置就是这里，limit的作用开始慢慢体现了
position = 0; // position归零
mark = -1; // 标记还原为-1，但是现在我们还没用到
return this;
}
In this way, the result of calculating remaining() again is the quantity we need to read. This is why remaining() is used in the put method to calculate. , let’s test it again:
public static void main(String[] args) {
IntBuffer src = IntBuffer.allocate(5);
for (int i = 0; i < 5; i++) src.put(i);
IntBuffer buffer = IntBuffer.allocate(10);
src.flip(); //我们可以通过flip来翻转缓冲区
// limit置为了当前position
// position置为了0
// mark置为了-1
buffer.put(src);
System.out.println(Arrays.toString(buffer.array()));
}
After flipping and transferring again, it will be normal.
Buffer read operation
We have looked at the write operation before, now let's take a look at the read operation. There are four methods for read operations:
public abstract int get();- Directly obtain the data of the current position (each time it is read, the position moves backward by 1 position), implemented by subclassespublic abstract int get(int index);- Obtain the data at the specified position (position will not move backward), which is also implemented by subclassespublic IntBuffer get(int[] dst)- Read data into the given array (start reading from the current position, read dst.length length elements into the specified dst array, and start from index 0 in the dst array, and then position will follow move)public IntBuffer get(int[] dst, int offset, int length)- Same as above, with a range added (start reading from the current position, read elements of the specified length into the specified dst array, and start placing them from the index offset in the dst array, and then move the position accordingly)
Let’s start with the simplest one. The implementation of the first get method is in the IntBuffer class:
public int get() {
return hb[ix(nextGetIndex())]; //直接从数组中取就完事
}
final int nextGetIndex() {
// 好家伙，这不跟前面那个一模一样吗
int p = position;
if (p >= limit)
throw new BufferUnderflowException();
position = p + 1;
return p;
}
You can see that after each read operation, the position will be +1 until the last position. If you continue to read, an exception will be thrown directly.
Let’s look at the second one:
public int get(int i) {
return hb[ix(checkIndex(i))]; //这里依然是使用checkIndex来检查位置是否非法
}
Let’s look at the third and fourth:
public IntBuffer get(int[] dst, int offset, int length) {
checkBounds(offset, length, dst.length); // 跟put操作一样，也是需要检查是否越界
if (length > remaining()) // 如果读取的长度比可以读的长度大，那肯定是不行的
throw new BufferUnderflowException();
int end = offset + length; // 计算出最终读取位置
for (int i = offset; i < end; i++)
dst[i] = get(); // 开始从position把数据读到数组中，注意是在数组的offset位置开始
return this;
}
public IntBuffer get(int[] dst) {
return get(dst, 0, dst.length); // 不指定范围的话，那就直接用上面的
}
Let's take a look at the effect:
public static void main(String[] args) {
IntBuffer buffer = IntBuffer.wrap(new int[]{
1, 2, 3, 4, 5});
int[] arr = new int[10];
buffer.get(arr, 2, 5);
System.out.println(Arrays.toString(arr));
}
You can see that the data is successfully read into the array.
Of course, if we need to get the array directly, we can also use the array() method to get it:
public final int[] array() {
if (hb == null) //为空那说明底层不是数组实现的，肯定就没法转换了
throw new UnsupportedOperationException();
if (isReadOnly) //只读也是不让直接取出的，因为一旦取出去岂不是就能被修改了
throw new ReadOnlyBufferException();
return hb; //直接返回hb
}
Let’s try it out:
public static void main(String[] args) {
IntBuffer buffer = IntBuffer.wrap(new int[]{
1, 2, 3, 4, 5});
System.out.println(Arrays.toString(buffer.array()));
}
Of course, now that we have obtained the bottom layerhb, let’s see if what we read after direct modification is the result of our modification: a>
public static void main(String[] args) {
IntBuffer buffer = IntBuffer.wrap(new int[]{
1, 2, 3, 4, 5});
int[] arr = buffer.array();
arr[0] = 99999; //拿到数组对象直接改
System.out.println(buffer.get());
}
It can be seen that in this way, since the underlying array is obtained directly, all modifications will take effect directly in the buffer.
Of course, in addition to the conventional reading method, we can also use mark() to Implement jump reading, here we need to introduce a few operations:
public final Buffer mark()- Mark current locationpublic final Buffer reset()- Let the current position jump to the position of mark at that time.
Let’s look at the marking method first:
public final Buffer mark() {
mark = position; //直接标记到当前位置，mark变量终于派上用场了，当然这里仅仅是标记
return this;
}
Let’s take a look at the reset method:
public final Buffer reset() {
int m = mark; // 存一下当前的mark位置
if (m < 0) // 因为mark默认是-1，要是没有进行过任何标记操作，那reset个毛
throw new InvalidMarkException();
position = m; // 直接让position变成mark位置
return this;
}
For example, when we read position 1, we mark it:
Then we can go back directly using the reset method:
Now let's test it:
public static void main(String[] args) {
IntBuffer buffer = IntBuffer.wrap(new int[]{
1, 2, 3, 4, 5});
buffer.get(); // 读取一位，那么position就变成1了
buffer.mark(); // 这时标记，那么mark = 1
buffer.get(); // 又读取一位，那么position就变成2了
buffer.reset(); // 直接将position = mark，也就是变回1
System.out.println(buffer.get());
}
As you can see, the reading position changes according to our operations. We will stop here for the time being about the buffer reading operation.
hasRemaining() can be used to determine whether there are still elements readable in the current buffer. Its implementation is very simple. It just determines whether the position is less than the limit. If it is less than the limit, it proves that there are still elements readable, so you can use while plus this as Loop condition, traverse the entire buffer
public final boolean hasRemaining() {
return position < limit;
}
Other buffer operations
Earlier we had a general understanding of the read and write operations of the buffer, so let's take a look at what other operations there are in addition to regular read and write operations:
public abstract IntBuffer compact()- Compression buffer, implemented by specific implementation classpublic IntBuffer duplicate()- Copying the buffer will directly create a new buffer with the same datapublic abstract IntBuffer slice()- Divide the buffer, which will divide the buffer with the original capacity into smaller ones for operation.public final Buffer rewind()- Rewinding the buffer actually means returning the position to zero, and then changing the mark back to -1public final Buffer clear()- Clear the buffer and return all variables to their original state
Let's start with the compressed buffer. It will change the size and data content of the entire buffer into the data between the position position and the limit, move it to the head of the array, and move the position back to the bottom after moving the complete body. 1 position:
public IntBuffer compact() {
int pos = position(); // 获取当前位置
int lim = limit(); // 获取当前最大position位置
assert (pos <= lim); // 断言表达式，position必须小于最大位置，肯定的
int rem = (pos <= lim ? lim - pos : 0); // 计算pos距离最大位置的长度
System.arraycopy(hb, ix(pos), hb, ix(0), rem); // 直接将hb数组当前position位置的数据拷贝到头部去，然后长度改成刚刚计算出来的空间
position(rem); // 直接将position移动到rem位置（就是将从position位置开始到最后的这个整体搬到最前面, 并且把position移动到搬的整体的下一个位置）
limit(capacity()); // pos最大位置修改为最大容量
discardMark(); // mark变回-1
return this;
}
For example, the current status is:
Then after executing the compact() method, we will intercept. At this timelimit - position = 6, then we will intercept the 4、5、6、7、8、9These 6 data are then thrown to the front, and then the position runs to7, indicating that this is the next continuing position:
Now let's check it through the code:
public static void main(String[] args) {
IntBuffer buffer = IntBuffer.wrap(new int[]{
1, 2, 3, 4, 5, 6, 7, 8, 9, 0});
for (int i = 0; i < 4; i++) buffer.get(); //先正常读4个
buffer.compact(); //压缩缓冲区
System.out.println("压缩之后的情况："+Arrays.toString(buffer.array()));
System.out.println("当前position位置："+buffer.position());
System.out.println("当前limit位置："+buffer.limit());
}
You can see that there is no problem with the final result:
Let’s look at the second method. So what should we do if we need to copy a buffer with exactly the same content? You can copy it directly using theduplicate() method:
public IntBuffer duplicate() {
//直接new一个新的，但是是吧hb给丢进去了，而不是拷贝一个新的
return new HeapIntBuffer(hb,
this.markValue(),
this.position(),
this.limit(),
this.capacity(),
offset);
}
So please guess, if a new IntBuffer is created in this way, what will happen in the following example:
public static void main(String[] args) {
IntBuffer buffer = IntBuffer.wrap(new int[]{
1, 2, 3, 4, 5});
IntBuffer duplicate = buffer.duplicate();
System.out.println(buffer == duplicate);
System.out.println(buffer.array() == duplicate.array());
}
Since the buffer is new, the first one is false, and the underlying array is passed directly without any copying during construction, so in fact the underlying arrays of the two buffers are the same object. Therefore, if one is modified, the other will also change:
public static void main(String[] args) {
IntBuffer buffer = IntBuffer.wrap(new int[]{
1, 2, 3, 4, 5});
IntBuffer duplicate = buffer.duplicate();
buffer.put(0, 66666);
System.out.println(duplicate.get());
}
Let's take a look at the source code of duplicate. It actually shares the same array and copies the variables mark, position, limit, capacity, and offset.
public IntBuffer duplicate() {
return new HeapIntBuffer(hb,
this.markValue(),
this.position(),
this.limit(),
this.capacity(),
offset);
}
Now let’s look at the next method. The slice() method will divide the buffer:
public IntBuffer slice() {
int pos = this.position(); // 获取当前position
int lim = this.limit(); // 获取position最大位置
int rem = (pos <= lim ? lim - pos : 0); // 求得剩余空间
return new HeapIntBuffer(hb, // 返回一个新的划分出的缓冲区，但是底层的数组用的还是同一个
-1,
0,
rem, // 新的容量变成了剩余空间的大小
rem,
pos + offset); //可以看到offset的地址不再是0了，而是当前的position加上原有的offset值
}
Although the bottom layer is still using the previous array, due to setting the offset value, our previous operations seem to be different (in fact, we still share the bottom array in the original buffer, and then use the new position+relative offset Measure the offset to locate the new position and operate the underlying array):
Recalling what we explained earlier, when reading and storing, it will be adjusted by theix method:
protected int ix(int i) {
return i + offset; //现在offset为4，那么也就是说逻辑上的i是0但是得到真实位置却是4
}
public int get() {
return hb[ix(nextGetIndex())]; //最后会经过ix方法转换为真正在数组中的位置
}
Of course, logically we can think of it like this:
Now let's test it:
public static void main(String[] args) {
IntBuffer buffer = IntBuffer.wrap(new int[]{
1, 2, 3, 4, 5, 6, 7, 8, 9, 0});
// 这会将position移动到索引为4的位置
for (int i = 0; i < 4; i++) buffer.get();
IntBuffer slice = buffer.slice();
System.out.println("划分之后的情况："+Arrays.toString(slice.array()));
System.out.println("划分之后的偏移地址："+slice.arrayOffset());
System.out.println("当前position位置："+slice.position());
System.out.println("当前limit位置："+slice.limit());
while (slice.hasRemaining()) {
//（limit-position > 0）
// 将所有的数据全部挨着打印出来
System.out.print(slice.get()+", "); // hb[ix(nextGetIndex())], 会通过当前的相对偏移量offset, 基于当前position，计算出绝对position，从hb数组中定位出元素, 并让position加1
}
}
As you can see, the final result is:
The last two methods are relatively simple. Let’s look at rewind() first (the Chinese translation of rewind is: rewind), which is equivalent to resetting position and mark. :
public final Buffer rewind() {
position = 0;
mark = -1;
return this;
}
is followed by clear(), which is equivalent to returning the entire buffer to its original state:
public final Buffer clear() {
position = 0; // 当前position位置 置为0
limit = capacity; // limit变回capacity
mark = -1;
return this;
}
At this point, we will explain some other operations on the buffer.
buffer comparison
Buffers can be compared. We can see that both the equals method and the compareTo method have been rewritten.
Let’s take a look at theequals method first. Note that it determines whether the remaining contents of the two buffers are consistent (note that it is the remaining content, up to the limit): a>
public boolean equals(Object ob) {
if (this == ob) //要是两个缓冲区是同一个对象，肯定一样
return true;
if (!(ob instanceof IntBuffer)) //类型不是IntBuffer那也不用比了
return false;
IntBuffer that = (IntBuffer)ob; //转换为IntBuffer
int thisPos = this.position(); //获取当前缓冲区的相关信息
int thisLim = this.limit();
int thatPos = that.position(); //获取另一个缓冲区的相关信息
int thatLim = that.limit();
int thisRem = thisLim - thisPos;
int thatRem = thatLim - thatPos;
if (thisRem < 0 || thisRem != thatRem) //如果剩余容量小于0或是两个缓冲区的剩余容量不一样，也不行
return false;
//注意比较的是剩余的内容
for (int i = thisLim - 1, j = thatLim - 1; i >= thisPos; i--, j--) //从最后一个开始倒着往回比剩余的区域
if (!equals(this.get(i), that.get(j)))
return false; //只要发现不一样的就不用继续了，直接false
return true; //上面的比较都没问题，那么就true
}
private static boolean equals(int x, int y) {
return x == y;
}
Then let’s verify it according to its ideas:
public static void main(String[] args) {
IntBuffer buffer1 = IntBuffer.wrap(new int[]{
1, 2, 3, 4, 5, 6, 7, 8, 9, 0});
IntBuffer buffer2 = IntBuffer.wrap(new int[]{
6, 5, 4, 3, 2, 1, 7, 8, 9, 0});
System.out.println(buffer1.equals(buffer2)); //直接比较
buffer1.position(6);
buffer2.position(6);
// （从position后面开始比，所以是true）
System.out.println(buffer1.equals(buffer2)); //比较从下标6开始的剩余内容
}
We can see that the result is what we thought:
Then let’s look at the comparison, compareTo method, which is actually the method provided by the Comparable interface, and it actually compares the starting remainder of pos Contents:
public int compareTo(IntBuffer that) {
int thisPos = this.position(); //获取并计算两个缓冲区的pos和remain
int thisRem = this.limit() - thisPos;
int thatPos = that.position();
int thatRem = that.limit() - thatPos;
int length = Math.min(thisRem, thatRem); //选取一个剩余空间最小的出来
if (length < 0) //如果最小的小于0，那就返回-1
return -1;
int n = thisPos + Math.min(thisRem, thatRem); //计算n的值当前的pos加上剩余的最小空间
for (int i = thisPos, j = thatPos; i < n; i++, j++) {
//从两个缓冲区的当前位置开始，一直到n结束
int cmp = compare(this.get(i), that.get(j)); //比较
if (cmp != 0)
return cmp; //只要出现不相同的，那么就返回比较出来的值
}
return thisRem - thatRem; //如果没比出来个所以然，那么就比长度
}
private static int compare(int x, int y) {
return Integer.compare(x, y);
}
We won’t introduce much here.
read-only buffer
Next let's take a look at the read-only buffer. Just like its name, the read-only buffer can only perform read operations and does not allow write operations.
So how do we create a read-only buffer?
public abstract IntBuffer asReadOnlyBuffer();- Generate a read-only buffer based on the current buffer.
Let's take a look at the specific implementation of this method:
public IntBuffer asReadOnlyBuffer() {
return new HeapIntBufferR(hb, //注意这里并不是直接创建了HeapIntBuffer，而是HeapIntBufferR，并且直接复制的hb数组
this.markValue(),
this.position(),
this.limit(),
this.capacity(),
offset);
}
So what is the difference between this HeapIntBufferR class and our ordinary HeapIntBuffer?
You can see that it inherits from HeapIntBuffer, so let's take a look at the differences in its implementation:
protected HeapIntBufferR(int[] buf,
int mark, int pos, int lim, int cap,
int off)
{
super(buf, mark, pos, lim, cap, off);
this.isReadOnly = true;
}
You can see that in its construction method, in addition to directly calling the construction method of the parent class, it also changes theisReadOnly tag to true. Let's see what the difference is in the put operation. Where:
public boolean isReadOnly() {
return true;
}
public IntBuffer put(int x) {
throw new ReadOnlyBufferException();
}
public IntBuffer put(int i, int x) {
throw new ReadOnlyBufferException();
}
public IntBuffer put(int[] src, int offset, int length) {
throw new ReadOnlyBufferException();
}
public IntBuffer put(IntBuffer src) {
throw new ReadOnlyBufferException();
}
You can see that all put methods are cool, and ReadOnlyBufferException will be thrown directly as long as they are called. However, other get methods are still not rewritten, which means that the get operation can still be used normally, but as long as it is a write operation, it will not work:
public static void main(String[] args) {
IntBuffer buffer = IntBuffer.wrap(new int[]{
1, 2, 3, 4, 5, 6, 7, 8, 9, 0});
IntBuffer readBuffer = buffer.asReadOnlyBuffer();
System.out.println(readBuffer.isReadOnly());
System.out.println(readBuffer.get());
readBuffer.put(0, 666);
}
You can see the result is:
This is the buffer in read-only state.
ByteBuffer and CharBuffer
Through the previous study, we have basically understood the use of buffers, but they are all explained based on IntBuffer. Now let's take a look at the other two basic types of buffers, ByteBuffer and CharBuffer, because the bottom layer of ByteBuffer stores many single bytes. Bytes, so there are more ways to play. Similarly, CharBuffer is a series of bytes, so there are many convenient operations.
Let’s take a look at ByteBuffer first. We can click on it directly to see:
public abstract class ByteBuffer extends Buffer implements Comparable<ByteBuffer> {
final byte[] hb; // Non-null only for heap buffers
final int offset;
boolean isReadOnly; // Valid only for heap buffers
....
You can see that if it is also implemented using a heap buffer subclass, the data will still be saved in the form of byte[]. Let’s try it out:
public static void main(String[] args) {
ByteBuffer buffer = ByteBuffer.allocate(10);
//除了直接丢byte进去之外，我们也可以丢其他的基本类型（注意容量消耗）
buffer.putInt(Integer.MAX_VALUE); //丢个int的最大值进去，注意一个int占4字节
System.out.println("当前缓冲区剩余字节数："+buffer.remaining()); // 只剩6个字节了
//我们来尝试读取一下，记得先翻转（limit指向当前position，然后让position回到0，从0开始读）
buffer.flip();
while (buffer.hasRemaining()) {
// position < limit
System.out.println(buffer.get()); //一共四个字节
}
}
The final result is:
You can see that the first byte is 127, and the next three are -1. Let's analyze it:
127converted to two's complement form is01111111, and-1converted to two's complement form is11111111
That is to say, the first byte is 01111111, and the subsequent bytes are 11111111. Splicing them together:
- Two's complement representation
01111111 11111111 11111111 11111111converted to decimal is2147483647, which is the maximum value of int.
So based on our above derivation, can you calculate the following results?
public static void main(String[] args) {
ByteBuffer buffer = ByteBuffer.allocate(10);
buffer.put((byte) 0);
buffer.put((byte) 0);
buffer.put((byte) 1);
buffer.put((byte) -1);
buffer.flip(); //翻转一下
System.out.println(buffer.getInt()); // 以int形式获取，那么就是一次性获取4个字节的哦
}
After the above calculation, the result is:
- The above data is expressed in two's complement format as:
00000000 00000000 00000001 11111111 - Convert this to decimal: 256 + 255 = 511
Okay, here’s another devilish question, replace the first one with 1:10000000 00000000 00000001 11111111, do the math yourself.
Let's take a look at CharBuffer. This buffer actually stores a lot of char type data:
public static void main(String[] args) {
CharBuffer buffer = CharBuffer.allocate(10);
buffer.put("lbwnb"); //除了可以直接丢char之外，字符串也可以一次性丢进入
System.out.println(Arrays.toString(buffer.array()));
}
But thanks to the char array, it contains a lot of string operations and can store an entire string at one time. We can even process it as a String:
public static void main(String[] args) {
CharBuffer buffer = CharBuffer.allocate(10);
buffer.put("lbwnb");
buffer.append("!"); //可以像StringBuilder一样使用append来继续添加数据
System.out.println("剩余容量："+buffer.remaining()); //已经用了6个字符了
buffer.flip(); // 注意，这里要将position设置为0，下面这个输出才能有内容的哦
System.out.println("整个字符串为："+buffer); //直接将内容转换为字符串
System.out.println("第3个字符是："+buffer.charAt(2)); //直接像String一样charAt
buffer //也可以转换为IntStream进行操作
.chars()
.filter(i -> i < 'l')
.forEach(i -> System.out.print((char) i));
}
Of course, in addition to some regular operations, we can also directly create a string as a parameter:
public static void main(String[] args) {
//可以直接使用wrap包装一个字符串，但是注意，包装出来之后是只读的
CharBuffer buffer = CharBuffer.wrap("收藏等于学会~");
System.out.println(buffer);
buffer.put("111"); //这里尝试进行一下写操作
}
It can be seen that the results are also what we expected:
For these two special buffers, we will stop here for now.
direct buffer
Note: It is recommended to complete the JVM chapter before studying this part.
Finally, let's take a look at the direct buffer. We have been using heap buffers before, which means that the data is actually stored in an array. If you have completed the JVM chapter, you must know that it actually occupies It is heap memory, and we can also create a direct buffer, that is, apply for off-heap memory to save data, and use the operating system's local IO, which is faster than the heap buffer.
So how to use the direct buffer? We can create it through the allocateDirect method:
public static void main(String[] args) {
//这里我们申请一个直接缓冲区
ByteBuffer buffer = ByteBuffer.allocateDirect(10);
//使用方式基本和之前是一样的
buffer.put((byte) 66);
buffer.flip();
System.out.println(buffer.get());
}
Let’s take a look at how thisallocateDirect method creates a direct buffer:
public static ByteBuffer allocateDirect(int capacity) {
return new DirectByteBuffer(capacity);
}
This method directly creates a new DirectByteBuffer object, so how is this class created?
You can see that it does not directly inherit from ByteBuffer, but MappedByteBuffer, and implements the interface DirectBuffer. Let's take a look at this interface first:
public interface DirectBuffer {
public long address(); // 获取内存地址
public Object attachment(); // 附加对象，这是为了保证某些情况下内存不被释放，我们后面细谈
public Cleaner cleaner(); // 内存清理类
}
public abstract class MappedByteBuffer extends ByteBuffer {
//这三个方法目前暂时用不到，后面文件再说
public final MappedByteBuffer load();
public final boolean isLoaded();
public final MappedByteBuffer force();
}
Next let's take a look at the member variables of the DirectByteBuffer class:
// 把Unsafe类取出来
protected static final Unsafe unsafe = Bits.unsafe();
// 在内存中直接创建的内存空间地址
private static final long arrayBaseOffset = (long)unsafe.arrayBaseOffset(byte[].class);
// 是否具有非对齐访问能力，根据CPU架构而定，intel、AMD、AppleSilicon 都是支持的
protected static final boolean unaligned = Bits.unaligned();
// 直接缓冲区的内存地址，为了提升速度就放到Buffer类中去了
// protected long address;
// 附加对象，一会有大作用
private final Object att;
Next let's take a look at the construction method:
DirectByteBuffer(int cap) {
// package-private
super(-1, 0, cap, cap);
boolean pa = VM.isDirectMemoryPageAligned(); //是否直接内存分页对齐，需要额外计算
int ps = Bits.pageSize();
long size = Math.max(1L, (long)cap + (pa ? ps : 0)); //计算出最终需要申请的大小
//判断堆外内存是否足够，够的话就作为保留内存
Bits.reserveMemory(size, cap);
long base = 0;
try {
//通过Unsafe申请内存空间，并得到内存地址
base = unsafe.allocateMemory(size);
} catch (OutOfMemoryError x) {
//申请失败就取消一开始的保留内存
Bits.unreserveMemory(size, cap);
throw x;
}
//批量将申请到的这一段内存每个字节都设定为0
unsafe.setMemory(base, size, (byte) 0);
if (pa && (base % ps != 0)) {
// Round up to page boundary
address = base + ps - (base & (ps - 1));
} else {
//将address变量（在Buffer中定义）设定为base的地址
address = base;
}
//创建一个针对于此缓冲区的Cleaner，由于是堆外内存，所以现在由它来进行内存清理
cleaner = Cleaner.create(this, new Deallocator(base, size, cap));
att = null;
}
You can see that in the construction method, we directly apply for enough off-heap memory to save data through the Unsafe class. So when we do not use this buffer, how will the memory be cleaned? Let’s take a look at this Cleaner:
public class Cleaner extends PhantomReference<Object>{
//继承自鬼引用，也就是说此对象会存放一个没有任何引用的对象
//引用队列，PhantomReference构造方法需要
private static final ReferenceQueue<Object> dummyQueue = new ReferenceQueue<>();
//执行清理的具体流程
private final Runnable thunk;
static private Cleaner first = null; //Cleaner双向链表，每创建一个Cleaner对象都会添加一个结点
private Cleaner
next = null,
prev = null;
private static synchronized Cleaner add(Cleaner cl) {
//添加操作会让新来的变成新的头结点
if (first != null) {
cl.next = first;
first.prev = cl;
}
first = cl;
return cl;
}
//可以看到创建鬼引用的对象就是传进的缓冲区对象
private Cleaner(Object referent, Runnable thunk) {
super(referent, dummyQueue);
//清理流程实际上是外面的Deallocator
this.thunk = thunk;
}
//通过此方法创建一个新的Cleaner
public static Cleaner create(Object ob, Runnable thunk) {
if (thunk == null)
return null;
return add(new Cleaner(ob, thunk)); //调用add方法将Cleaner添加到队列
}
//清理操作
public void clean() {
if (!remove(this))
return; //进行清理操作时会从双向队列中移除当前Cleaner，false说明已经移除过了，直接return
try {
thunk.run(); //这里就是直接执行具体清理流程
} catch (final Throwable x) {
...
}
}
So let's first take a look at what the specific cleaner is doing. Deallocator is declared in the direct buffer:
private static class Deallocator implements Runnable {
private static Unsafe unsafe = Unsafe.getUnsafe();
private long address; //内存地址
private long size; //大小
private int capacity; //申请的容量
private Deallocator(long address, long size, int capacity) {
assert (address != 0);
this.address = address;
this.size = size;
this.capacity = capacity;
}
public void run() {
//具体的清理操作
if (address == 0) {
// Paranoia
return;
}
unsafe.freeMemory(address); //这里是直接调用了Unsafe进行内存释放操作
address = 0; //内存地址改为0，NULL
Bits.unreserveMemory(size, capacity); //取消一开始的保留内存
}
}
Okay, now we can make it clear that the Unsafe class is actually called to perform the memory release operation during cleaning. So, when is this cleaning operation performed? First of all, we need to make it clear that if it is an ordinary heap buffer, due to the array used, once the object does not have any references, it will be recycled by the GC at any time, but now it is off-heap memory, and we can only manually recycle the memory. , then when DirectByteBuffer also loses its reference, will memory recycling be triggered?
The answer is yes, remember we just saw that Cleaner is a subclass of PhantomReference, and DirectByteBuffer is an object referenced by ghosts, and the specific cleaning operation is the clean method of the Cleaner class. Is there any connection between the two?
Don't tell me, it really does exist. We directly see PhantomReference's parent class Reference, and we will find such a class:
private static class ReferenceHandler extends Thread {
...
static {
// 预加载并初始化 InterruptedException 和 Cleaner 类
// 以避免出现在循环运行过程中时由于内存不足而无法加载
ensureClassInitialized(InterruptedException.class);
ensureClassInitialized(Cleaner.class);
}
public void run() {
while (true) {
tryHandlePending(true); //这里是一个无限循环调用tryHandlePending方法
}
}
}
private T referent; /* 会被GC回收的对象，也就是我们给过来被引用的对象 */
volatile ReferenceQueue<? super T> queue; //引用队列，可以和下面的next搭配使用，形成链表
//Reference对象也是一个一个连起来的节点，这样才能放到ReferenceQueue中形成链表
volatile Reference next;
//即将被GC的引用链表
transient private Reference<T> discovered; /* 由虚拟机操作 */
//pending与discovered一起构成了一个pending单向链表，标记为static类所有，pending为链表的头节点，discovered为链表当前
//Reference节点指向下一个节点的引用，这个队列是由JVM构建的，当对象除了被reference引用之外没有其它强引用了，JVM就会将指向
//需要回收的对象的Reference对象都放入到这个队列里面，这个队列会由下面的 Reference Hander 线程来处理。
private static Reference<Object> pending = null;
static {
//Reference类的静态代码块
ThreadGroup tg = Thread.currentThread().getThreadGroup();
for (ThreadGroup tgn = tg;
tgn != null;
tg = tgn, tgn = tg.getParent());
Thread handler = new ReferenceHandler(tg, "Reference Handler"); //在一开始的时候就会创建
handler.setPriority(Thread.MAX_PRIORITY); //以最高优先级启动
handler.setDaemon(true); //此线程直接作为一个守护线程
handler.start(); //也就是说在一开始的时候这个守护线程就会启动
...
}
That means the Reference Handler thread is started at the beginning, so our focus can be on thetryHandlePending method to see what this thing is doing:
static boolean tryHandlePending(boolean waitForNotify) {
Reference<Object> r;
Cleaner c;
try {
synchronized (lock) {
//加锁办事
//当Cleaner引用的DirectByteBuffer对象即将被回收时，pending会变成此Cleaner对象
//这里判断到pending不为null时就需要处理一下对象销毁了
if (pending != null) {
r = pending;
// 'instanceof' 有时会导致内存溢出，所以将r从链表中移除之前就进行类型判断
// 如果是Cleaner类型就给到c
c = r instanceof Cleaner ? (Cleaner) r : null;
// 将pending更新为链表下一个待回收元素
pending = r.discovered;
r.discovered = null; //r不再引用下一个节点
} else {
//否则就进入等待
if (waitForNotify) {
lock.wait();
}
return waitForNotify;
}
}
} catch (OutOfMemoryError x) {
Thread.yield();
return true;
} catch (InterruptedException x) {
return true;
}
// 如果元素是Cleaner类型，c在上面就会被赋值，这里就会执行其clean方法（破案了）
if (c != null) {
c.clean();
return true;
}
ReferenceQueue<? super Object> q = r.queue;
if (q != ReferenceQueue.NULL) q.enqueue(r); //这个是引用队列，实际上就是我们之前在JVM篇中讲解的入队机制
return true;
}
By interpreting the source code, we understand the entire process of memory loading and releasing of the direct buffer. Like the heap buffer, when the direct buffer does not have any strong references, it has a chance to be recycled normally by the GC and the requested memory is automatically released.
Let's take a look at how the read and write operations of the direct buffer are performed:
public byte get() {
return ((unsafe.getByte(ix(nextGetIndex())))); //直接通过Unsafe类读取对应地址上的byte数据
}
private long ix(int i) {
return address + ((long)i << 0); //ix现在是内存地址再加上i
}
Let's move on to the write operation:
public ByteBuffer put(byte x) {
unsafe.putByte(ix(nextPutIndex()), ((x)));
return this;
}
It can be seen that both reading and writing operations are completed through the memory address corresponding to the Unsafe class operation.
So how is its copy operation implemented?
public ByteBuffer duplicate() {
return new DirectByteBuffer(this,
this.markValue(),
this.position(),
this.limit(),
this.capacity(),
0);
}
DirectByteBuffer(DirectBuffer db, // 这里给的db是进行复制操作的DirectByteBuffer对象
int mark, int pos, int lim, int cap,
int off) {
super(mark, pos, lim, cap);
address = db.address() + off; //直接继续使用之前申请的内存空间
cleaner = null; //因为用的是之前的内存空间，已经有对应的Cleaner了，这里不需要再搞一个
att = db; //将att设定为此对象
}
It can be seen that if a copy operation is performed, the memory space applied for by the DirectByteBuffer that performs the copy operation will be directly used. I don’t know if you can immediately think of a problem. We know that if the DirectByteBuffer object performing the copy operation loses its strong reference and is recycled, the Cleaner will be triggered and the memory will be released. However, there is a problem that this memory space may be copied. The outgoing DirectByteBuffer object needs to continue to be used, and it must not be recycled at this time, so the att variable is used here to reference the previous DirectByteBuffer object to prevent it from losing its strong reference and being garbage collected, so as long as it is not the original DirectByteBuffer object When both the copied DirectByteBuffer object and the copied DirectByteBuffer object lose strong references, this memory space will not be recycled.
In this way, our previous unsolved mystery of why there is oneatt has been answered. The introduction to the direct buffer ends here.
aisle
Earlier we learned about the cornerstone of NIO - the buffer, so where exactly the buffer is used? After we learn about channels in this section, I believe you will know. So, what is a channel?
In traditional IO, we all transmit through streams, and data will be continuously transmitted from the stream; in NIO, data is managed in buffers, and channels are used to transmit the data in the buffer to the destination. land.
channel interface level
The basic interface of channel isChannel, so the derived interfaces and classes all start from here. Let’s take a look at the basic functions it defines:
public interface Channel extends Closeable {
//通道是否处于开启状态
public boolean isOpen();
//因为通道开启也需要关闭，所以实现了Closeable接口，所以这个方法懂的都懂
public void close() throws IOException;
}
Let's take a look at some of its sub-interfaces, starting with the most basic read and write operations:
public interface ReadableByteChannel extends Channel {
//将通道中的数据读取到给定的缓冲区中
public int read(ByteBuffer dst) throws IOException;
}
public interface WritableByteChannel extends Channel {
//将给定缓冲区中的数据写入到通道中
public int write(ByteBuffer src) throws IOException;
}
After having the read and write functions, it was finally integrated into a ByteChannel interface:
public interface ByteChannel extends ReadableByteChannel, WritableByteChannel{
}
Under ByteChannel, there are more derived interfaces:
//允许保留position和更改position的通道，以及对通道连接实体的相关操作
public interface SeekableByteChannel extends ByteChannel {
...
//获取当前的position
long position() throws IOException;
//修改当前的position
SeekableByteChannel position(long newPosition) throws IOException;
//返回此通道连接到的实体（比如文件）的当前大小
long size() throws IOException;
//将此通道连接到的实体截断（比如文件，截断之后，文件后面一半就没了）为给定大小
SeekableByteChannel truncate(long size) throws IOException;
}
Next let's look at, in addition to reading and writing, Channel can also have the ability to respond to interrupts:
public interface InterruptibleChannel extends Channel {
//当其他线程调用此方法时，在此通道上处于阻塞状态的线程会直接抛出 AsynchronousCloseException 异常
public void close() throws IOException;
}
//这是InterruptibleChannel的抽象实现，完成了一部分功能
public abstract class AbstractInterruptibleChannel implements Channel, InterruptibleChannel {
//加锁关闭操作用到
private final Object closeLock = new Object();
//当前Channel的开启状态
private volatile boolean open = true;
protected AbstractInterruptibleChannel() {
}
//关闭操作实现
public final void close() throws IOException {
synchronized (closeLock) {
//同时只能有一个线程进行此操作，加锁
if (!open) //如果已经关闭了，那么就不用继续了
return;
open = false; //开启状态变成false
implCloseChannel(); //开始关闭通道
}
}
//该方法由 close 方法调用，以执行关闭通道的具体操作，仅当通道尚未关闭时才调用此方法，不会多次调用。
protected abstract void implCloseChannel() throws IOException;
public final boolean isOpen() {
return open;
}
//开始阻塞（有可能一直阻塞下去）操作之前，需要调用此方法进行标记，
protected final void begin() {
...
}
//阻塞操作结束之后，也需要需要调用此方法，为了防止异常情况导致此方法没有被调用，建议放在finally中
protected final void end(boolean completed)
...
}
...
}
Some subsequent implementation classes are implemented based on the methods defined by these interfaces, such as FileChannel:
In this way, we have a general understanding of the channel-related interface definitions, so let me take a look at how to use them.
For example, now we want to read data from the input stream and print it out, then the previous way of writing traditional IO:
public static void main(String[] args) throws IOException {
//数组创建好，一会用来存放从流中读取到的数据
byte[] data = new byte[10];
//直接使用输入流
InputStream in = System.in;
while (true) {
int len;
while ((len = in.read(data)) >= 0) {
//将输入流中的数据一次性读取到数组中
System.out.println("读取到一批数据："+new String(data, 0, len)); //读取了多少打印多少
}
}
}
/** 输入26个字母, 观察 控制台输出 **/
/*
abcdefghijklmnopqrstuvwxyz
读取到一批数据：abcdefghij
读取到一批数据：klmnopqrst
读取到一批数据：uvwxyz
*/
And now after we use the channel:
public static void main(String[] args) throws IOException {
//缓冲区创建好，一会就靠它来传输数据
ByteBuffer buffer = ByteBuffer.allocate(10);
//将System.in作为输入源，一会Channel就可以从这里读取数据，然后通过缓冲区装载一次性传递数据
ReadableByteChannel readChannel = Channels.newChannel(System.in);
while (true) {
//将通道中的数据写到缓冲区中，缓冲区最多一次装10个
readChannel.read(buffer);
//写入操作结束之后，需要进行翻转，以便接下来的读取操作
buffer.flip(); // 会做3个动作：limit = position; position = 0; mark = -1;
//最后转换成String打印出来康康
System.out.println("读取到一批数据："+new String(buffer.array(), 0, buffer.remaining())); // remaining() = limit - position
//回到最开始的状态
buffer.clear(); // 会做3个动作：position = 0; limit = capacity; mark = -1;
}
}
At first glance, it seems that there is no difference. Isn't it just replacing the array with a buffer? The effect is the same. The data is also read from the Channel, and the data is loaded through the buffer and the result is obtained. However, Channel is not one-way like a stream. It is just like its name. A channel can go from one end to the other end or from the other end to this end. We will introduce it later.
File transferFileChannel
Earlier we introduced the basic situation of the channel. Here we will try to realize the reading and writing of files. In traditional IO, the writing and output of files are completed by FileOutputStream and FileInputStream:
public static void main(String[] args) throws IOException {
try(FileOutputStream out = new FileOutputStream("test.txt");
FileInputStream in = new FileInputStream("test.txt")
)
{
String data = "伞兵一号卢本伟准备就绪！";
out.write(data.getBytes()); //向文件的输出流中写入数据，也就是把数据写到文件中
out.flush();
// 创建1个与输入流大小相同的字节数组, 以方便 1次 读完
byte[] bytes = new byte[in.available()];
in.read(bytes); //从文件的输入流中读取文件的信息
System.out.println(new String(bytes));
}
}
Now, we only need to use a FileChannel to complete the two operations. There are several ways to obtain the file channel:
public static void main(String[] args) throws IOException {
//1. 直接通过 输入流 或 输出流 获取对应的通道
FileInputStream in = new FileInputStream("test.txt");
//但是这里的通道 只支持 读取 或是 写入操作
FileChannel channel = in.getChannel();
//创建一个容量为128的缓冲区
ByteBuffer buffer = ByteBuffer.allocate(128);
//从通道中将数据读取到缓冲区中
// 过程：从channel中按顺序取到字节数据, 1个1个的写到buffer中, 每写1个, position往后移动1位!!
// 如果channel中的数据不够写完这个buffer, 那么这1次读取就会全部写到这个buffer中
// 如果channel中的数据比这个buffer的容量要大, 那么这1次读取就会写满这个buffer
channel.read(buffer);
// 翻转一下，接下来要读取了
// 因为上面是把内容写到了ByteBuffer中, position的位置移动到了内容的后面, 因此这里需要把limit=position, 并且position的位置置为0, 开始读取内容
buffer.flip();
System.out.println(new String(buffer.array(), 0, buffer.remaining()));
}
You can see that there is no problem reading the file channel obtained through the input stream, but the writing operation:
public static void main(String[] args) throws IOException {
//1. 直接通过输入或输出流获取对应的通道
FileInputStream in = new FileInputStream("test.txt");
//但是这里的通道只支持读取或是写入操作
FileChannel channel = in.getChannel();
//尝试写入一下
channel.write(ByteBuffer.wrap("伞兵一号卢本伟准备就绪！".getBytes()));
}
Report an error directly, indicating that only read operations are supported. What about the output stream?
public static void main(String[] args) throws IOException {
//1. 直接通过输入或输出流获取对应的通道
FileOutputStream out = new FileOutputStream("test.txt");
//但是这里的通道只支持读取或是写入操作
FileChannel channel = out.getChannel();
//尝试写入一下
// 过程：从byteBuffer中的当前position开始到limit为止, 每读取1个字节, position就向后移动1位, 并且把读取的字节数据写到channel中,
channel.write(ByteBuffer.wrap("伞兵一号卢本伟准备就绪！".getBytes()));
}
You can see that writing can be performed normally, but what about reading?
public static void main(String[] args) throws IOException {
//1. 直接通过输入或输出流获取对应的通道
FileOutputStream out = new FileOutputStream("test.txt");
//但是这里的通道只支持读取或是写入操作
FileChannel channel = out.getChannel();
ByteBuffer buffer = ByteBuffer.allocate(128);
//从通道中将数据读取到缓冲区中
channel.read(buffer);
//翻转一下，接下来要读取了
buffer.flip();
System.out.println(new String(buffer.array(), 0, buffer.remaining()));
}
You can see that the Channel generated by the output stream does not support reading, so it essentially maintains the characteristics of the input and output streams. But wasn't it said before that the Channel can be both input and output? Here we take a look at the second way:
//RandomAccessFile能够支持文件的随机访问，并且实现了数据流
public class RandomAccessFile implements DataOutput, DataInput, Closeable {
We can create channels through RandomAccessFile:
public static void main(String[] args) throws IOException {
/*
通过RandomAccessFile进行创建，注意后面的mode有几种：
r 以只读的方式使用
rw 读操作和写操作都可以
rws 每当进行写操作，同步的刷新到磁盘，刷新内容和元数据
rwd 每当进行写操作，同步的刷新到磁盘，刷新内容
*/
try(RandomAccessFile f = new RandomAccessFile("test.txt", "")){
}
}
Now let's test its read and write operations:
public static void main(String[] args) throws IOException {
/*
通过RandomAccessFile进行创建，注意后面的mode有几种：
r 以只读的方式使用
rw 读操作和写操作都可以
rws 每当进行写操作，同步的刷新到磁盘，刷新内容和元数据
rwd 每当进行写操作，同步的刷新到磁盘，刷新内容
*/
try(RandomAccessFile f = new RandomAccessFile("test.txt", "rw"); //这里设定为支持读写，这样创建的通道才能具有这些功能
FileChannel channel = f.getChannel()){
//通过RandomAccessFile创建一个通道
// 刚刚创建出来的byteBuffer [pos=0 lim=36 cap=36]
ByteBuffer byteBuffer = ByteBuffer.wrap("伞兵二号马飞飞准备就绪！".getBytes())
// 从byteBuffer的当前position开始读, 一直读到limit为止, position在不断+1 在读的过程中写入到channel
// (并且注意：这个channel在写的时候是从文件开始位置开始往后写的， 如果原文件中有内容则会覆盖的哦，
// 如果需要追加内容的话，可以先调用channel.position(channel.size());)
channel.write(byteBuffer);
System.out.println("写操作完成之后文件访问位置："+channel.position()); //注意读取也是从现在的位置开始
channel.position(0); //需要将位置变回到最前面，这样下面才能从文件的最开始进行读取
ByteBuffer buffer = ByteBuffer.allocate(128);
// 从channel中读取数据写入到byteBuffer中，并且是从byteBuffer的当前position开始写, 一直写到limit为止, position在不断+1
channel.read(buffer); // 该方法返回读取到了多少个字节
// 接下来要读取了, 因此limit = position; position = 0; mark = -1;
buffer.flip();
System.out.println(new String(buffer.array(), 0, buffer.remaining()));
}
}
As you can see, a FileChannel can complete both file reading and file writing.
In addition to basic read and write operations, we can also truncate files directly:
public static void main(String[] args) throws IOException {
try(
// RandomAccessFile可以先把文件指针移动最后面（通过seek或skipBytes），然后开始写入, 否则会直接从文件的起始位置开始写而覆盖原有内容
RandomAccessFile f = new RandomAccessFile("test.txt", "rw");
FileChannel channel = f.getChannel()
)
{
//截断文件，只留前20个字节
channel.truncate(20);
ByteBuffer buffer = ByteBuffer.allocate(128);
channel.read(buffer);
buffer.flip();
System.out.println(new String(buffer.array(), 0, buffer.remaining()));
}
}
You can see that the content of the file is directly truncated, leaving only half of the file content.
Of course, if we want to copy files, it is also very convenient. We only need to use channels. For example, if we need to write data from one channel to another channel, we can directly use the transferTo method:
public static void main(String[] args) throws IOException {
try(
FileOutputStream out = new FileOutputStream("test2.txt");
FileInputStream in = new FileInputStream("test.txt")
)
{
FileChannel inChannel = in.getChannel(); //获取到test文件的通道
// 当test2在磁盘上不存在时，会自动创建这个文件
inChannel.transferTo(0, inChannel.size(), out.getChannel()); //直接将test文件通道中的数据转到test2文件的通道中
}
}
You can see that after execution, all the contents of the file are copied to another file.
Of course, the reverse operation is also possible (reverse means: before, inChannel actively reads and then writes to outChannel, here, outChannel actively reads content from inChannel and then writes):
public static void main(String[] args) throws IOException {
try(FileOutputStream out = new FileOutputStream("test2.txt");
FileInputStream in = new FileInputStream("test.txt")){
FileChannel inChannel = in.getChannel(); //获取到test文件的通道
out.getChannel().transferFrom(inChannel, 0, inChannel.size()); //直接将从test文件通道中传来的数据转给test2文件的通道
}
}
When we want to edit a file, by using the MappedByteBuffer class, we can map it into memory for editing, and the edited content will be updated to the file synchronously:
//注意一定要是可写的，不然无法进行修改操作
try(RandomAccessFile f = new RandomAccessFile("test.txt", "rw");
FileChannel channel = f.getChannel()){
//通过map方法映射文件的某一段内容，创建MappedByteBuffer对象
//比如这里就是从第四个字节开始，映射10字节内容到内存中
//注意这里需要使用MapMode.READ_WRITE模式，其他模式无法保存数据到文件
MappedByteBuffer buffer = channel.map(FileChannel.MapMode.READ_WRITE, 4, 10);
//我们可以直接对在内存中的数据进行编辑，也就是编辑Buffer中的内容
//注意这里写入也是从pos位置开始的，默认是从0开始，相对于文件就是从第四个字节开始写
//注意我们只映射了10个字节，也就是写的内容不能超出10字节了
buffer.put("yyds".getBytes());
//编辑完成后，通过force方法将数据写回文件的映射区域
buffer.force();
}
As you can see, a certain area of the file has been modified by us, and the DirectByteBuffer direct buffer is actually used here, which is very efficient.
FileLockFileLock
We can create a cross-process file lock to prevent file contention operations between multiple processes (note that this is a process, not a thread). FileLock is a file lock, which can ensure that only one process (program) can modify it at the same time, or They can only be read, which solves the problem of synchronizing files between multiple processes and ensures security. However, it should be noted that it is at the process level, not at the thread level. It can solve the problem of multiple processes accessing the same file concurrently, but it is not suitable for controlling the access of a file by multiple threads in the same process.
So let's take a look at how to use file locks:
public static void main(String[] args) throws IOException, InterruptedException {
//创建RandomAccessFile对象，并拿到Channel
RandomAccessFile f = new RandomAccessFile("test.txt", "rw");
FileChannel channel = f.getChannel(); // 可读可写
System.out.println(new Date() + " 正在尝试获取文件锁...");
//接着我们直接使用lock方法进行加锁操作（如果其他进程已经加锁，那么会一直阻塞在这里）
//加锁操作支持对文件的某一段进行加锁，比如这里就是从0开始后的6个字节加锁，false代表这是一把独占锁
//范围锁甚至可以提前加到一个还未写入的位置上
FileLock lock = channel.lock(0, 6, false);
System.out.println(new Date() + " 已获取到文件锁！");
Thread.sleep(5000); //假设要处理5秒钟
System.out.println(new Date() + " 操作完毕，释放文件锁！");
//操作完成之后使用release方法进行锁释放
lock.release();
}
Regarding shared locks and exclusive locks:
- After a process adds an exclusive lock to a file, the current process can read and write to the file and exclusively owns the file. Other processes cannot read or write the file.
- After a process adds a shared lock to a file, the process can read the file, but cannot write. Shared locks can be added by multiple processes, but as long as shared locks exist, exclusive locks cannot be added.
Now let's try starting two processes. We need to configure two startup items in IDEA:
Now we start them in sequence:
It can be seen that only one of the two processes can access at the same time, while the other needs to wait for the lock to be released.
So what if we are applying for different parts of the document?
//其中一个进程锁 0 - 5
FileLock lock = channel.lock(0, 6, false);
//另一个进程锁 6 - 11
FileLock lock = channel.lock(6, 6, false);
It can be seen that the two processes can perform locking operations at the same time because they lock different paragraphs.
So what if there is a crossover?
//其中一个进程锁 0 - 5
FileLock lock = channel.lock(0, 6, false);
//另一个进程锁 3 - 8
FileLock lock = channel.lock(3, 6, false);
It can be seen that blockage will also occur in the case of intersection.
Next let's take a look at shared locks. Shared locks allow multiple processes to lock at the same time, but write operations cannot be performed:
public static void main(String[] args) throws IOException, InterruptedException {
RandomAccessFile f = new RandomAccessFile("test.txt", "rw");
FileChannel channel = f.getChannel();
System.out.println(new Date() + " 正在尝试获取文件锁...");
//现在使用共享锁
FileLock lock = channel.lock(0, Long.MAX_VALUE, true);
System.out.println(new Date() + " 已获取到文件锁！");
//进行写操作
channel.write(ByteBuffer.wrap(new Date().toString().getBytes()));
System.out.println(new Date() + " 操作完毕，释放文件锁！");
//操作完成之后使用release方法进行锁释放
lock.release();
}
When we do a write operation:
You can see that an exception is thrown directly, saying that another program has locked part of the file and the process cannot access it (actual testing in some systems or environments is invalid. For example, the UP master's arm architecture MacOS does not take effect. This exception is running in a Windows environment. owned)
Of course, we can also test multiple processes and add shared locks at the same time:
public static void main(String[] args) throws IOException, InterruptedException {
RandomAccessFile f = new RandomAccessFile("test.txt", "rw");
FileChannel channel = f.getChannel();
System.out.println(new Date() + " 正在尝试获取文件锁...");
FileLock lock = channel.lock(0, Long.MAX_VALUE, true);
System.out.println(new Date() + " 已获取到文件锁！");
Thread.sleep(5000); //假设要处理5秒钟
System.out.println(new Date() + " 操作完毕，释放文件锁！");
lock.release();
}
You can see that the result is that multiple processes can add shared locks:
Of course, in addition to directly using the lock() method to lock, we can also use the tryLock() method to obtain the file lock in a non-blocking manner, but if Failure to acquire the lock will result in null:
public static void main(String[] args) throws IOException, InterruptedException {
RandomAccessFile f = new RandomAccessFile("test.txt", "rw");
FileChannel channel = f.getChannel();
System.out.println(new Date() + " 正在尝试获取文件锁...");
FileLock lock = channel.tryLock(0, Long.MAX_VALUE, false);
System.out.println(lock);
Thread.sleep(5000); //假设要处理5秒钟
lock.release();
}
As you can see, both processes try to acquire exclusive locks:
The first process that successfully locked obtained the corresponding lock object, while the second process directly obtained null.
At this point, the relevant content about file locking is almost complete.
Multiplexed network communications
We have already introduced the two cores of the NIO framework: Buffer and Channel. Let's take a look at the last content.
Traditional blocking I/O network communication
Speaking of network communication, I believe everyone is familiar with it. It is precisely because of the existence of the network that we can enter the modern society. In the JavaWeb stage, we learned how to use Socket to establish a TCP connection for network communication:
Server code:
public static void main(String[] args) {
try(ServerSocket server = new ServerSocket(8080)){
//将服务端创建在端口8080上
System.out.println("正在等待客户端连接...");
// 如果当前没有客户端过来连接，则会一直阻塞在这里，直到有客户端过来建立连接，然后得到与客户端通信的socket
Socket socket = server.accept();
System.out.println("客户端已连接，IP地址为："+socket.getInetAddress().getHostAddress());
BufferedReader reader = new BufferedReader(new InputStreamReader(socket.getInputStream())); //通过
System.out.print("接收到客户端数据：");
// 这里会一直阻塞在这里，直到客户端发送数据过来。
// 如果这里一直读取不到数据，这里就一直阻塞，当客户端挂掉时，这里会抛出SocketException异常
System.out.println(reader.readLine());
OutputStreamWriter writer = new OutputStreamWriter(socket.getOutputStream());
// 这里写数据给客户端, 当客户端挂掉时，这里也会抛出异常
writer.write("已收到！");
writer.flush();
}catch (IOException e){
e.printStackTrace();
}
}
Client code:
public static void main(String[] args) {
// 与服务端建立连接，得到与服务端通信的socket对象
try (Socket socket = new Socket("localhost", 8080);
Scanner scanner = new Scanner(System.in)
)
{
System.out.println("已连接到服务端！");
// 从 与服务端通信的socket对象 中，拿到OutputStream
OutputStream stream = socket.getOutputStream();
// 写数据给服务器
OutputStreamWriter writer = new OutputStreamWriter(stream); //通过转换流来帮助我们快速写入内容
System.out.println("请输入要发送给服务端的内容：");
String text = scanner.nextLine();
writer.write(text+'\n'); //因为对方是readLine()这里加个换行符
writer.flush();
System.out.println("数据已发送："+text);
BufferedReader reader = new BufferedReader(new InputStreamReader(socket.getInputStream()));
// 这里会一直阻塞在这里，直到服务端发送数据过来。
// 如果服务端一直不发送过来，这里就一直阻塞，当服务端挂掉时，这里会抛出SocketException异常
System.out.println("收到服务器返回："+reader.readLine());
}catch (IOException e){
System.out.println("服务端连接失败！");
e.printStackTrace();
}finally {
System.out.println("客户端断开连接！");
}
}
Above we get the inputStream input stream and outputStream output stream through socket, where inputStream is used to receive data and outputStream is used to send data.
Of course, we can also use the channels explained earlier to communicate:
Server code:
public static void main(String[] args) {
// 创建一个新的ServerSocketChannel，一会直接使用SocketChannel进行网络IO操作
try (ServerSocketChannel serverChannel = ServerSocketChannel.open()){
// 依然是将其绑定到8080端口
serverChannel.bind(new InetSocketAddress(8080));
// 同样是调用accept()方法，阻塞一直等到新的连接过来
SocketChannel socket = serverChannel.accept();
// 因为是通道，两端的信息都是可以明确的，这里获取远端地址，当然也可以获取本地地址
System.out.println("客户端已连接，IP地址为："+socket.getRemoteAddress());
// 因为是基于通道，所以使用缓冲区进行数据接收
ByteBuffer buffer = ByteBuffer.allocate(128);
// SocketChannel同时实现了读写通道接口，所以可以直接进行双向操作, 这里从socketChannel中读取数据
// 从通道中读取数据，存入到buffer缓冲区中， 在这里会一直等到客户端发送数据过来， 或者客户端在此等待期间断开连接，这里会抛出异常
socket.read(buffer);
// 读取完数据后，将 limit = position; position = 0; mark = -1
buffer.flip();
System.out.print("接收到客户端数据："+new String(buffer.array(), 0, buffer.remaining()));
// 使用ByteBuffer.wrap(..)根据指定内容创建的缓冲区的position是0,limit是capacity
// (将position到limit间的数据写到socketChannel中)
socket.write(ByteBuffer.wrap("已收到！".getBytes())); // 向通道中写入数据就行
//记得关
socket.close();
} catch (IOException e) {
throw new RuntimeException(e);
}
}
Client code:
public static void main(String[] args) {
//创建一个新的SocketChannel，一会通过通道进行通信
try (SocketChannel channel = SocketChannel.open(new InetSocketAddress("localhost", 8080));
Scanner scanner = new Scanner(System.in)
)
{
System.out.println("已连接到服务端！");
System.out.println("请输入要发送给服务端的内容：");
String text = scanner.nextLine();
//直接向通道中写入数据，真舒服
channel.write(ByteBuffer.wrap(text.getBytes()));
ByteBuffer buffer = ByteBuffer.allocate(128);
// 从通道中读取数据，存入到buffer缓冲区中， 在这里会一直等到对方发送数据过来， 或者对方在此等待期间断开连接，这里会抛出异常
channel.read(buffer); //直接从通道中读取数据
// 读取完数据后，将 limit = position; position = 0; mark = -1
buffer.flip();
System.out.println("收到服务器返回："+new String(buffer.array(), 0, buffer.remaining()));
} catch (IOException e) {
throw new RuntimeException(e);
}
}
Although network communication can be carried out through traditional Socket, we found that if we want to perform IO operations, we need to create a separate thread for processing. For example, there are many clients and the server needs to process them at the same time. So if we want to process For these client requests, we can only create a separate thread for processing:
Although this seems reasonable,但是随着客户端数量的增加，如果要保持持续通信，那么就不能摧毁这些线程，而是需要一直保留 (but in fact, many times it just maintains the connection and is blocked waiting for the client's read and write operations. The frequency of IO operations is very low, so It occupies a thread in vain, and most of the time I just stand in the pit without shit), but our 线程不可能无限制的进行创建，总有一天会耗尽服务端的资源, so what should we do now? The key is that now there are many clients continuously connecting and To operate, at this time, we就可以利用NIO为我们提供的多路复用编程模型.
is equivalent to saying: 按照之前的模型：来一个客户端跟我们的服务端建立连接，我们服务端的ServerSocketChannel在accept()监听到这个建立连接的请求后，就会得到1个SocketChannel，然后就开1个线程专门处理与这个客户端通信的socketChannel（如果不开线程的话，对这个客户端的socketChannel的读的操作，就会阻塞ServerSocket#accept所在的线程，而导致服务端接收不了其它客户端的连接请求了，所以，我们是不得不开1个线程专门处理这个客户端的读写操作，这样ServerSocket#accept所在的线程就可以继续接收新的客户端连接了），但是如果这个客户端连接上后啥也不干，这样服务端就浪费了1个线程在白白的等待，这里其实就是服务端单独开了1个线程在主动的等待客户端，我们其实并不需要主动的等待，我们需要当客户端来连接时，当客户端发送了数据过来可读时， 这时，我们才有必要去处处理这个客户端，所以nio它提出了1种selector选择器的模型，,
Let’s take a look at the model NIO provides us:
The server is no longer a mechanism that simply creates connections through theaccept() method. Instead, the Selector will continuously poll according to the different statuses of the client, and only the client responds. When the state is in, for example, when the read and write operations actually start, the thread will be created or processed (so that it will not be blocked waiting for a client's IO operation), instead of needing to maintain the connection after creation, even if there is no read write operation. In this way, threads will not be created indefinitely because they are occupying the pit and not taking a shit.
In this way, even a single thread can achieve efficient reuse. The most typical example is Redis. Because the memory is very fast, the overhead of multi-threaded context will be a bit of a hindrance. It is not as simple and efficient as directly using a single thread. , which is why Redis can be so fast in a single thread.
Therefore, we will start with the third core content of the NIO framework: Selector.
Selectors and I/O multiplexing
We have briefly learned about the selector before. We know that the selector will only be processed when a specific state (such as read, write, request) is ready, instead of letting our program actively wait.
Since we now need to implement IO multiplexing, let's take a look at the common IO multiplexing model, which is the implementation of Selector. For example, there are many users connected to our server now:
- select: When these connections appear in a specific state, we only know that they are ready, but we do not know which connection is ready. Each call is made. The time complexity of linearly traversing all connections is
O(n), and there is a limit on the maximum number of connections. - poll: Same as above, but because the underlying layer uses a linked list, there is no limit on the maximum number of connections.
- epoll: Using the event notification method, when a connection is ready, it can directly and accurately notify (this is because in the kernel implementation, epoll is based on the callback function on each fd Implemented, as long as it is ready, the callback function will be directly called back to achieve precise notification, but only Linux supports this method), time complexity
O(1), this is the mode adopted by Java in the Linux environment implemented.
Okay, now that we understand the multiplexing model, let’s take a look at how to multiplex our network communications:
public static void main(String[] args) {
try (ServerSocketChannel serverChannel = ServerSocketChannel.open();
Selector selector = Selector.open() //开启一个新的Selector，这玩意也是要关闭释放资源的
)
{
serverChannel.bind(new InetSocketAddress(8080));
// 要使用选择器进行操作，必须使用非阻塞的方式(如果这里不配置为非阻塞模式，则下面将ServerSocketChannel注册到Selector时，会抛出IllegalBlockingModeException异常)，
// 并且，这样才不会像阻塞IO那样卡在accept()，而是直接通过，让选择器去进行下一步操作
serverChannel.configureBlocking(false);
//SelectionKey.OP_CONNECT --- 连接就绪事件，表示客户端与服务器的连接已经建立成功
//SelectionKey.OP_ACCEPT --- 接收连接事件，表示服务器监听到了客户连接，服务器可以接收这个连接了
//SelectionKey.OP_READ --- 读 就绪事件，表示通道中已经有了可读的数据，可以执行读操作了
//SelectionKey.OP_WRITE --- 写 就绪事件，表示已经可以向通道写数据了（这玩意比较特殊，一般情况下因为都是可以写入的，所以可能会无限循环）
//将选择器注册到ServerSocketChannel中，后面是选择需要监听的事件，只有发生对应事件时才会进行选择（意思是：当监听的事件有发生时，selector#select方法才会停止阻塞），多个事件用 | 连接，注意，并不是所有的Channel都支持以下全部四个事件，可能只支持部分
//因为是ServerSocketChannel这里我们就监听accept就可以了(注意：这里是只能写SelectionKey.OP_ACCEPT，如果写其它的，会抛出IllegalArgumentException异常)，等待客户端连接
serverChannel.register(selector, SelectionKey.OP_ACCEPT);
while (true) {
//无限循环等待新的用户网络操作
//每次选择都可能会选出多个已经就绪的网络操作，注册到该selector上的channel所监听的事件没有发生时，会暂时阻塞
int count = selector.select();
System.out.println("监听到 "+count+" 个事件");
Set<SelectionKey> selectionKeys = selector.selectedKeys();
Iterator<SelectionKey> iterator = selectionKeys.iterator();
while (iterator.hasNext()) {
SelectionKey key = iterator.next();
// 发生事件后，不能置之不理，如果置之不理，则这个事件在下次selector#select()不会阻塞（即使在这里把这个selectionKey移除也没用）
//根据不同的事件类型，执行不同的操作即可
if(key.isAcceptable()) {
// 如果当前ServerSocketChannel已经做好准备处理Accept
// 因为前面设置了ServerSocketChannel非阻塞, 因此这里不会阻塞，
// 但是因为当前是有了客户端来连接，才有的OP_ACCEPT事件，因此，当前肯定有客户端来建立连接，因此这里不会返回null
//（在非阻塞模式下，如果当前没有客户端来建立连接，那么此方法会返回null）
SocketChannel channel = serverChannel.accept(); // 处理accept事件
System.out.println("客户端已连接，IP地址为："+channel.getRemoteAddress());
// 现在连接就建立好了，接着我们需要将连接也注册选择器，比如我们需要当这个连接有内容可读时就进行处理
// (如果这里不配置为非阻塞模式，则下面将ServerSocketChannel注册到Selector时，会抛出IllegalBlockingModeException异常)
channel.configureBlocking(false);
// 这样就在连接建立时完成了注册
// 这里将新的SocketChannel注册到了Selector上，并且监听该channel的read事件
//（监听该channel的read事件,需要注意的是：当客户端发送消息过来的时候，会触发read事件，selector#select方法停止阻塞而向下运行，
// 此时一定要调用socketChannel的read方法(或调用这个key的cancel方法取消这个key)，
// 不然的话，即使移除该selectionKey，在进入while下一次循环到selector#select方法时, 仍然会有该事件的存在而不会阻塞。
// 并且，客户端断开连接也会触发read事件）
// 这里不能写SelectionKey.OP_CONNECT会导致selector#select不能阻塞，返回的是0，又没有事件，原因不清楚
// 这里不能写SelectionKey.OP_WRITE，它会一直触发Write事件
// 这里不能写 SelectionKey.OP_ACCEPT，会直接抛出IllegalArgumentException异常
channel.register(selector, SelectionKey.OP_READ);
} else if(key.isReadable()) {
//如果当前连接有可读的数据并且可以写，那么就开始处理
SocketChannel channel = (SocketChannel) key.channel();
ByteBuffer buffer = ByteBuffer.allocate(128);
channel.read(buffer); // 处理read事件
buffer.flip();
System.out.println("接收到客户端数据："+new String(buffer.array(), 0, buffer.remaining()));
//直接向通道中写入数据就行
channel.write(ByteBuffer.wrap("已收到！".getBytes()));
//别关，说不定用户还要继续通信呢
}
//处理完成后，一定记得移出迭代器，不然下次还有
iterator.remove();
}
}
} catch (IOException e) {
throw new RuntimeException(e);
}
}
Next let's write the client client:
public static void main(String[] args) {
//创建一个新的SocketChannel，一会通过通道进行通信
try (SocketChannel channel = SocketChannel.open(new InetSocketAddress("localhost", 8080));
Scanner scanner = new Scanner(System.in)
)
{
System.out.println("已连接到服务端！");
while (true) {
//咱给它套个无限循环，这样就能一直发消息了
System.out.println("请输入要发送给服务端的内容：");
String text = scanner.nextLine();
//直接向通道中写入数据，真舒服
channel.write(ByteBuffer.wrap(text.getBytes()));
System.out.println("已发送！");
ByteBuffer buffer = ByteBuffer.allocate(128);
channel.read(buffer); //直接从通道中读取数据
buffer.flip();
System.out.println("收到服务器返回："+new String(buffer.array(), 0, buffer.remaining()));
}
} catch (IOException e) {
throw new RuntimeException(e);
}
}
Let's take a look at the effect:
You can see that it has been successfully implemented. Of course, you can also open the client with your roommates to test. Now, we只用了一个线程，就能够同时处理多个请求 can see how important multiplexing is.
通过selector这个组件，channel将自己和自己感兴趣的事件注册到selector组件上，当selector监测到channel上发生该channel感兴趣的事件时，selector#select方法就会停止阻塞，让服务端能够处理该事件。而1个selector上可以监听多个channel，这样就实现了当有对应的事件发生了，才会去处理，而不是让服务端对每个连接的客户端都开1个线程去主动的等待客户端发送消息
Implementing the Reactor pattern
Earlier we simply implemented multiplexed network communication. Let's take a look at the Reactor mode and optimize our server.
Now let's see how to optimize. We first abstract two components, Reactor线程 and Handler处理器:
- Reactor thread: Responsible for responding to IO events and distributing them to Handler processors. New events include connection establishment ready, read ready, write ready, etc.
- Handler processor: performs non-blocking operations.
In fact, what we wrote before is a simple model of single-threaded Reactor (process-oriented writing). Let's take a look at the standard writing:
The client still connects to the Reactor as we did above, and goes to the Acceptor or Handler through the selector. The Acceptor is mainly responsible for establishing the client connection, and the Handler is responsible for reading and writing operations. The code is as follows, first is the Handler:
public class Handler implements Runnable{
private final SocketChannel channel;
public Handler(SocketChannel channel) {
this.channel = channel;
}
@Override
public void run() {
try {
ByteBuffer buffer = ByteBuffer.allocate(128);
channel.read(buffer);
buffer.flip();
System.out.println("接收到客户端数据："+new String(buffer.array(), 0, buffer.remaining()));
channel.write(ByteBuffer.wrap("已收到！".getBytes()));
}catch (IOException e){
e.printStackTrace();
}
}
}
Next is Acceptor, which is actually just moving the above business code to another location:
/**
* Acceptor主要用于处理连接操作
*/
public class Acceptor implements Runnable{
private final ServerSocketChannel serverChannel;
private final Selector selector;
public Acceptor(ServerSocketChannel serverChannel, Selector selector) {
this.serverChannel = serverChannel;
this.selector = selector;
}
@Override
public void run() {
try{
SocketChannel channel = serverChannel.accept();
System.out.println("客户端已连接，IP地址为："+channel.getRemoteAddress());
channel.configureBlocking(false);
//这里在注册时，创建好对应的Handler，这样在Reactor中分发的时候就可以直接调用Handler了
// (这里第三个参数，在将channel注册到selector时，添加了1个附加对象，
// 这个附加对象，可以在后面此channel发生可读事件时，从SelectionKey中拿到)
channel.register(selector, SelectionKey.OP_READ, new Handler(channel));
}catch (IOException e){
e.printStackTrace();
}
}
}
Here we throw in an additional object during registration. This additional object can be obtained through the attachment() method when the selector is selected on this channel. For us to simplify the code It has a great effect and will be shown in a moment. Let’s take a look at Reactor:
public class Reactor implements Closeable, Runnable{
private final ServerSocketChannel serverChannel;
private final Selector selector;
public Reactor() throws IOException{
serverChannel = ServerSocketChannel.open();
selector = Selector.open();
}
@Override
public void run() {
try {
serverChannel.bind(new InetSocketAddress(8080));
serverChannel.configureBlocking(false);
//注册时，将Acceptor作为附加对象存放，当选择器选择后也可以获取到
serverChannel.register(selector, SelectionKey.OP_ACCEPT, new Acceptor(serverChannel, selector));
while (true) {
int count = selector.select();
System.out.println("监听到 "+count+" 个事件");
Set<SelectionKey> selectionKeys = selector.selectedKeys();
Iterator<SelectionKey> iterator = selectionKeys.iterator();
while (iterator.hasNext()) {
this.dispatch(iterator.next()); //通过dispatch方法进行分发
iterator.remove();
}
}
}catch (IOException e) {
e.printStackTrace();
}
}
//通过此方法进行分发
private void dispatch(SelectionKey key){
Object att = key.attachment(); //获取attachment，ServerSocketChannel和对应的客户端Channel都添加了的
if(att instanceof Runnable) {
((Runnable) att).run(); //由于Handler和Acceptor都实现自Runnable接口，这里就统一调用一下
} //这样就实现了对应的时候调用对应的Handler或是Acceptor了
}
//用了记得关，保持好习惯，就像看完视频要三连一样
@Override
public void close() throws IOException {
serverChannel.close();
selector.close();
}
}
（
这里面就很巧妙：当selector#select方法停止阻塞时，有可能是新的客户端连接上来了，也有可能是已连接的客户端发送数据过来了。但是这2种情况都会交给SelectionKey#attachment()返回的附件对象，即对应的Acceptor 或者是 Handler处理，并且因为Acceptor和Handler都实现了Runnable接口，因此这里直接调用Runnable#run方法
)
Finally, we write the main class:
public static void main(String[] args) {
//创建Reactor对象，启动，完事
try (Reactor reactor = new Reactor()){
reactor.run();
}catch (IOException e) {
e.printStackTrace();
}
}
The client code remains unchanged, as follows:
public class Client01 {
public static void main(String[] args) {
//创建一个新的SocketChannel，一会通过通道进行通信
try (SocketChannel channel = SocketChannel.open(new InetSocketAddress("localhost", 8080));
Scanner scanner = new Scanner(System.in)
) {
System.out.println("已连接到服务端！");
while (true) {
//咱给它套个无限循环，这样就能一直发消息了
System.out.println("请输入要发送给服务端的内容：");
String text = scanner.nextLine();
//直接向通道中写入数据，真舒服
channel.write(ByteBuffer.wrap(text.getBytes()));
System.out.println("已发送！");
ByteBuffer buffer = ByteBuffer.allocate(128);
channel.read(buffer); //直接从通道中读取数据
buffer.flip();
System.out.println("收到服务器返回：" + new String(buffer.array(), 0, buffer.remaining()));
}
} catch (IOException e) {
throw new RuntimeException(e);
}
}
}
In this way, we have implemented the single-threaded Reactor mode. Note that only one thread is used throughout the process, and no new threads are created to handle anything.
However, a single thread can never cope with a large number of requests. If the number of requests increases, a single thread is still not enough. Next, let's take a look at the multi-threaded Reactor mode. It creates multiple threads for processing. After the data is read, we can The operation is handed over to the thread pool for execution:
In fact, we only need to slightly modify the Handler:
public class Handler implements Runnable{
//把线程池给安排了，10个线程
private static final ExecutorService POOL = Executors.newFixedThreadPool(10);
private final SocketChannel channel;
public Handler(SocketChannel channel) {
this.channel = channel;
}
@Override
public void run() {
try {
ByteBuffer buffer = ByteBuffer.allocate(1024);
channel.read(buffer);
buffer.flip();
POOL.submit(() -> {
try {
System.out.println("接收到客户端数据："+new String(buffer.array(), 0, buffer.remaining()));
channel.write(ByteBuffer.wrap("已收到！".getBytes()));
}catch (IOException e){
e.printStackTrace();
}
});
} catch (IOException e) {
throw new RuntimeException(e);
}
}
}
In this way, after the data is read, the data processing can be handed over to the thread pool for execution.
But this still feels like it is not divided enough. One Reactor needs to handle all operation requests from the client at the same time, which seems a bit weak (在上面的例子中，我们发现不管是新客户端的连接，还是已连接的客户端发送数据过来而触发channel的可读事件，都由同一个selector，并且是在同1个线程中处理，读取完客户端的内容后再交给线程池处理的，这样此selector的功能不够单一，我们可以让此selector只负责监听serverSocketChannel的accept事件，当监听到客户端连接时，通过serverSocketChannel#acceopt方法得到与客户端通信的socketChannel，将此socketChannel注册到另外1个selector上，并开启1个线程监听此socketChannel的可读事件，而1个selector可监听多个channel，这样就避免了bio模型下线程无限制的创建的问题), so we might as well make the Reactor into one master and multiple servers. In the slave mode, the main Reactor is only responsible for the Accept operation, while the other slave Reactors perform other operations:
Now let's redesign our code. The Reactor class will be used as the main node without any modifications. Let's modify the others:
//SubReactor作为从Reactor
public class SubReactor implements Runnable, Closeable {
//每个从Reactor也有一个Selector
private final Selector selector;
//创建一个4线程的线程池，也就是四个从Reactor工作
private static final ExecutorService POOL = Executors.newFixedThreadPool(4);
private static final SubReactor[] reactors = new SubReactor[4];
private static int selectedIndex = 0; //采用轮询机制，每接受一个新的连接，就轮询分配给四个从Reactor
static {
//在一开始的时候就让4个从Reactor跑起来
for (int i = 0; i < 4; i++) {
try {
reactors[i] = new SubReactor();
POOL.submit(reactors[i]);
} catch (IOException e) {
e.printStackTrace();
}
}
}
//轮询获取下一个Selector（Acceptor用）
public static Selector nextSelector(){
Selector selector = reactors[selectedIndex].selector;
selectedIndex = (selectedIndex + 1) % 4;
return selector;
}
private SubReactor() throws IOException {
selector = Selector.open();
}
@Override
public void run() {
try {
//启动后直接等待selector监听到对应的事件即可，其他的操作逻辑和Reactor一致
while (true) {
int count = selector.select();
System.out.println(Thread.currentThread().getName()+" >> 监听到 "+count+" 个事件");
Set<SelectionKey> selectionKeys = selector.selectedKeys();
Iterator<SelectionKey> iterator = selectionKeys.iterator();
while (iterator.hasNext()) {
this.dispatch(iterator.next());
iterator.remove();
}
}
}catch (IOException e) {
e.printStackTrace();
}
}
private void dispatch(SelectionKey key){
Object att = key.attachment();
if(att instanceof Runnable) {
((Runnable) att).run();
}
}
@Override
public void close() throws IOException {
selector.close();
}
}
Let's next modify the Acceptor class:
public class Acceptor implements Runnable{
private final ServerSocketChannel serverChannel; //只需要一个ServerSocketChannel就行了
public Acceptor(ServerSocketChannel serverChannel) {
this.serverChannel = serverChannel;
}
@Override
public void run() {
try{
SocketChannel channel = serverChannel.accept(); //还是正常进行Accept操作，得到SocketChannel
System.out.println(Thread.currentThread().getName()+" >> 客户端已连接，IP地址为："+channel.getRemoteAddress());
channel.configureBlocking(false);
Selector selector = SubReactor.nextSelector(); //选取下一个从Reactor的Selector
selector.wakeup(); //在注册之前唤醒一下防止卡死
channel.register(selector, SelectionKey.OP_READ, new Handler(channel)); //注意现在注册的是从Reactor的Selector
}catch (IOException e){
e.printStackTrace();
}
}
}
Now, SocketChannel related operations are handled by the slave Reactor, instead of being handed over to the main Reactor.
So far, we have learned about the three major components of NIO: Buffer, Channel, Selector, about the basics of NIO The relevant content is explained here.
In the next chapter, we will continue to explain Netty, a high-performance network communication framework based on NIO.