Netty源码学习系列①JavaNIO概览

在正式开始Netty相关的学习之前，我决定还是要先回顾一下Java NIO，至少要对Java NIO相关的概念有一个了解，如Channel、ByteBuffer、Selector等。要自己动手写一写相关的demo实例、并且要尽可能地去了解其后面是如何实现的，也就是稍微看看相关jdk的源代码。

Java NIO 由以下几个核心部分组成：Buffer, Channel, Selector。传统的IO操作面向数据流，面向流的 I/O 系统一次一个字节地处理数据，意味着每次从流中读一个或多个字节，直至完成，数据没有被缓存在任何地方；NIO操作面向缓冲区（面向块），数据从Channel读取到Buffer缓冲区，随后在Buffer中处理数据。

Buffer

可以理解成煤矿里面挖煤的小车，把煤从井底运地面上面。它的属性与子类如下：

Buffer是一个抽象类，继承自Object，拥有多个子类。此类在JDK源码中的注释如下：

A container for data of a specific primitive type.

A buffer is a linear, finite sequence of elements of a specific primitive type. Aside from its content, the essential properties of a buffer are its capacity, limit, and position:

A buffer’s capacity is the number of elements it contains. The capacity of a buffer is never negative and never changes.

A buffer’s limit is the index of the first element that should not be read or written. A buffer’s limit is never negative and is never greater than its capacity.

写模式下，limit表示最多能往Buffer里写多少数据，等于capacity值；读模式下，limit表示最多可以读取多少数据，小于等于 capacity 值。

A buffer’s position is the index of the next element to be read or written. A buffer’s position is never negative and is never greater than its limit.

There is one subclass of this class for each non-boolean primitive type.

Transferring data

Each subclass of this class defines two categories of get and put operations:

Relative operations read or write one or more elements starting at the current position and then increment the position by the number of elements transferred. If the requested transfer exceeds the limit then a relative get operation throws a BufferUnderflowException and a relative put operation throws a BufferOverflowException; in either case, no data is transferred.

Absolute operations take an explicit element index and do not affect the position. Absolute get and put operations throw an IndexOutOfBoundsException if the index argument exceeds the limit.

Data may also, of course, be transferred in to or out of a buffer by the I/O operations of an appropriate channel, which are always relative to the current position.

Marking and resetting

A buffer’s mark is the index to which its position will be reset when the reset method is invoked. The mark is not always defined, but when it is defined it is never negative and is never greater than the position. If the mark is defined then it is discarded when the position or the limit is adjusted to a value smaller than the mark. If the mark is not defined then invoking the reset method causes an InvalidMarkException to be thrown.

Invariants

The following invariant holds for the mark, position, limit, and capacity values:

0 <= mark <= position <= limit <= capacity

A newly-created buffer always has a position of zero and a mark that is undefined. The initial limit may be zero, or it may be some other value that depends upon the type of the buffer and the manner in which it is constructed. Each element of a newly-allocated buffer is initialized to zero.

Clearing, flipping, and rewinding

In addition to methods for accessing the position, limit, and capacity values and for marking and resetting, this class also defines the following operations upon buffers:

clear() makes a buffer ready for a new sequence of channel-read or relative put operations: It sets the limit to the capacity and the position to zero.

flip() makes a buffer ready for a new sequence of channel-write or relative get operations: It sets the limit to the current position and then sets the position to zero.

rewind() makes a buffer ready for re-reading the data that it already contains: It leaves the limit unchanged and sets the position to zero.

Read-only buffers

Every buffer is readable, but not every buffer is writable. The mutation methods of each buffer class are specified as optional operations that will throw a ReadOnlyBufferException when invoked upon a read-only buffer. A read-only buffer does not allow its content to be changed, but its mark, position, and limit values are mutable. Whether or not a buffer is read-only may be determined by invoking its isReadOnly method.

Thread safety

Buffers are not safe for use by multiple concurrent threads. If a buffer is to be used by more than one thread then access to the buffer should be controlled by appropriate synchronization.

Invocation chaining

Methods in this class that do not otherwise have a value to return are specified to return the buffer upon which they are invoked. This allows method invocations to be chained; for example, the sequence of statements
b.flip();
b.position(23);
b.limit(42);
can be replaced by the single, more compact statement
b.flip().position(23).limit(42);

clear()方法

// 清除Buffer中的信息，只将参数恢复成默认
public final Buffer clear() {
    position = 0;
    limit = capacity;
    mark = -1;
    return this;
}

flip()方法

// 将limit记录成当前的位置，指针指向头部，为读取做准备
public final Buffer flip() {
    limit = position;
    position = 0;
    mark = -1;
    return this;
}

rewind()方法

// 指针指向头部，可以用于再次读取
public final Buffer rewind() {
    position = 0;
    mark = -1;
    return this;
}

如何使用Java NIO读取文件内容

遇到坑了，但是感觉可以透过这个问题更加深入理解Java NIO的这些概念。出现问题的代码：

public static void fileChannel() throws IOException {
    FileInputStream fis = new FileInputStream("/Users/yangyu/Documents/data.json");
    FileChannel fileChannel = fis.getChannel();
    ByteBuffer byteBuffer = ByteBuffer.allocate(1024);
    while ((fileChannel.read(byteBuffer)) != -1) {
        while (byteBuffer.hasRemaining()) {
            System.out.print((char) byteBuffer.get());
        }
    }
}

上面的代码读取不到数据，一直在做循环，但是不输出数据。为什么？因为hasRemaining()是以position和limit作对比，如下：

public final boolean hasRemaining() {
		return position < limit;
}

当从fileChannel中读取数据到byteBuffer中之后，limit与capacity相等（初始化既如此），此时的position也与capacity相同，导致hasRemaining()为false，无法向控制台输出。

所以需要将position设置成从0开始，让读取从0开始，直到读到之前的容量，所以使用flip()来完成这个目的，即：

此时却发现控制台无限打印东西，为了弄明白这是为什么，我把byteBuffer的大小调成了8，跑起来之后的输出如下：

这是为什么呢？这个问题应该与byteBuffer里面的那几个参数有关系：

猜测应该是与fileChannel.read(byteBuffer)中的具体实现有关。粗略看了看fileChannel.read(byteBuffer)的实现，大致流程如下：

计算byteBuffer的剩余量，即limit - position。对于上面的情况，剩余量为0。
找出缓存buffer，此时缓存buffer为上次read得到的，第一次为空会直接分配；第二次read的时候，其3大属性全部为8，也即上次读取的结果。
将缓存的buffer进行rewind()、flip(剩余量)，得到一个[pos=0, limit=0, capacity=8]的buffer。

进行读取的时候回根据缓存buffer的pos、limit来确定能读取的数量，也即：

// 其中var1为缓存buffer
int var5 = var1.position();
int var6 = var1.limit();

assert var5 <= var6;
// var6 - var5 = limit - position = 0
int var7 = var5 <= var6 ? var6 - var5 : 0;
// var7 = 0
if (var7 == 0) {
  // 0 即读取的字节数
  return 0;
}

如果能读到数据，会将缓存buffer里面的的内容再转移到byteBuffer（也就是我们read()里面传的ByteBuffer）中：

// var5即缓存buffer，读取内容到var5中
int var6 = readIntoNativeBuffer(var0, var5, var2, var4);
// 准备用来读取
var5.flip();
// var1是我们传入的byteBuffer，如果读取到的字节数大于0，
if (var6 > 0) {
  // 将var5中的内容拷贝到var1中
  var1.put(var5);
}

直到发现flip()的注释里面有这样一段注释：

Compacts this buffer (optional operation).
The bytes between the buffer’s current position and its limit, if any, are copied to the beginning of the buffer. That is, the byte at index p = position() is copied to index zero, the byte at index p + 1 is copied to index one, and so forth until the byte at index limit() - 1 is copied to index n = limit() - 1 - p. The buffer’s position is then set to n+1 and its limit is set to its capacity. The mark, if defined, is discarded.

The buffer’s position is set to the number of bytes copied, rather than to zero, so that an invocation of this method can be followed immediately by an invocation of another relative put method.

Invoke this method after writing data from a buffer in case the write was incomplete. The following loop, for example, copies bytes from one channel to another via the buffer buf:
buf.clear();          // Prepare buffer for use
while (in.read(buf) >= 0 || buf.position != 0) {
  buf.flip();
  out.write(buf);
  buf.compact();    // In case of partial write
}

加上这段代码buf.compact()便可以正常读取文件内容。到这里就有点心累了，为什么写个读取都这么多坑。感觉有问题的时候往这三个参数上面想就行了。

Channel

煤矿厂里面运煤的通道，需要看的子类总共有4个，分别为：

FileChannel：文件通道，用于文件的读和写。不支持非阻塞
DatagramChannel：用于 UDP 连接的接收和发送
SocketChannel：把它理解为 TCP 连接通道，简单理解就是 TCP 客户端
ServerSocketChannel：TCP 对应的服务端，用于监听某个端口进来的请求

Selector

只有自己写过的代码才会有更深刻的印象，哪怕是从别的地方抄来的，自己慢慢debug一下，找出自己对代码的疑问，然后再去搞清楚这些问题，我觉得这样让我对它的了解会更深。这两段代码基本和网上的教程类似，大部分是抄的，但是自己有一定的加工，也遇到了1个问题，外加一个疑问。

客户端代码

客户端的代码很简单：①读标准输入。②发送给Server端。

// Client端的代码很像八股文，这样弄就行了。
public static void main(String[] args) throws Exception {
    SocketChannel sc = SocketChannel.open();
    sc.configureBlocking(false);
    sc.connect(new InetSocketAddress("127.0.0.1", 8086));

    Scanner scanner = new Scanner(System.in);
    if (sc.finishConnect()) {
        ByteBuffer buffer = ByteBuffer.allocate(1024);
        while (scanner.hasNextLine()) {
          	// 读标准输入
            String info = scanner.nextLine();
            buffer.clear();
            buffer.put(info.getBytes());
            buffer.flip();
            while (buffer.hasRemaining()) {
                System.out.println(buffer);
              	// 发送给Server端
                sc.write(buffer);
            }
        }
    }
}

服务端代码

主要参考了一篇CSDN上的博客和一篇简书上的博客，简书上面的这边对我的帮助很大，十分感谢。我的问题主要有两点，第一个是少了it.remove();，第二个是关于如何触发SelectionKey.OP_WRITE事件。

// 八股文的感觉。
public static void startServer() throws IOException {
    ServerSocketChannel ssc = ServerSocketChannel.open();
    ssc.configureBlocking(false);
    ssc.socket().bind(new InetSocketAddress(8086));

    Selector selector = Selector.open();
    ssc.register(selector, SelectionKey.OP_ACCEPT);

    while (true) {
        selector.select();
        Iterator<SelectionKey> it = selector.selectedKeys().iterator();
        while (it.hasNext()) {
            SelectionKey key = it.next();
            // 一定要remove掉，不然上次的事件会累积。
            // 也就是对同一事件会处理两次，这样可能会导致报错。
            it.remove();
            if (key.isAcceptable()) {
                System.out.println("ACCEPT");
                ServerSocketChannel ssChannel = (ServerSocketChannel)key.channel();
                SocketChannel sc = ssChannel.accept();
                sc.configureBlocking(false);
                sc.register(key.selector(), SelectionKey.OP_READ, ByteBuffer.allocateDirect(BUF_SIZE));
            } else if (key.isReadable()) {
                System.out.print("READ：");
                SocketChannel sc = (SocketChannel)key.channel();
                ByteBuffer buf = (ByteBuffer)key.attachment();
                long bytesRead = sc.read(buf);
                while(bytesRead>0){
                    buf.flip();
                    while(buf.hasRemaining()){
                        System.out.print((char)buf.get());
                    }
                    System.out.println();
                    buf.clear();
                    bytesRead = sc.read(buf);
                }
                if(bytesRead == -1){
                    sc.close();
                }
            } else if (key.isWritable()) {
                // OP_WRITE事件如何触发？
                System.out.print("WRITE：");
                ByteBuffer buf = (ByteBuffer) key.attachment();
                buf.flip();
                SocketChannel sc = (SocketChannel) key.channel();
                while(buf.hasRemaining()){
                    sc.write(buf);
                }
                buf.compact();
            } else if (key.isConnectable()) {
                System.out.println("CONNECT");
            } else {
                System.out.println("UNKNOWN");
            }
        }
    }
}

如果缺少it.remove()方法的调用，那么会导致事件会堆积在Selector的Set<SelectionKey> publicSelectedKeys中，引发对同一事件会处理两次，这样可能会导致报错。
如何触发SelectionKey.OP_WRITE？因为我看到大部分关于selector的博客，都没有写如何触发该事件，并且也未对读事件做出说明。

首先肯定要在调用ssChannel.accept()之后，将得到的SocketChannel多注册一个OP_WRITE事件。即修改成：

SocketChannel sc = ssChannel.accept();
sc.configureBlocking(false);
sc.register(key.selector(), SelectionKey.OP_READ | SelectionKey.OP_WRITE,ByteBuffer.allocateDirect(BUF_SIZE));