Teach you how to learn Java NIO

Original: braces MC (micro-channel public number: huakuohao-mc). JAVA programming and focus on large data base, focusing on experience sharing and personal growth.

The article describes the Javatradition I/O, that is, B-I/O(Blocking IO). This article describes what N-I/O(Non-Block) of the basic knowledge, and why N-I/Omore advantages in high concurrency and processing large files.

Local file I / O operations --NIO small scale chopper

Channel和Buffer

BIOIt is in operation InputStreamand OutputStreamin NIOthe operation are Channeland Buffer. We can Channelthink of mineral resources, the Bufferimagination to transport ore car. If you want to move data, it must help Buffer, this is the only way to move data. That is Buffernow Channelinseparable must.

NIOWith up to three of Channel, respectively FileChannel, SocketChannelas well ServerSocketChannel. FileChannelIt is used to operate a local file, while the other two are network I/Ooperations.

FileChannel here by file test-io.tmpmovement inside the contents to a file test-io.md, so that you feel on how to use Channeland Bufferperform file I/Ooperations.

Example: NIOmanipulate local files.

//通过FileInputstream拿到输入FileChannel。
 FileChannel in = new FileInputStream("test-io.tmp").getChannel();
 //通过FileOutPutStream拿到输出FileChannel
FileChannel out = new FileOutputStream("test-io.md").getChannel();
//创建一个字节缓冲器，用于运送数据。
  ByteBuffer buffer = ByteBuffer.allocate(1024);
        while (in.read(buffer) != -1){
            //相当于缓冲器的开关，只有调用该方法，缓冲器里面的数据才能被写入到输出Channel.
            buffer.flip();
            out.write(buffer);
            buffer.clear();
        }
复制代码

The above code easily achieved, file test-io.tmpmobile content in the test-io.mdmiddle.

Code Reading

By FileInputStreamobject getChannelmethods get Channel.

By ByteBufferthe allocateprocess (may be allocateDirectya method) declare a buffer capacity of 1024bytes used to transmit data.

The source data channelinside the data readread into the buffer method. By out.write()the method, the data is written to the output buffer inside Channel. Finally clears the buffer, read data for the next preparation.

Note: Data is written to the output buffer inside channelbefore be sure to call bufferthe flip（）method. You can understand the role of this method to open the Buffervalve. Only data in order to open the valve to be removed.

ByteBuffer

ByteBufferIt is Buffera sub-class. There are many other subclass, such as CharBuffer, DoubleBufferetc., ByteBufferwith the most buffer.

We can ByteBufferimagine a byte array. Probably like this.

The figure is a schematic view of just initialized, position the cursor indicates, each byte read, position moves one position.

ByteBufferThere are several important ways, as follows

allocate(): Create a buffer, for example ByteBuffer.allocate(1024). allocateDirect(): Creating a more buffers coupled with the underlying operating system. capacity(): Returns the buffer capacity of the array. position(): Location of the next element to be operated. limit(): Returns the value of the limit. flip(): Open the valve of the buffer, ready to be read. put(): The byte is stored into the buffer. For example byteBuffer.put("hello".getBytes("utf-8")); wrap(): an array of bytes into the buffer store. For example ByteBuffer.wrap("hello".getBytest()) rewind(): the position is set to 0. clear(): Empty the buffer. hasRemaining()If there is interposed between the position values and the limit, it returns true.

Zero-copy

Examples of the above there is another one implementation, look at the code.

public class ChannelTransfer {
    public static void main(String[] args) throws Exception {
        FileChannel in = new FileInputStream("test-io.tmp").getChannel();
        FileChannel out = new FileOutputStream("test-io.md").getChannel();
        in.transferTo(0,in.size(),out);
        //或者
        //out.transferFrom(in,0,in.size());
    }
}

复制代码

Directly to the input terminal and the output terminal for docking, without the operating system kernel mode. This is the use of the famous zero-copy technology. Kafka's performance are so vital, in large part because the use of a zero-copy technology.

Large memory file read

The so-called large file is the file to be operated is larger than available memory on your system, then you can use NIOthe following method of operating the library provides.

public static void main(String[] args) throws Exception {
        FileChannel fileChannel = new FileInputStream("test-io.tmp").getChannel();
        //通过map()方法产生一个缓冲器.
        MappedByteBuffer mappedByteBuffer = fileChannel.map(FileChannel.MapMode.READ_ONLY, 0, fileChannel.size());

        if (mappedByteBuffer != null){
            CharBuffer charBuffer = Charset.forName("utf-8").decode(mappedByteBuffer);
            System.out.println(charBuffer.toString());
        }
    }
复制代码

Note that map () function has three parameters, respectively, read-write mode, an initial position and a mapping length. Because my test file is very small, so it all mapped. If the source file is large (100G) can be mapped per 500M or 1G, to find a different optimum value in accordance with the machine performance.

FileChannelThe knowledge base on these, and I believe that through the above description, for everybody NIOin Channeland Bufferhave a basic understanding.

Network I / O - NIO flourish.

We have been saying NIOis non-blocking I/O, but described above FileChaneland can not be set to non-blocking mode, you do not say funny. FileChannelCompared to the traditional (BIO), the biggest advantage is the use and processing of large files, as well as technologies such as zero-copy. If you ask me what the underlying implementation principle is, in fact, I do not know, just know that FileChannelmany of the methods provided in a more systematic way to cater to the operation work. The so-called good ass shot, promotion and pay rise come early.

If you really want to get to the bottom underlying principle, it is recommended to go to understand the knowledge of the operating system, and then go Grilled JDKsource.

Real support non-blocking operations are ServerSocketChanneland SocketChannel. Only during the network I/Owhen a non-blocking I/Ostrengths can be the greatest degree of play.

If you want to know all kinds of I/Odetails I can see this article .

Demand proposed

Suppose we want to implement a simple server program, the only function is to receive the client sent me a request, then the contents of the request after converting to uppercase sent back to the client.

BIO implementation

When a client sends a request when the server is to create a thread for processing. When clients simultaneously sending 100 requests are processed on the server to create 100 threads. That looks pretty good, but there are a few thousand higher if the number of requests or when, the service might end up a bit too much.

For the following reasons:

Thread creation and destruction of occupied system resources, even with the thread pool, nor can fundamentally solve the problem, but also in the Linux thread which is lightweight processes
The thread can not continue to create unlimited, Java inside each thread takes up 512K-1M memory space.
Constantly switching between threads is consuming system resources, because you want to keep the context and so on.

BIO is a real child.

BIOSelect multithreaded fashion and helpless choice. Because Socket.writeand Socket.readare blocked. The so-called blocking, which means that once the thread starts executing socket.readthe operation, and then you need to wait for the read operation is completed. If this time is no data can be read, then you need to wait until a few so far. This is a BIOnatural property, there is no way, it is simply too much of. So if you want to make full use of CPU, much more to create several threads, a thread is no data, there is always another bar, which is called the East does not shine bright.

To some simple pseudo-code, we feel it a little.

//整个线程池
ExecutorService executor = Executors.newFixedThreadPool(100);
ServerSocket serverSocket = new ServerSocket();
serverSocket.bind(new InetSocketAddress(8888));
//循环监听等待新连接到来
while(true) {
    Socket socket = serverSocket.accept();
    //为新的连接创建新的线程
    executor.submit(new Task(socket));
}
class Task implements Runnable {
    private Socket socket;
    public Task(Socket socket) {
        this.socket = socket;
    }
    @Override
    public void run() {
        while (!socket.isClosed()) {
            //读数据，阻塞
            String someThing = socket.read();
            if (someThing != null) {
                //处理数据,返回客户端，阻塞
                socket.write();
            }
        }
    }
}
复制代码

NIO is a smart kid.

BIOThe problem lies in the reading and writing above the obstruction. Because blocking I/Otoo real, there is no data on death and other data, resulting in CPUnot fully utilized embarrassing situation. Compared to BIO, NIOjust more clever, because it simply can not wait, but there is data, you inform me, I sent CPUto fetch. Where it is taken, to take complete left, not a little nonsense, called a fast speed. To CPUthe (intelligence) operation speed, a person manage thousands of channels is not a thing. This is the Reactorprogramming model, also known as event-based programming.

Since it is event-based programming, NIOthere are few events more important is, Read, Write, Accept, Connect.

In the NIOprogramming model, each client establishes a connection with the server is a Channelthat Channelonce the data will be notified CPUto take the corresponding channel number. It will not be like BIOthat, the situation is dead thread and other data occurs. This is CPUthe use of reason for the high.

NIO network programming model a bit like the Monkey King Cremaster diagnose the troubles.

Use NIO network programming

Mentioned above, NIO network programming is event-based programming, then have someone listening in charge of the event. This work by the Selectcompletion. When something interesting happens, Selectyou will get to know.

SelectionKeyIt is also a very important role, the equivalent Selectand Channela bridge. Because Selectnot only to know something interesting happened, but also know which Channelwhat events happened.

NIONetwork programming inside the protagonist give everybody introduction is over, respectively, selectors Selector, channel ServerSocketChanneland SocketChanel, as well as in the above-mentioned buffer ByteBuffer, as well SelectionKey.

Here to tell you about a simple interpretation of how to use NIO way to achieve that end service program mentioned above. Look at the code.

public class EchoNioServer {

    public static final int BUF_SIZE = 1024;

    public static void main(String[] args) {

        ByteBuffer byteBuffer = ByteBuffer.allocate(BUF_SIZE);

        try {
            Selector selector = Selector.open();

            ServerSocketChannel serverSocketChannel = ServerSocketChannel.open();
            serverSocketChannel.configureBlocking(false);
            serverSocketChannel.bind(new InetSocketAddress(8888));

            System.out.println("正在8888端口监听...");
            serverSocketChannel.register(selector, SelectionKey.OP_ACCEPT, byteBuffer);

            while (true) {
                selector.select();
                Iterator<SelectionKey> iterator = selector.selectedKeys().iterator();
                while (iterator.hasNext()) {
                    SelectionKey key = iterator.next();
                    iterator.remove();

                    if (!key.isValid()) {
                        continue;
                    }

                    if (key.isAcceptable()) {
                        ServerSocketChannel serverSocketChannel1 = (ServerSocketChannel) key.channel();
                        SocketChannel socketChannel = serverSocketChannel1.accept();
                        socketChannel.configureBlocking(false);
                        socketChannel.register(selector, SelectionKey.OP_READ, byteBuffer);

                    } else if (key.isReadable()) {
                        SocketChannel socketChannel = (SocketChannel) key.channel();
                        ByteBuffer readBuffer = (ByteBuffer) key.attachment();
                        readBuffer.clear();
                        socketChannel.read(readBuffer);
                        readBuffer.flip();
                        System.out.println("received from client: " + new String(readBuffer.array()).trim());
                        socketChannel.register(selector, SelectionKey.OP_WRITE, readBuffer);

                    } else if (key.isWritable()) {
                        SocketChannel socketChannel = (SocketChannel) key.channel();
                        ByteBuffer writeBuffer = (ByteBuffer) key.attachment();
                        String msg = new String(writeBuffer.array()).trim().toUpperCase();
                        writeBuffer.clear();
                        writeBuffer.put(msg.getBytes("utf-8"));
                        writeBuffer.flip();
                        socketChannel.write(writeBuffer);
                        writeBuffer.clear();
                        socketChannel.close();
                    }
                }
            }

        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}
复制代码

Interpretation of the code to help you to be a simple interpretation. To facilitate understanding.

Create a selector and a buffer spare, an event of interest to listen and one for transporting data. Selector select = Selector.open(); ByteBuffer byteBuffer = ByteBuffer.allocate(1024);
CreateServerSocketChannel ServerSocketChannel ssc = ServerSocketChannel.open();
Is set to non-blocking mode (must be set to non-blocking, or you or what NIO) ssc.configureBlocking(false)
Binding port ssc.bind(8888)
The channel selector to register and tell a selector, I'm interested in what these events right. When the event comes to call the appropriate logic for processing.sss.register(select,SelectionKey.Accept)
Call the select.selct()method to find available channels, this method is blocked, so put while (true) will not cause CPU idle.
Do different treatment for different events.

Supporting the above server-side code client code, I do not do too much explanation.

public class EchoNioClient {
    public static void main(String[] args) {
        ExecutorService executor = Executors.newFixedThreadPool(5);
        for (int i = 0; i < 10; i++) {
            executor.submit(new Task());
        }
        executor.shutdown();
    }
}

class Task implements Runnable {
    InetSocketAddress remoteAddress = new InetSocketAddress(8888);
    static final int BUF_SIZE = 1024;
    @Override
    public void run() {
        try {
            String msg = "hello I'm " + Thread.currentThread().getName();
            SocketChannel socketChannel = SocketChannel.open(remoteAddress);
            ByteBuffer byteBuffer = ByteBuffer.allocate(BUF_SIZE);
            byteBuffer.clear();
            byteBuffer.put(msg.getBytes("utf-8"));
            byteBuffer.flip();
            socketChannel.write(byteBuffer);
            byteBuffer.clear();
            ByteBuffer receiveBuffer = ByteBuffer.allocate(1024);
            while (socketChannel.read(receiveBuffer) != -1) {
                receiveBuffer.flip();
                System.out.println("received from server: " + new String(receiveBuffer.array()).trim());
                receiveBuffer.clear();
            }
            socketChannel.close();
        } catch (IOException e) {
            e.printStackTrace();
        }
    }
}
复制代码

End

I hope this article will help you better understand the NIO basic programming. After understanding the basics, bored when you can go and see Tomcat source code, you can also have the opportunity to chat up with those big cattle often Netty write high-performance gateway services.

Finally, strongly suggest that, in the example of the text put inside their IDE, run it again, the best in their own hands to write about, I do not look at a city, a written Mongolian circle, unrealistic expectations but taboo.