Java I / O system from the principle to the application, this one of the all-clear

This article describes the operating system I / O works, Java I / O design, the basic use, high-performance I / O common methods and implement open source projects implemented thoroughly get to know the high-performance I / O Road

Basic concepts

Before introducing the I / O principle, to review a few basic concepts:

• (1) operating system with the kernel

Operating System : computer hardware and software resources management system software
Kernel : The core operating system software, the process responsible for managing the system memory, device drivers, and network file systems, etc., to provide secure access services to computer hardware for the application

• 2 kernel space and user space

To prevent the user from directly operating kernel process, kernel security guarantee, the operating system memory address space is divided into two parts:
kernel space (Kernel-Space) , for the kernel program using
the user space (the User-Space) , for the user process
for security , kernel space and user space is isolated, even if the user program crashes, the kernel will not be affected

• 3 data stream

The computer is based on the data with the converted low voltage signal transmission time, the data signals continuously, with fixed transmission direction, similar to the flow of water in the pipe, so the concept of abstract data stream (I / O stream): refers to a sequential group, starting and ending with a collection of bytes ,

Abstract stream role: to achieve decoupling of the underlying hardware and program logic , through the incoming data stream as an abstraction layer between the application and the hardware device, for general-purpose programming data stream input and output interfaces, rather than specific hardware features, the program and the underlying hardware can be replaced independently of flexibility and scalability

I / O Works

1 磁盘 I / O

Typical I / O read and write disk works as follows:

Tips : the DMA: full name of the direct memory access (Direct Memory Access), the mechanism to directly access the system main memory of the peripheral device (hardware subsystems) is an allowed. DMA-based access method, the data transmission system main memory can be dispensed with a hardware device of CPU scheduling full

It is worth noting:

• Based on system calls read and write operations
• Read and write operations through user buffer, a kernel buffer, the application process does not directly operate the disk
• Application process to be blocked until the read data read operation

2 网络 I / O

Here the first in the most classic of blocking I / O model description:

Tips : recvfrom, function to receive information via a socket

It is worth noting:

• Network I / O read and write operations through the user buffer, Sokcet buffer
• After the server thread from beginning to call recvfrom it returns the data reported ready this time is blocked, recvfrom returns success, the thread begins processing the datagram

Java I / O design

1 I / O Category

Java in the data stream and achieve concrete, on Java data stream following a few general points:

• (1) the direction of flow
  from the outside to the program, referred to the input stream ; program from outside, called the output stream

• Data Unit (2) the flow
  program to read and write data in bytes as a minimum unit called a byte stream , a character reading and writing data as a minimum unit, referred to as character stream

• Functional role (3) flow

From / to a particular IO device (e.g., disk, network) or the storage objects (e.g., a memory array) read / write data stream, called nodes stream ;
flow through the package to be connected to an existing stream and packaging, for data read / write function, called process flow (or flow filtration);

2 I / O operations interfaces

There java.io package under a pile of I / O operations, a beginner do not understand looked easy, in fact, careful observation of which there are rules:
these I / O operations are in classes inheriting four basic abstract streams on either flow nodes, either process stream

2.1 four basic abstract stream

java.io package contains all classes Stream I / O required, there are four basic java.io package abstract stream, byte stream and processing each character stream:

• InputStream
• OutputStream
• Reader
• Writer

Node Flow 2.2

Node Stream I / O by the class name of an abstract node type + stream stream type, which is common node types are:

• File file
• Piped-process communication pipe thread
• ByteArray / CharArray (byte array / character array)
• StringBuffer / String (String Buffer / String)

Creating a node incoming data stream source is typically in the constructor, eg:

FileReader reader = new FileReader(new File("file.txt"));
FileWriter writer = new FileWriter(new File("file.txt"));

2.3 Process Flow

Stream processing I / O by the class name of the function of existing + abstract stream wrapper stream type, which is common features are:

• Buffer : the data stream read and write buffer node provides functions to read and write data may be based bulk buffer, increase efficiency. Common with BufferedInputStream, BufferedOutputStream
• Byte stream into a character stream : realized by InputStreamReader, OutputStreamWriter
• Byte stream type data base conversion : Here the basic data types of data such as int, long, short, a DataInputStream, DataOutputStream achieve
• Byte stream object instance and conversion : for realizing the target sequence, implemented by ObjectInputStream, ObjectOutputStream

Application flow processing adapter / decorative mode, conversion / expansion of existing flow, create process flow is usually in the constructor of the incoming stream or an existing node process flow:

FileOutputStream fileOutputStream = new FileOutputStream("file.txt");
// 扩展提供缓冲写
BufferedOutputStream bufferedOutputStream = new BufferedOutputStream(fileOutputStream);
 // 扩展提供提供基本数据类型写
DataOutputStream out = new DataOutputStream(bufferedOutputStream);

3 Java NINE

Problems 3.1 Standard I / O

Java NIO (New I / O) is a possible alternative to the standard Java I / O API of IO API (starting from Java 1.4), Java NIO provides the standard I / O different I / O mode of operation, the purpose is to solve standard I / O presence of the following issues:

• (1) multiple copies of the data

Standard I / O processing, to complete a full read and write data, at least from the underlying hardware kernel space to read, then read the user file, and from user space into the kernel space, and then write the underlying hardware

Further, the bottom for the I / O system calls, where the need to pass the data through the buffer write, read, etc. functions starting address and length
due to the presence of the JVM GC, tend to cause the object moves in the position of the stack, moving the transfer address parameters into the system function is not the real address of the buffer zone

When reading and writing may cause an error, in order to solve the above problems, the use of standard I / O system calls, but also leads to additional time data copying: copying the data from the JVM stack of contiguous space into an outer memory heap (heap memory outside)

Therefore, a total of six copies of data experience, low efficiency

• (2) from blocking

Traditional network I / O processing, since the request to establish a connection (Connect), reads the network I / O data (Read), data transmission (Send) operations such as thread is blocked

// 等待连接
Socket socket = serverSocket.accept();

// 连接已建立，读取请求消息
StringBuilder req = new StringBuilder();
byte[] recvByteBuf = new byte[1024];
int len;
while ((len = socket.getInputStream().read(recvByteBuf)) != -1) {
    req.append(new String(recvByteBuf, 0, len, StandardCharsets.UTF_8));
}

// 写入返回消息
socket.getOutputStream().write(("server response msg".getBytes()));
socket.shutdownOutput();

In the above server program as an example, when a request for connection has been established, a read request message, the server calls the read method, the client may not have data ready (e.g., the client is still writing data or transmission), thread needs in the read method blocks waiting until data ready

To achieve concurrent response server, each connection requires a separate processing threads alone, when a large amount of concurrent requests in order to maintain the connection, memory, thread switching overhead is too large

3.2 Buffer

Java NIO core of three core components are Buffer (buffer), Channel (channel), Selector

Buffer used to provide I / O operations of the byte buffer has a common buffer ByteBuffer, CharBuffer, DoubleBuffer, FloatBuffer, IntBuffer, LongBuffer, ShortBuffer, respectively corresponding to the basic data types: byte, char, double, float, int, long, short, Here are the most commonly used mainly in the ByteBuffer, for example, Buffer underlying foreign-based Java heap memory

Outer heap memory means corresponding to the heap memory, the memory object allocated in memory outside the JVM stack, which directly affected by the operating system memory management (rather than a virtual machine, as compared to the heap memory, I / O operations heap outer memory of advantages:

• JVM GC line without being recycled, reducing GC threads possession of resources
• When I / O system calls, direct operation outside the heap memory can be saved within a stack memory and heap memory external replication

Allocation and release based on the underlying ByteBuffer malloc and free functions, the external allocateDirect method may apply outside the allocated heap memory, and returns the object DirectByteBuffer ByteBuffer class inheritance:

public static ByteBuffer allocateDirect(int capacity) {
    return new DirectByteBuffer(capacity);
}

Heap memory based recovery outer member variable classes DirectByteBuffer Cleaner, there is provided a method may be used to clean active recovery, Netty most of the heap memory by the presence of outer Cleaner positioning of the recording, the initiative to call recovery method to clean;
Further, when the object is GC DirectByteBuffer external memory heap, the association will be recovered

Tips : the JVM argument does not recommend setting -XX: + DisableExplicitGC, because some dependent Java NIO framework (for example, Netty) at the time of abnormal memory is exhausted, will take the initiative to call System.gc (), trigger Full GC, recycling DirectByteBuffer objects as recycling Finally safeguard mechanism outside the heap memory, after setting the parameters result in a heap outside the memory in this case being cleaned

External heap memory based DirectByteBuffer class member variable basis ByteBuffer class: Cleaner objects, the objects will execute Cleaner unsafe.freeMemory (address) at the right time, to recover this memory heap outside

Buffer understood can be seen as a set of elementary data type array, the memory address of the continuous support read and write operations corresponding to the read mode and a write mode, to save the state of the current position data by several variables: capacity, position, limit:

• The total length of the buffer capacity of the array
• Location data element in a position to operate
• Position of the next element inoperable buffer array limit: limit <= capacity

3.3 Channel

Channel concept (channel) may be analog I / O stream objects, the NIO the I / O operation is mainly based on Channel:
to read data from Channel: to create a buffer, and then read the data request Channel
writing data from Channel: Create a buffer, stuffing data, write data request Channel

Channel and flow are very similar, the main difference between the following points:

• Channel reads and writes, the standard I / O stream is unidirectional
• Channel asynchronously read and write, the standard I / O stream for a thread blocked until the write operation is completed to wait
• Channel always based buffer Buffer to read and write

The most important Java NIO implementation of several Channel:

• FileChannel: for data read and write files, based on providing a method FileChannel reduces the number of reading and writing data file copy, will be introduced later
• DatagramChannel: read and write data for the UDP
• SocketChannel: TCP for data reading and writing, the connection on behalf of the client
• ServerSocketChannel: listening TCP connection requests, each request will create a SocketChannel, generally used for the server

Standard I / O-based, we obtain this first step may be the input stream as follows, the byte data on the disk is read into the program, then the next step, in NIO programming, the need to obtain Channel and then read and write

FileInputStream fileInputStream = new FileInputStream("test.txt");
FileChannel channel = fileInputStream.channel();

Tips : FileChannel only operate in blocking mode, asynchronous processing file I / O in JDK 1.7 was only added java.nio.channels.AsynchronousFileChannel

// server socket channel:
ServerSocketChannel serverSocketChannel = ServerSocketChannel.open();
serverSocketChannel.bind(new InetSocketAddress(InetAddress.getLocalHost(), 9091));

while (true) {
    SocketChannel socketChannel = serverSocketChannel.accept();
    ByteBuffer buffer = ByteBuffer.allocateDirect(1024);
    int readBytes = socketChannel.read(buffer);
    if (readBytes > 0) {
        // 从写数据到buffer翻转为从buffer读数据
        buffer.flip();
        byte[] bytes = new byte[buffer.remaining()];
        buffer.get(bytes);
        String body = new String(bytes, StandardCharsets.UTF_8);
        System.out.println("server 收到：" + body);
    }
}

3.4 Selector

Selector (selection unit), which is the Java NIO a core assembly, for checking whether one or more NIO Channel (channel) state in the readable, writable. To achieve single-threaded manage multiple Channel, which is to manage multiple network connections

Selector core is based on the operating system I / O multiplexing function, a single thread can be monitored simultaneously connected to a plurality of descriptors, once a connection is ready (ready typically read or write-ready), the program can be notified accordingly read and write , common with select, poll, epoll and other different implementations

Java NIO Selector basic working principle is as follows:

• (1) Initialization Selector object, the object server ServerSocketChannel
• (2) registered ServerSocketChannel the socket-accept event to Selector
• (3) threads are blocked in selector.select (), when there is a client requests the server, the thread exits blocked
• (4) Based selector to get all ready event, this time to get to the socket-accept event, registered client SocketChannel ready to read data Selector Event
• (5) the threads again blocked in the selector.select (), when the client connection data ready, read
• (6) based on the read ByteBuffer client request data, the response data is then written, closed channel

The following examples, the complete executable code has been uploaded github (https://github.com/caison/caison-blog-demo):

Selector selector = Selector.open();
ServerSocketChannel serverSocketChannel = ServerSocketChannel.open();
serverSocketChannel.bind(new InetSocketAddress(9091));
// 配置通道为非阻塞模式
serverSocketChannel.configureBlocking(false);
// 注册服务端的socket-accept事件
serverSocketChannel.register(selector, SelectionKey.OP_ACCEPT);

while (true) {
    // selector.select()会一直阻塞，直到有channel相关操作就绪
    selector.select();
    // SelectionKey关联的channel都有就绪事件
    Iterator<SelectionKey> keyIterator = selector.selectedKeys().iterator();

    while (keyIterator.hasNext()) {
        SelectionKey key = keyIterator.next();
        // 服务端socket-accept
        if (key.isAcceptable()) {
            // 获取客户端连接的channel
            SocketChannel clientSocketChannel = serverSocketChannel.accept();
            // 设置为非阻塞模式
            clientSocketChannel.configureBlocking(false);
            // 注册监听该客户端channel可读事件，并为channel关联新分配的buffer
            clientSocketChannel.register(selector, SelectionKey.OP_READ, ByteBuffer.allocateDirect(1024));
        }

        // channel可读
        if (key.isReadable()) {
            SocketChannel socketChannel = (SocketChannel) key.channel();
            ByteBuffer buf = (ByteBuffer) key.attachment();

            int bytesRead;
            StringBuilder reqMsg = new StringBuilder();
            while ((bytesRead = socketChannel.read(buf)) > 0) {
                // 从buf写模式切换为读模式
                buf.flip();
                int bufRemain = buf.remaining();
                byte[] bytes = new byte[bufRemain];
                buf.get(bytes, 0, bytesRead);
                // 这里当数据包大于byteBuffer长度，有可能有粘包/拆包问题
                reqMsg.append(new String(bytes, StandardCharsets.UTF_8));
                buf.clear();
            }
            System.out.println("服务端收到报文：" + reqMsg.toString());
            if (bytesRead == -1) {
                byte[] bytes = "[这是服务回的报文的报文]".getBytes(StandardCharsets.UTF_8);

                int length;
                for (int offset = 0; offset < bytes.length; offset += length) {
                    length = Math.min(buf.capacity(), bytes.length - offset);
                    buf.clear();
                    buf.put(bytes, offset, length);
                    buf.flip();
                    socketChannel.write(buf);
                }
                socketChannel.close();
            }
        }
        // Selector不会自己从已selectedKeys中移除SelectionKey实例
        // 必须在处理完通道时自己移除 下次该channel变成就绪时，Selector会再次将其放入selectedKeys中
        keyIterator.remove();
    }
}

Tips : the Java based on the NIO Selector high performance network I / O using this more cumbersome, unfriendly use, typically used in the industry for packaging Optimization Java NIO, feature-rich Netty extended framework to achieve elegance

High performance I / O optimization

The following description in conjunction with the industry's popular open source projects to optimize performance I / O's

1 zero-copy

Zero-copy (zero copy) technology, used to reduce the read and write data even entirely avoid unnecessary CPU copies, reduce memory bandwidth usage, improve efficiency, zero-copy principle there are several different implementations, here are the common open source projects achieve zero copy

1.1 Kafka zero-copy

Kafka kernel based on Linux 2.1, and 2.4 of the improved kernel sendfile + functions provided in hardware DMA Gather Copy zero copy, file transfer through the socket

By one function call to complete the file transfer system, reducing the original mode read / write mode switching. While reducing the copy data, sendfile detailed procedure is as follows:

The basic process is as follows:

• (1) user process initiated sendfile system call
• (2) the core is based DMA Copy file data copied from the kernel buffer to disk
• (3) The kernel of the kernel buffer file descriptor (file descriptor, data length) are copied to the buffer Socket
• (4) Socket-based kernel buffer and file description information provided Gather Copy DMA hardware feature to copy the data to the kernel buffer NIC
• (5) user process sendfile system call and returns

Compared to conventional I / O mode, sendfile + zero copy DMA Gather Copy manner, the number of times data is copied from the four reduced twice, from the system call reduced switching times 2 times 1, changes from the user process context 4 to 2 DMA Copy, greatly improve the processing efficiency

Kafka FileChannel based underlayer under java.nio package transferTo:

public abstract long transferTo(long position, long count, WritableByteChannel target)

transferTo transmitting FileChannel associated files to the specified channel, when Comsumer consumption data, Kafka Server based FileChannel sends a message to the data file SocketChannel

1.2 RocketMQ zero-copy

Based RocketMQ mmap + write the zero-copy mode:
the mmap () may be a buffer address and user space mapping in the kernel buffer, data sharing, eliminating the need to copy data from the kernel buffer to the user buffer

tmp_buf = mmap(file, len); 
write(socket, tmp_buf, len);

mmap + write zero copy the basic process is as follows:

• (1) initiates a user process to the kernel system call mmap
• (2) the read buffer cache kernel space and user space of the user process's memory-mapped address
• (3) Copy the kernel file data from disk-based DMA Copy to the kernel buffer
• (4) user process mmap system call and returns
• (5) user process initiated by write system calls to the kernel
• (6) based on the core CPU Copy copies data from the kernel buffer into the buffer Socket
• (7) based on the core DMA Copy copies data from the buffer to the card Socket
• (8) write system calls the user process to complete and return

RocketMQ mmap message based on the logic implemented in the storage and loading of write org.apache.rocketmq.store.MappedFile internal implementations, map-based method to obtain FileChannel mmap buffer based java.nio.MappedByteBuffer nio provided:

// 初始化
this.fileChannel = new RandomAccessFile(this.file, "rw").getChannel();
this.mappedByteBuffer = this.fileChannel.map(MapMode.READ_WRITE, 0, fileSize);

CommitLog query message, based on the shift amount mappedByteBuffer POS, query data size size:

public SelectMappedBufferResult selectMappedBuffer(int pos, int size) {
    int readPosition = getReadPosition();
    // ...各种安全校验
    
    // 返回mappedByteBuffer视图
    ByteBuffer byteBuffer = this.mappedByteBuffer.slice();
    byteBuffer.position(pos);
    ByteBuffer byteBufferNew = byteBuffer.slice();
    byteBufferNew.limit(size);
    return new SelectMappedBufferResult(this.fileFromOffset + pos, byteBufferNew, size, this);
}

tips: transientStorePoolEnable mechanism
portion of the Java NIO mmap memory is not permanent memory, can be replaced to swap memory (virtual memory), RocketMQ order to improve the performance of the message sent by the introduction of the memory locking mechanisms, CommitLog file is about to be mapped to the recent need to operate memory and provides memory lock function to ensure that these files are always stored in memory, the control parameters of the mechanism is transientStorePoolEnable

Thus, MappedFile data storage CommitLog brush disc in two ways:

• 1 Open transientStorePoolEnable: byte is written to memory buffers (writeBuffer) -> submitted from memory byte buffer (writeBuffer) (commit) to file channel (fileChannel) -> file channel (fileChannel) -> flush to disk
• 2 is not turned transientStorePoolEnable: mapping file write byte buffer (mappedByteBuffer) -> mapping file byte buffer (mappedByteBuffer) -> flush to disk

RocketMQ based mmap + write zero copy for data services such small-level message file transfer and persistence
data Kafka based on zero-copy sendfile this manner, the system is suitable for such high throughput message log file chunk persistence and transmission

Tips: Kafka index file using mmap + write mode, the data file is transmitted using the network mode sendfile

1.3 Netty zero-copy

Netty zero-copy divided into two types:

• 1 based on the operating system to achieve zero-copy, on the bottom of transferTo method FileChannel
• Optimization of Java-based operating layer 2, an array of cached objects (ByteBuf) encapsulated optimization, through the establishment of ByteBuf view data object consolidation ByteBuf support, segmentation, only when the underlying keep a data storage, reduce unnecessary copies

2 Multiplexer

Netty of the function package after Java NIO optimized to achieve I / O multiplexer elegant lot codes:

// 创建mainReactor
NioEventLoopGroup boosGroup = new NioEventLoopGroup();
// 创建工作线程组
NioEventLoopGroup workerGroup = new NioEventLoopGroup();

final ServerBootstrap serverBootstrap = new ServerBootstrap();
serverBootstrap 
     // 组装NioEventLoopGroup 
    .group(boosGroup, workerGroup)
     // 设置channel类型为NIO类型
    .channel(NioServerSocketChannel.class)
    // 设置连接配置参数
    .option(ChannelOption.SO_BACKLOG, 1024)
    .childOption(ChannelOption.SO_KEEPALIVE, true)
    .childOption(ChannelOption.TCP_NODELAY, true)
    // 配置入站、出站事件handler
    .childHandler(new ChannelInitializer<NioSocketChannel>() {
        @Override
        protected void initChannel(NioSocketChannel ch) {
            // 配置入站、出站事件channel
            ch.pipeline().addLast(...);
            ch.pipeline().addLast(...);
        }
    });

// 绑定端口
int port = 8080;
serverBootstrap.bind(port).addListener(future -> {
    if (future.isSuccess()) {
        System.out.println(new Date() + ": 端口[" + port + "]绑定成功!");
    } else {
        System.err.println("端口[" + port + "]绑定失败!");
    }
});

3 cache (PageCache)

Page cache (PageCache) operating system file caching, to reduce disk I / O operations, in page units, the content is the physical blocks on disk page cache can help program to read and write files sequentially almost close to the speed of read and write speed of the memory , the main reason is due to the mechanism used by the OS to read and write access operations PageCache optimized for performance:

Page cache read policy : whether to initiate the process when a read operation (for example, initiated a process read () system call), it first checks the required data in the page cache:

• If so, then give up access to the disk, and read directly from the page cache
• If not, then the kernel scheduler block I / O operation to read data from the disk, and reads a page followed by a few (less than a page, usually three pages), then the page data into cache

Page cache write policy : When the process is initiated by the write system call to write data to a file, first wrote the page cache, then the method returns. At this point the data has not really saved to a file go, Linux only will this data page cache pages marked as "dirty", and is added to the list of dirty pages

Then, a flusher thread periodically will write back page of the list of dirty pages written to disk, so that data on disk and in memory consistent, and finally clean up "dirty" logo. In the following three cases, dirty pages are written back to disk:

• Free memory is below a certain threshold
• When dirty pages resident in memory exceeds a specific threshold value
• When a user process calls sync () and fsync () system call

RocketMQ, the less data ConsumeQueue logic consumption stored in the queue, and the read sequence, in the pre-read page cache mechanism of action, read performance is almost close to Consume Queue file memory read, nor even in the case where bulk message will affect the performance, it provides two kinds of messages brush disk strategy:

• Synchronization brush plate: In a message to the real disk persistent Broker after the end of RocketMQ Producer will actually be returned to the end of a successful ACK response
• Asynchronous brush plate, PageCache can take advantage of the operating system, as long as the message is written to the PageCache successful ACK is returned to the Producer end. Message by way of background brush disc submitted asynchronous threads, reducing possible latency, improved performance and throughput MQ

Kafka message achieve high performance using the page cache is also read, not expand here

reference

"Understanding the Linux Kernel - Daniel P.Bovet"

Netty outside of Java heap memory stick literacy - Jiangnan white

Java NIO? Look at this one is enough! - Zhu servant

RocketMQ message storing process - Zhao Kun (Zhao Kun)

Netty understand a text model architecture --caison

More exciting, welcome attention to public numbers distributed system architecture