Original: chenmingyu.top/nio/
After learning java IO system, the focus is to learn IO models to understand the various IO model can better understand java IO
Java to Java API is a set of the IO write data (input and output). Most programs have to deal with some input by the input to produce some output. It aims to provide a Java package java.io
The system can be divided into the java io Bio, Nio, Aio three kinds of models io
- About Bio, we need to know what is synchronous blocking IO model, object Bio operations: flow, as well as how to use the Bio network programming, network programming using Bio problem
- About Nio, we need to know what is synchronous non-blocking IO model, what is the multiplex Io model, and Nio in Buffer, Channel, concept Selector, and how to use network programming Nio
- About Aio, we need to know what is the asynchronous non-blocking IO model, Aio can use several ways to achieve asynchronous operation, and how to use network programming Aio
BIO
BIO is a synchronous blocking IO, objects before JDK1.4 only this one IO model, BIO operation is a stream, a thread can handle a flow of IO requests, if you want to handle multiple streams need to use multiple threads
Stream and byte stream including a character stream, the stream is conceptually a continuous stream of data. When the program needs to read data on the read data requires the use of the input stream, when the need to write out the data of the output stream needs to
Blocking IO model
In Linux, when the application process calls recvfrom when the method call data, if the data is not ready kernel does not return immediately, but will wait for data experience ready to copy the data back from the kernel to user space after the application process during which It has been blocked until you return, so called blocking IO model
flow
BIO stream operating in two major categories, byte stream and character stream can be divided into two types of input and output streams in accordance with the direction of flow
According to the type of input and output directions can be divided into:
- Input byte stream: InputStream
- Output byte stream: OutputStream
- Enter the character stream: Reader
- Output character stream: Writer
The main stream of bytes or byte binary objects for processing, for processing a character stream or string of characters in the text
Using InputStreamReaderthe input byte may flow into the input character stream
Reader reader = new InputStreamReader(inputStream);
复制代码Using the OutputStreamWriteroutput bytes can flow into the output character stream
Writer writer = new OutputStreamWriter(outputStream)
复制代码We can read the data from the data source via InputStream and Reader in the program, and can also be output by the target OutputStream and Writer program data in the media
When using byte stream, InputStream and OutputStream are abstract classes, we instantiate them are subclasses, each subclass has its own scope
When using a character stream also, Reader and Writer are abstract class, we instantiate them are subclasses, each subclass has its own scope
To read and write files, for example
Reading data from data source
Input byte stream: the InputStream
public static void main(String[] args) throws Exception{
File file = new File("D:/a.txt");
InputStream inputStream = new FileInputStream(file);
byte[] bytes = new byte[(int) file.length()];
inputStream.read(bytes);
System.out.println(new String(bytes));
inputStream.close();
}
复制代码Enter the character stream: Reader
public static void main(String[] args) throws Exception{
File file = new File("D:/a.txt");
Reader reader = new FileReader(file);
char[] bytes = new char[(int) file.length()];
reader.read(bytes);
System.out.println(new String(bytes));
reader.close();
}
复制代码Output to the target media
Output byte stream: the OutputStream
public static void main(String[] args) throws Exception{
String var = "hai this is a test";
File file = new File("D:/b.txt");
OutputStream outputStream = new FileOutputStream(file);
outputStream.write(var.getBytes());
outputStream.close();
}
复制代码Output character stream: Writer
public static void main(String[] args) throws Exception{
String var = "hai this is a test";
File file = new File("D:/b.txt");
Writer writer = new FileWriter(file);
writer.write(var);
writer.close();
}
复制代码BufferedInputStream
When using an InputStream, a byte is a byte read or write, but BufferedInputStream provides a buffer for the input byte stream to read data when a block of data is read into the buffer once, when the buffer after being read the data in the input data stream will fill the buffer again until the input stream is read, it is possible to improve the buffer zone with a lot of speed io
The use of the input stream packaged into BufferedInputStream
/**
* inputStream 输入流
* 1024 内部缓冲区大小为1024byte
*/
BufferedInputStream bufferedInputStream = new BufferedInputStream(inputStream,1024);
复制代码BufferedOutputStream
BufferedOutputStream may provide a buffer for the output byte stream, with the effect similar to BufferedInputStream
The output streams use the package to BufferedOutputStream
/**
* outputStream 输出流
* 1024 内部缓冲区大小为1024byte
*/
BufferedOutputStream bufferedOutputStream = new BufferedOutputStream(outputStream,1024);
复制代码Byte stream provided with a buffer, it certainly provides a character stream BufferedReader and BufferedWriter
BufferedReader
Providing a buffer for the input character stream, the following use
BufferedReader bufferedReader = new BufferedReader(reader,1024);
复制代码BufferedWriter
To provide the output character stream buffer, use the following
BufferedWriter bufferedWriter = new BufferedWriter(writer,1024);
复制代码Network programming model BIO
When using BIO Socket programming model, the server usually accept method is called while loop, in the absence of client requests, will accept method blocks until the request is received and the process returns to the appropriate, the process is linear only after handling the current request processing requests will only be accepted back, this usually results in a communication thread is blocked for a long time
BIO model handle multiple connections:
In this mode, we typically use a thread to accept the request, and then use a thread pool to process the request, in this way to manage multiple concurrent client connections Socket, like this:
Use BIO model of network programming problem is the lack of flexibility scalability, client number and the number of concurrent access server thread is 1: 1 relationship, but usually due to the obstruction there will be a large number of threads in a wait state, waiting for input or output data is ready, waste of resources, in the face of a large number of concurrent, if not using direct new thread pool thread, it would roughly threads expansion, system performance, it may cause a stack overflow memory, and destroy threads frequently create more waste resources
Use the thread pool could be a little better program, but can not solve the problem of blocking blocking IO, but also need to consider if the number of thread pool settings will reject a large number of smaller Socket client connections, if the number of threads set a large pool time, will lead to a large number of context switches, and the program are to allocate memory for each thread's call stack, its default value size range of 64 KB to 1 MB, a waste of virtual machine memory
BIO model is suitable for fixed and relatively small number of links architecture, but the code written using this model more intuitively simple and easy to understand
NIO
Since JDK 1.4 version, JDK released a new I / O library, referred to as NIO, is a synchronous non-blocking IO model
Non-blocking IO model
Synchronous non-blocking IO model implementation:
Non-blocking IO model
Application process calls the recvfrom system call, if the kernel data is not ready, will return to direct a EWOULDBLOCK error, the application process will not be blocked, but the application process requires constant polling call recvfrom , until the kernel data is ready, after waiting for data to be copied from the kernel to user space (this time will be blocked, but time-consuming minimal), after the copy is complete return
IO multiplexing model
IO multiplexing model, using the Linux system provides select, poll system call, one or more file handle (the client link network programming) passed to the select or poll system call, the application process blocked on select, thus forming a process corresponding number Socket link, then select poll will be linear sweep this collection / Socket link, when there are only a few socket data, will lead to reduced efficiency, and select / poll is limited to the number of file handles held by some The default value is 1024
IO signal drive model
Sigaction system call to perform a signal processing function, the system call does not block the application process, when the data is ready, it generates a SIGIO signal to the process, to read the data signal by calling recvfrom callback notification application
NIO core concepts
Buffer (buffer)
Buffer is an object that contains the data to be written to or read out, in NIO All data are treated with a buffer zone, in reading data from the buffer when reading and writing data when the buffer will first be written zone, the buffer zone is essentially a data can be written, then read from a data array, to provide access to the structured data and a write position within the maintenance information such as
Instantiate a ByteBuffer
//创建一个容量为1024个byte的缓冲区
ByteBuffer buffer=ByteBuffer.allocate(1024);
复制代码How to Use Buffer:
- Data is written to Buffer
- Call the
flip()method Buffer switching from write mode to read mode - Reading data from the Buffer
- Call the
clear()method orcompact()methods to clear the buffer, it can be written again
For more details see this: ifeve.com/buffers/
Channel (Channel)
Channel (channel) data is always read from the buffer to the channel, or written from the buffer to the channel, Channel only responsible for the transport of data, the data is operated Buffer
The flow channel is similar in different places:
- In that the channels are bi-directional, you can simultaneously read and write operations, and the flow is one-way flow, can be written or read
- Write stream is blocked, asynchronous read and write channels may
Buffer read data from Channel
inChannel.read(buffer);
复制代码Write data from Buffer Channel
outChannel.write(buffer);
复制代码For more details see this: < ifeve.com/channels/ >
To copy the file, for example
FileInputStream fileInputStream=new FileInputStream(new File(src));
FileOutputStream fileOutputStream=new FileOutputStream(new File(dst));
//获取输入输出channel通道
FileChannel inChannel=fileInputStream.getChannel();
FileChannel outChannel=fileOutputStream.getChannel();
//创建容量为1024个byte的buffer
ByteBuffer buffer=ByteBuffer.allocate(1024);
while(true){
//从inChannel里读数据，如果读不到字节了就返回-1，文件就读完了
int eof =inChannel.read(buffer);
if(eof==-1){
break;
}
//将Buffer从写模式切换到读模式
buffer.flip();
//开始往outChannel写数据
outChannel.write(buffer);
//清空buffer
buffer.clear();
}
inChannel.close();
outChannel.close();
fileInputStream.close();
fileOutputStream.close();
复制代码Selector (multiplexer selector)
NIO Selector is the basis for programming, the main role is to register more than one Channel to the Selector, if read or write event occurs on the Channel, Channel is in a ready state, polling will be out Selector, then you can get to have passed SelectionKey Channel-ready collection of the IO operation conducted
The relationship between Selector and Channel, Buffer
For more details see this: < ifeve.com/selectors/
NIO model of network programming
NIO IO model using the JDK multiplexed, multiplexed onto a select blocked by blocking the plurality of IO, the system implemented in a single thread can handle multiple client requests simultaneously, saving system overhead, in JDK1.4 and 1.5 update10 previous versions, the JDK's Selector select / poll model implemented in the JDK version 1.5 update10 above, the bottom instead of using epoll select / poll
epoll more select / poll advantage in that:
- Epoll support open file descriptor number is not restricted, limited number of file descriptors select / poll can be opened
- select / poll polling is used to traverse the entire set of file descriptors, epoll based callback function for each file descriptor callback
select, poll, epoll is IO multiplexing mechanism. I / O multiplexing is through a mechanism, a plurality of descriptors may monitor process, once a descriptor is ready (ready generally read or write-ready), the program can be notified accordingly read and write operations. But they are synchronous I / O the select, poll, epoll essence, because they need to read and write after the event is ready responsible for their own reading and writing, that the reading and writing process is blocked, and asynchronous I / O is no need to own responsible for reading and writing
NIO provides two different sets of socket channel for network programming, server: ServerSocketChannel and client SocketChannel, two tunneling supports blocking and non-blocking mode
Server code
The server receiving the client message sent by the output, and sends a message to the client
//创建多路复用选择器Selector
Selector selector=Selector.open();
//创建一个通道对象Channel，监听9001端口
ServerSocketChannel channel = ServerSocketChannel.open().bind(new InetSocketAddress(9001));
//设置channel为非阻塞
channel.configureBlocking(false);
//
/**
* 1.SelectionKey.OP_CONNECT：连接事件
* 2.SelectionKey.OP_ACCEPT：接收事件
* 3.SelectionKey.OP_READ：读事件
* 4.SelectionKey.OP_WRITE：写事件
*
* 将channel绑定到selector上并注册OP_ACCEPT事件
*/
channel.register(selector,SelectionKey.OP_ACCEPT);
while (true){
//只有当OP_ACCEPT事件到达时，selector.select()会返回（一个key），如果该事件没到达会一直阻塞
selector.select();
//当有事件到达了，select()不在阻塞，然后selector.selectedKeys()会取到已经到达事件的SelectionKey集合
Set keys = selector.selectedKeys();
Iterator iterator = keys.iterator();
while (iterator.hasNext()){
SelectionKey key = (SelectionKey) iterator.next();
//删除这个SelectionKey，防止下次select方法返回已处理过的通道
iterator.remove();
//根据SelectionKey状态判断
if (key.isConnectable()){
//连接成功
} else if (key.isAcceptable()){
/**
* 接受客户端请求
*
* 因为我们只注册了OP_ACCEPT事件，所以有客户端链接上，只会走到这
* 我们要做的就是去读取客户端的数据，所以我们需要根据SelectionKey获取到serverChannel
* 根据serverChannel获取到客户端Channel，然后为其再注册一个OP_READ事件
*/
// 1，获取到ServerSocketChannel
ServerSocketChannel serverChannel = (ServerSocketChannel) key.channel();
// 2，因为已经确定有事件到达，所以accept()方法不会阻塞
SocketChannel clientChannel = serverChannel.accept();
// 3，设置channel为非阻塞
clientChannel.configureBlocking(false);
// 4，注册OP_READ事件
clientChannel.register(key.selector(),SelectionKey.OP_READ);
} else if (key.isReadable()){
// 通道可以读数据
/**
* 因为客户端连上服务器之后，注册了一个OP_READ事件发送了一些数据
* 所以首先还是需要先获取到clientChannel
* 然后通过Buffer读取clientChannel的数据
*/
SocketChannel clientChannel = (SocketChannel) key.channel();
ByteBuffer byteBuffer = ByteBuffer.allocate(BUF_SIZE);
long bytesRead = clientChannel.read(byteBuffer);
while (bytesRead>0){
byteBuffer.flip();
System.out.println("client data ："+new String(byteBuffer.array()));
byteBuffer.clear();
bytesRead = clientChannel.read(byteBuffer);
}
/**
* 我们服务端收到信息之后，我们再给客户端发送一个数据
*/
byteBuffer.clear();
byteBuffer.put("客户端你好，我是服务端，你看这NIO多难".getBytes("UTF-8"));
byteBuffer.flip();
clientChannel.write(byteBuffer);
} else if (key.isWritable() && key.isValid()){
//通道可以写数据
}
}
}
复制代码Client code
After the client is connected on the service side, the service sends give a message, and acceptance message sent by the server
Selector selector = Selector.open();
SocketChannel clientChannel = SocketChannel.open();
//将channel设置为非阻塞
clientChannel.configureBlocking(false);
//连接服务器
clientChannel.connect(new InetSocketAddress(9001));
//注册OP_CONNECT事件
clientChannel.register(selector, SelectionKey.OP_CONNECT);
while (true){
//如果事件没到达就一直阻塞着
selector.select();
Iterator<SelectionKey> iterator = selector.selectedKeys().iterator();
while (iterator.hasNext()){
SelectionKey key = iterator.next();
iterator.remove();
if (key.isConnectable()){
/**
* 连接服务器端成功
*
* 首先获取到clientChannel，然后通过Buffer写入数据，然后为clientChannel注册OP_READ时间
*/
clientChannel = (SocketChannel) key.channel();
if (clientChannel.isConnectionPending()){
clientChannel.finishConnect();
}
clientChannel.configureBlocking(false);
ByteBuffer byteBuffer = ByteBuffer.allocate(1024);
byteBuffer.clear();
byteBuffer.put("服务端你好，我是客户端，你看这NIO难吗".getBytes("UTF-8"));
byteBuffer.flip();
clientChannel.write(byteBuffer);
clientChannel.register(key.selector(),SelectionKey.OP_READ);
} else if (key.isReadable()){
//通道可以读数据
clientChannel = (SocketChannel) key.channel();
ByteBuffer byteBuffer = ByteBuffer.allocate(BUF_SIZE);
long bytesRead = clientChannel.read(byteBuffer);
while (bytesRead>0){
byteBuffer.flip();
System.out.println("server data ："+new String(byteBuffer.array()));
byteBuffer.clear();
bytesRead = clientChannel.read(byteBuffer);
}
} else if (key.isWritable() && key.isValid()){
//通道可以写数据
}
}
}
复制代码Use native NIO library is very complex, NIO libraries and Api complicated, cumbersome to use, need to be very familiar with network programming in order to prepare high-quality NIO program, it does not directly recommend the use of native NIO network programming, but the use of some mature framework, such as Netty
AIO
JDK1.7 upgraded Nio library, become Nio2.0, the most important is that it provides asynchronous file IO operations, as well as event-driven IO, AIO asynchronous socket channel is a true non-blocking asynchronous IO
Asynchronous IO model
In the Linux system, the application process initiated read operation, can immediately do other things, and the kernel will copy the data is ready to use the space to tell the application process, the data read operation has been completed copy
aio model of network programming
Asynchronous operation
aio polling operation is not required by multiplexer channels can be registered asynchronous read and write, thereby simplifying the programming model NIO
aio asynchronous channel provides two asynchronous operation, asynchronous channel operation by obtaining results of:
- To get the results of asynchronous operations by Future class, but note that future.get () method is blocked, the thread will be blocked
- By way asynchronous callback, the callback performed by passing a CompletionHandler implementation class, CompletionHandler defines two methods, two methods failed Completed and corresponding successes and failures
Aio in Channel support the above two ways
AIO provides corresponding sockets asynchronous channel for network programming, server: AsynchronousServerSocketChannel and client AsynchronousSocketChannel
Server
The server sends a message to the client, and accepts the message sent by the client
AsynchronousServerSocketChannel server = AsynchronousServerSocketChannel.open().bind(new InetSocketAddress("127.0.0.1", 9001));
//异步接受请求
server.accept(null, new CompletionHandler<AsynchronousSocketChannel, Void>() {
//成功时
@Override
public void completed(AsynchronousSocketChannel result, Void attachment) {
try {
ByteBuffer buffer = ByteBuffer.allocate(1024);
buffer.put("我是服务端，客户端你好".getBytes());
buffer.flip();
result.write(buffer, null, new CompletionHandler<Integer, Void>(){
@Override
public void completed(Integer result, Void attachment) {
System.out.println("服务端发送消息成功");
}
@Override
public void failed(Throwable exc, Void attachment) {
System.out.println("发送失败");
}
});
ByteBuffer readBuffer = ByteBuffer.allocate(1024);
result.read(readBuffer, null, new CompletionHandler<Integer, Void>() {
//成功时调用
@Override
public void completed(Integer result, Void attachment) {
System.out.println(new String(readBuffer.array()));
}
//失败时调用
@Override
public void failed(Throwable exc, Void attachment) {
System.out.println("读取失败");
}
});
} catch (Exception e) {
e.printStackTrace();
}
}
//失败时
@Override
public void failed(Throwable exc, Void attachment) {
exc.printStackTrace();
}
});
//防止线程执行完
TimeUnit.SECONDS.sleep(1000L);
复制代码Client
The client sends a message to the server and receive the message sent by the server
AsynchronousSocketChannel client = AsynchronousSocketChannel.open();
Future<Void> future = client.connect(new InetSocketAddress("127.0.0.1", 9001));
//阻塞，获取连接
future.get();
ByteBuffer buffer = ByteBuffer.allocate(1024);
//读数据
client.read(buffer, null, new CompletionHandler<Integer, Void>() {
//成功时调用
@Override
public void completed(Integer result, Void attachment) {
System.out.println(new String(buffer.array()));
}
//失败时调用
@Override
public void failed(Throwable exc, Void attachment) {
System.out.println("客户端接收消息失败");
}
});
ByteBuffer writeBuffer = ByteBuffer.allocate(1024);
writeBuffer.put("我是客户端，服务端你好".getBytes());
writeBuffer.flip();
//阻塞方法
Future<Integer> write = client.write(writeBuffer);
Integer r = write.get();
if(r>0){
System.out.println("客户端消息发送成功");
}
//休眠线程
TimeUnit.SECONDS.sleep(1000L);
复制代码to sum up
Each IO model comparison:
Bio pseudo asynchronous IO model refers to processing the request thread pool
reference:
netty The Definitive Guide, Second Edition
ifeve.com/java-nio-al... concurrent programming network
tech.meituan.com/2016/11/04/... US Mission technical team
Text and images are subject infringing, contact me delete