Network model - four common network IO models

The network IO model introduces the following four types: synchronous blocking IO, synchronous non-blocking IO, IO multiplexing and asynchronous IO.

1. The principle of IO reading and writing

File reading and writing or socket reading and writing, developed in the Java application layer, are all input or output processing

The user program performs IO operations, depending on the underlying IO implementation, mainly **The underlying read&write two major system calls:**

• The read system call refers to copying data from the kernel buffer to the process buffer
• The write system call refers to copying data from the process buffer to the kernel buffer

That is to say, the IO operation of the upper-level program is actually not the reading and writing at the level of the physical device, but the copying of the cache.

1.1 Kernel buffer and process buffer

1.1.1 User Processes and the Operating System

User process (N) -> in user mode (user space)
system space -> kernel mode

In the user mode, system resources need to be accessed. With the help of the kernel mode , the system resources mainly include:

• 1) cpu : Control the execution of a program
• 2) Input and output : everything is a stream, and all streams need to use the kernel state
• 3) Process management : scheduling between process creation, destruction, blocking, and wake-up
• 4) Memory : application and release of memory
• 5) Inter-process communication : Processes cannot access each other's memory, so the interaction between processes requires communication, which is also a resource

The system resources mentioned above cannot be directly accessed in the user process, and can only be accessed through the operating system, so the function of the operating system to access these resources is called a system call

1.1.2 Purpose of the buffer

In order to reduce frequent system IO calls

The system call needs to switch from the user state to the kernel state . After the switch, the data and status of the user process and other information are saved. After the call is completed, the previous information needs to be replied .In order to reduce the time of this loss, there is also the time of loss of performance, so there is a buffer

With the buffer, the operating system uses the read function to read from the kernel buffercopyTo the process buffer , the write function is copied from the process buffer to the kernel buffer , and only when the data in the buffer reaches a certain amount will the IO system improve performance.

The IO operation of the user program, in most cases, does not perform the actual IO operation, butDirect data exchange between process buffer and kernel buffer。

1.2 The underlying process of Java reading and writing IO

If it is on the Java server side, complete a Socket request and details, the complete process is as follows:

• Client request : Linux reads the client request data through the network card, and reads the data into the kernel buffer .
• Get request data : The Java server reads data from the linux kernel buffer through the read system call , and sends it to the Java process buffer .
• Server-side business processing : The Java server processes client requests in its own user space.
• The server returns data : After the Java server completes the processing, it constructs the response data and writes the data from the user buffer to the kernel buffer . The write system call is used here .
• Send to the client : The Linux kernel writes the data in the kernel buffer into the network card through network IO , and the underlying communication protocol of the network card will send the data to the target client.

2. Four main IO models

2.1 Basic concepts

2.1.1 Blocking and non-blocking

• Blocking IO : After the kernel IO operation is completely completed, it returns to the user space and executes the user's operation

• Non-blocking IO : There is no need to wait for the kernel IO operation to complete before returning to user space

• Blocking/non-blocking refers to the execution state of user space programs

  in other words

• Blocking : User space (calling thread) waits for kernel IO, and does nothing during this period.

• Non-blocking : The user space (calling thread) directly returns to its own space after receiving the state returned by the kernel. IO operations can be done as soon as they can be done, and other things can be done if they can’t be done.

2.1.2 Synchronous and asynchronous

• Synchronous IO : The interaction between user space threads and kernel space threads. The user space thread is the party that initiates the IO request actively , and the kernel space is the party that passively receives
• Asynchronous IO : Contrary to the above, the kernel space is the party that initiates the IO request actively , and the thread in the user space is the passive receiver.

2.1.3 Four IO models

• Synchronous blocking IO (Blocking IO)
• Synchronous non-blocking IO (Non-Blocking IO)
• IO Multiplexing
• Asynchronous IO (Asynchronous IO)

2.2 Synchronous blocking IO (Blocking IO)

In the synchronous blocking IO model, the program starts with an IO call until the system call returns, during which time the process isblockof . After returning successfully, the application starts to process the buffer data in the user space . It is mainly divided into two stages:

• Waiting for data to be ready : Network IO is to wait for the remote data to arrive one after another ; disk IO is to wait until the disk data is read from the disk to the kernel buffer .
• Data copying : The program in user space does not have permission to directly read the data in the kernel buffer (the operating system is for security reasons), so the kernel needs to copy the data in the kernel buffer to the process buffer .

The synchronous blocking IO model is shown in the following figure:

The characteristic of blocking IO is that user threads are blocked during the two stages of IO execution performed by the kernel.

2.3 Synchronous non-blocking IO (None Blocking IO)

Set Socket to non-blocking , the current connection becomes non-blocking IO . IO read and write using non-blocking mode is called synchronous non-blocking IO (None Blocking IO), referred to as NIO model .

In the synchronous non-blocking IO model , the following situations will occur:

• In the case that there is no data in the kernel buffer , the system call will return immediately, returning a message that the call failed. This way the request will not block .
• User threads need to continuously initiate IO system calls ,Tests whether kernel data is ready.
• It is blocked when there is data in the kernel buffer . Until all the data in the kernel buffer is copied to the process buffer , the system call succeeds.

The synchronous non-blocking IO model is shown in the following figure:
Features of synchronous non-blocking IO: The program needs to continuously perform IO system calls , poll whether the data is ready, and continue polling if not .

2.4 IO multiplexing model (IO Multiplexing)

In the IO multiplexing model, a new system call select/epoll is introduced to query the ready status of IO . Multiple file descriptors can be monitored through this system call . Once a descriptor is ready (usually the kernel cache is readable/writable), the kernel can return the ready status to the application. Subsequently, the application performs corresponding IO system calls according to the ready state.

In the IO multiplexing model, through the select/epoll system call , the thread of a single application can continuously poll hundreds of socket connections ,When one or some socket network connections have IO ready status, the corresponding executable read and write operations will be returned。

The IO multiplexing model is shown in the following figure:

Features of the IO multiplexing model: The IO multiplexing model involves two system calls, one is ready query (select/epoll), and the other is IO operation.

Multiplexed IO also requires polling . The thread responsible for the readiness query system call needs to continuously perform select/epoll polling to find out the socket connections that are ready for IO operations .

2.5 Asynchronous IO model (Asynchronous IO)

The asynchronous IO model (Asynchronous IO) is referred to as AIO, and its basic process is: a user thread registers an IO operation with the kernel through a system call . After the entire IO operation (including data preparation and data copying) is completed, the kernel notifies the user program to perform subsequent business operations.

In the asynchronous IO model, the data processing process of the entire kernel includes the kernel reading data from the network physical device (network card) to the kernel buffer, copying the data in the kernel buffer to the user buffer,User programs do not need to block。

The asynchronous IO model is shown in the following figure:

The characteristics of the asynchronous IO model: in the two stages of the kernel waiting for data and copying data , the user thread is not blocked . When the kernel's IO operations (waiting for data and copying data) are all completed, the kernel will notify the application to read the data .

3. Advantages and disadvantages of the four IO models

3.1 Synchronous blocking IO

• Advantages : program development is simple; during blocking and waiting for data, user threads are suspended,Does not occupy CPU resources。
• Disadvantages : One thread maintains the reading and writing of one IO stream . In high-concurrency application scenarios , a large number of threads are required to maintain a large number of network connections , and the memory and thread switching overhead will be huge.The BIO model is not available in high concurrency scenarios。

3.2 Synchronous non-blocking IO

• Advantages : When there is no data in the kernel buffer, the initiated system call will not be blocked, the user program will not be blocked, and the real-time performance is better.
• Disadvantages : It is necessary to repeatedly initiate IO calls , this way of constantly polling and constantly asking the kernel,It will take up a lot of CPU time, and the resource utilization rate is relatively low; It is also blocked when there is data in the kernel buffer.The NIO model is not available in high concurrency scenarios。

3.3 IO multiplexing

• Advantages : select/epoll can handle hundreds of connections at the same time , compared with the previous thread maintaining a connection,IO multiplexing does not require thread creation and maintenance, thereby reducing system overhead.
• Disadvantages : The select/epoll system call is a blocking mode . After the read and write events are ready, the user can read and write by himself, and this read and write process is also blocked .

3.4 Asynchronous I/O

• Advantages :During the two phases of the kernel waiting for data and copying data, the user thread is not blocked。
• Disadvantages : The registration of events is required, and the operating system is required.