Author: Zen and the Art of Computer Programming
1 Introduction
As one of the most popular Java Web frameworks, Spring Boot is widely used in internal project development of Internet companies. Spring Boot allows developers to quickly build product-level, reliable web applications through features that are greater than configuration. However, in an actual production environment, in the face of high concurrency and a large number of requests, how to improve the performance of the Spring Boot web server is an important issue. This article will discuss two aspects:
1. Using the non-blocking I/O model to improve the processing capabilities of the web server;
2. Using the asynchronous programming model to achieve better request response delay.
2. Explanation of basic concepts and terms
In order to understand the content of this article, you need to understand the following basic knowledge. If you already understand these concepts, you can jump directly to the third part "Explanation of Core Algorithm Principles, Specific Operation Steps, and Mathematical Formulas".
1. Synchronous blocking I/O model (BIO)
The synchronous blocking I/O model is the simplest I/O model. In this model, the server process waits for the client's request to be completed before making the next request. This approach seriously affects the server's concurrent processing capabilities. When the number of clients is small or the network load is not high, this model can also achieve better performance. However, when the number of clients increases or the network load increases, BIO cannot meet the demand. Therefore, in actual production environments, more complex I/O models are often chosen.
2. Non-blocking I/O model (NIO)
NIO (Non-blocking IO) is an I/O model. It is different from the synchronous blocking I/O model. NIO allows the server process to handle multiple client requests at the same time, even if the client The number of client requests far exceeds the server's processing capabilities. However, NIO also has some shortcomings, such as buffer management. The SocketChannel interface is provided after JDK7.0, which uses Selector objects to implement non-blocking I/O. In addition, OpenJDK9.0 version also provides the AsynchronousFileChannel class, which provides a non-blocking file channel access API. In general, the non-blocking I/O model can maximize the concurrent processing capabilities of the server.
3.Reactor mode
Reactor mode is an event-driven multiplexed IO model that uses a single thread to handle all client requests. The Reactor pattern consists of three parts: event dispatcher, event processor and server or client. The event dispatcher monitors the server port and creates a new event handler when there is a client connection request. The event processor reads the client request data, passes the request to the server's logical processing module, and then returns the response information. The server-side logic processing module processes the request and sends a response to the client. Although the Reactor mode can improve the concurrent processing capabilities of the server, it still has some defects, such as waste of system resources.
4.Async/Await
Async/Await is a keyword used to handle asynchronous I/O. It can write code sequentially like synchronous programming, but the underlying layer still uses the Reactor pattern. Async/Await only simplifies the asynchronous programming process and does not change the underlying I/O model. You still need to use the Reactor mode to implement asynchronous I/O.
5. Servlet, JSP, Struts2, SpringMVC and other frameworks
The technologies involved in this article all belong to Web development-related frameworks, such as Servlet, JSP, Struts2, SpringMVC and other frameworks. Among them, Servlet is a specification in Java that defines how to develop Web applications based on the HTTP protocol. JSP (Java Server Pages) is a Java technology used to dynamically generate Web pages and act as a template language. Struts2 is an open source framework of Apache, used to build enterprise-level web applications. SpringMVC is currently the most popular Spring-based MVC framework.
6. Concurrent programming model
There are many concurrent programming models, mainly including shared memory (such as Java, C++), message queue (such as Kafka), pipeline (such as UNIX shell) and actor-based model (such as Erlang, Elixir). The concurrent programming model used in this article is based on the message queue model, which uses the message queue mechanism to implement an asynchronous programming model.
3. Explanation of core algorithm principles, specific operating steps and mathematical formulas
As can be seen from the above basic knowledge introduction, to improve the processing capabilities of the Spring Boot web server, you must first use the non-blocking I/O model to achieve concurrent processing. Therefore, next, I will explore how to improve the processing capabilities of the Spring Boot web server by combining Java NIO technology, Reactors mode, message queue and other technologies.
1. Reactor mode implements Spring Boot web server
Reactor mode is an event-driven multiplexed IO model. It uses a single thread to run and handle all client requests. The Reactor pattern consists of three parts: event dispatcher, event processor and server or client. The event dispatcher monitors the server port and creates a new event handler when there is a client connection request. The event processor reads the client request data, passes the request to the server's logical processing module, and then returns the response information. The server-side logic processing module processes the request and sends a response to the client.
In the Spring Boot web server, you can use Netty or Undertow to implement the Reactor mode. They are both asynchronous, non-blocking, lightweight web servers that support new protocols such as HTTP/2, WebSocket, and SSL/TLS. Below, we take Netty as an example to explain its implementation process.
First, create a Maven project and import dependencies.
<dependency>
<groupId>io.netty</groupId>
<artifactId>netty-all</artifactId>
<version>4.1.48.Final</version>
</dependency>
Then, create NettyWebServer, inherit from the WebServer class, and implement your own start method. In the start method, initialize the EventLoopGroup, set startup parameters, create ServerBootstrap, bind the listening address, register the read event handler, etc.
public class NettyWebServer implements WebServer {
private final EventLoopGroup group;
private volatile boolean running = false;
public NettyWebServer() {
this.group = new NioEventLoopGroup();
}
@Override
public void start(int port) throws Exception {
if (running) {
return;
}
try {
// Create server bootstrap
Bootstrap b = new ServerBootstrap();
// Set up options and handlers
b.option(ChannelOption.SO_BACKLOG, 1024);
// Set up child channel handler pipeline for each client connection
b.childHandler(new ChannelInitializer<SocketChannel>() {
@Override
protected void initChannel(SocketChannel ch) throws Exception {
ch.pipeline().addLast("http",
new HttpServerCodec());
ch.pipeline().addLast("chunkedWriter",
new ChunkedWriteHandler());
ch.pipeline().addLast("handler",
new HttpRequestHandler());
}
});
// Bind and register to the event loop
ChannelFuture f = b.bind(port).sync();
System.out.println("Netty HTTP server started on " + port);
// Wait until the server socket is closed
f.channel().closeFuture().sync();
} finally {
group.shutdownGracefully();
}
}
}
Next, create an HttpRequestHandler, inherit from SimpleChannelInboundHandler, and override the channelRead0 method, which is responsible for reading the client request data and parsing it into an HttpMessage.
public class HttpRequestHandler extends SimpleChannelInboundHandler<FullHttpRequest> {
private static final Logger LOGGER = LoggerFactory.getLogger(HttpRequestHandler.class);
@Override
protected void channelRead0(ChannelHandlerContext ctx, FullHttpRequest msg) throws Exception {
String uri = msg.uri();
// Process request here...
// Send response back to the client
sendResponse(ctx);
}
private void sendResponse(ChannelHandlerContext ctx) {
// Compose the response message content...
// Write a response message back to the client
DefaultFullHttpResponse res = new DefaultFullHttpResponse(
HttpVersion.HTTP_1_1, HttpResponseStatus.OK,
Unpooled.copiedBuffer(responseContent, CharsetUtil.UTF_8));
// Set headers
HttpHeaders httpHeaders = res.headers();
httpHeaders.set(HttpHeaders.Names.CONTENT_TYPE, "text/plain");
httpHeaders.setInt(HttpHeaders.Names.CONTENT_LENGTH, responseContent.length());
// Write response back to the client
ctx.writeAndFlush(res).addListener((ChannelFutureListener) future -> {
if (!future.isSuccess()) {
LOGGER.error("Failed to write data: ", future.cause());
} else {
LOGGER.info("Data written successfully");
}
});
}
/* Other implementation methods */
}
Finally, call the start method of NettyWebServer to start the Netty Web server.
public static void main(String[] args) {
int port = Integer.parseInt(args[0]);
try {
NettyWebServer server = new NettyWebServer();
server.start(port);
} catch (Exception e) {
e.printStackTrace();
}
}
At this point, the processing capabilities of the Spring Boot web server have been improved. However, due to the limitations of the Reactor mode, it can only handle I/O-intensive requests. This mode cannot be used if you encounter CPU-intensive requests (such as computationally intensive tasks). So, how to improve the processing capabilities of the Spring Boot web server and realize the processing of CPU-intensive requests?
The key to solving this problem is to use asynchronous programming models, such as asynchronous callbacks, Future, CompletableFuture, etc. As shown in the figure below, using an asynchronous programming model can effectively avoid competition for CPU resources.
Note: The CPU-intensive tasks (computing-intensive tasks) in the figure require a large amount of CPU resources and cannot be used exclusively by a single thread. Therefore, these tasks need to be allocated to different threads to achieve concurrent execution. Next, we will introduce in detail the asynchronous programming model based on the message queue and how to implement asynchronous processing of CPU-intensive requests in the Spring Boot web server.
2. Asynchronous processing of CPU-intensive requests based on message queue
In the previous analysis, it was mentioned that the Reactor mode is used to process I/O-intensive requests and the message queue is used to process CPU-intensive requests. This section will describe in detail the advantages and implementation methods of the asynchronous programming model based on message queues.
Asynchronous programming model: The asynchronous programming model means that developers use programming to solve time-consuming operations that may be encountered, instead of letting the current thread wait for the result to be returned before continuing to the next step. Commonly used asynchronous programming models include callback functions, Future and CompletableFuture.
Future and CompletableFuture are interfaces introduced in Java 5 that represent a value that represents a result that may not be completed yet. Normally, Future only focuses on the execution results of tasks, and CompletableFuture can help us manage Future objects.
Future and CompletableFuture are similar in some ways, but they are also different. Future represents the final result of an asynchronous task. It only has a value when the task is completed, and the value type is the same as the task type. CompletableFuture provides additional methods to help us manage asynchronous operations and can specify the result type of the task.
Asynchronous callback: Asynchronous callback, also called task callback, means that when an operation is completed, we want to perform a specific action immediately. Generally speaking, asynchronous callback means that a method can pass in a callback method, and when the asynchronous operation is completed, the callback method will be automatically executed.
Asynchronous programming model based on message queue: The asynchronous programming model based on message queue means that developers implement asynchronous programming model by passing messages. Developers can put time-consuming operations into the message queue and subscribe to the corresponding topic. When the message is published to the topic, the subscriber can receive the notification and perform the corresponding action at the appropriate time.
Asynchronous processing of CPU-intensive requests in Spring Boot web server: Asynchronous processing of CPU-intensive requests in Spring Boot web server mainly relies on Spring annotations, annotation-driven AOP, and message queues.
1. Use annotations to implement asynchronous processing
First, create a controller class, use the RequestMapping annotation to annotate the GetMapping annotation, and add the corresponding processing method.
@RestController
public class AsyncController {
@GetMapping("/async")
public Callable<String> async() throws InterruptedException {
return () -> {
TimeUnit.SECONDS.sleep(5); // Simulate CPU-consuming operation
return "Hello World!";
};
}
}
In the above code, we added a method called async, which returns a Callable object that represents a task that may take a long time. We assume that this task is a computationally intensive task, which requires a large amount of CPU resources. When the user requests this method, our goal is to return the result as quickly as possible, rather than waiting for the task to complete, because this will cause the user to wait for a long time.
Next, we use Spring annotations to enable asynchronous processing. First, we need to add the spring.aop.auto=true attribute in the application.properties file so that the Spring AOP proxy can take effect. Then, we use the @EnableAsync annotation on the controller class AsyncController to enable asynchronous processing.
@SpringBootApplication
@EnableAsync
public class Application {
public static void main(String[] args) {
SpringApplication.run(Application.class, args);
}
}
At this point, we can already use annotations to implement asynchronous processing. When the user requests /async, the server will immediately return a Future object, and the user can obtain the calculation result through the get() method of the Future object.
```java
@RestController
@EnableAsync
public class AsyncController {
@Autowired
private ThreadPoolTaskExecutor taskExecutor;
@GetMapping("/async")
public Callable<String> async() throws InterruptedException {
return () -> {
TimeUnit.SECONDS.sleep(5); // Simulate CPU-consuming operation
return "Hello World!";
};
}
}
```java
@Service
public class MyService {
public MyObject callSomeMethodWhichTakesLongTime() throws InterruptedException {
// do some work which takes long time
return resultObject;
}
}
@RestController
@EnableAsync
public class Controller {
@Autowired
private MyService myService;
@GetMapping("/callAsyncMethod")
public DeferredResult<MyObject> getAsyncValue() throws InterruptedException {
final DeferredResult<MyObject> deferredResult = new DeferredResult<>();
taskExecutor.execute(() -> {
try {
MyObject resultObject = myService.callSomeMethodWhichTakesLongTime();
deferredResult.setResult(resultObject);
} catch (InterruptedException ex) {
Thread.currentThread().interrupt();
throw ex;
}
});
return deferredResult;
}
}
At this point, we can already use annotations + asynchronous callbacks to implement asynchronous processing of CPU-intensive requests in the Spring Boot web server.
2. Use RabbitMQ to implement asynchronous processing
If we use Spring Messaging as the implementation of the message queue, we can use RabbitMQ to implement asynchronous processing based on the message queue. First, we need to install and start RabbitMQ, and then add RabbitMQ-related dependencies in the pom.xml file.
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-amqp</artifactId>
</dependency>
<!-- Add spring messaging RabbitMQ support -->
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-stream-binder-rabbit</artifactId>
</dependency>
<!-- For annotation based configuration of RabbitMQ binding -->
<dependency>
<groupId>org.springframework.integration</groupId>
<artifactId>spring-integration-amqp</artifactId>
</dependency>
Then, we can configure RabbitMQ-related connection information in the configuration file application.yaml, as shown below. Here, we configured an exchange with the name myExchange, the queue name with myQueue, and the binding key with myKey. We can also configure multiple binding keys to implement asynchronous processing of multiple message types.
rabbitmq:
host: localhost
port: 5672
username: guest
password: guest
listener:
simple:
concurrency: 5 # Concurrency level for blocking queue consumers.
template:
exchange: myExchange # The name of the exchange.
routing-key: myKey # The routing key for bindings.
receive-timeout: 1000 # Milliseconds to wait for a message when'receive' or'sendAndReceive' are used without a timeout value.
Next, we can declare a bean in the configuration file, whose type is RabbitTemplate, to easily send messages.
@Configuration
public class MyConfig {
@Bean
public RabbitTemplate rabbitTemplate(final ConnectionFactory cf) {
final RabbitTemplate rt = new RabbitTemplate(cf);
rt.setMandatory(true); // Throw exceptions if there is no connection set up instead of returning null values.
rt.setReplyTimeout(10000); // Milliseconds to wait for a reply from broker while executing a RPC method.
return rt;
}
}
Finally, we can use the @Async annotation in the controller class to annotate methods that require asynchronous processing and publish the message to the message queue.
@RestController
@EnableAsync
public class MessagePublisherController {
private final RabbitTemplate rabbitTemplate;
@Autowired
public MessagePublisherController(final RabbitTemplate rabbitTemplate) {
this.rabbitTemplate = rabbitTemplate;
}
@Async("asyncTaskExecutor")
public Future<Void> publishMessageToTopic(final String message) {
rabbitTemplate.convertAndSend("myExchange", "myKey", message);
return new AsyncResult<>(null);
}
}
Here, we use the @Async annotation and specify it as AsyncTaskExecutor, which will automatically delegate the method to the thread pool for execution. In the publishMessageToTopic method of the controller class, we use the convertAndSend method of RabbitTemplate to publish the message to the myKey route of myExchange. We don't need to get the return value, so we use the AsyncResult wrapper.
At this point, we have been able to use Spring Messaging + RabbitMQ to implement asynchronous processing of CPU-intensive requests in the Spring Boot web server.