How Tomcat is the Servlet asynchronous

Foreword

Tomcat series by my previous article, I believe the students to look at my blog Tomcat should have a clearer understanding, we discussed the first few blog Tomcat in SpringBoot framework is how to start, and discusses the internal components is Tomcat how to design and how to request a transfer, then we talk about here blog asynchronous Tomcat Servlet, Tomcat is how to implement asynchronous and asynchronous Servlet Servlet usage scenarios.

Hand line and an asynchronous Servlet

To implement a Servlet us directly With SpringBoot framework, just to show Servlet Code:

@WebServlet(urlPatterns = "/async",asyncSupported = true)
@Slf4j
public class AsyncServlet extends HttpServlet {

    ExecutorService executorService =Executors.newSingleThreadExecutor();

    @Override
     protected void doGet(HttpServletRequest req, HttpServletResponse resp) throws ServletException, IOException {
        //开启异步,获取异步上下文
        final AsyncContext ctx = req.startAsync();
        // 提交线程池异步执行
        executorService.execute(new Runnable() {


            @Override
            public void run() {
                try {
                    log.info("async Service 准备执行了");
                    //模拟耗时任务
                    Thread.sleep(10000L);
                    ctx.getResponse().getWriter().print("async servlet");
                    log.info("async Service 执行了");
                } catch (IOException e) {
                    e.printStackTrace();
                } catch (InterruptedException e) {
                    e.printStackTrace();
                }
                //最后执行完成后完成回调。
                ctx.complete();
            }
        });
    }

复制代码

The above code implements an asynchronous Servlet, realized doGetCAUTION need to use to start the class plus SpringBoot @ServletComponentScanannotations scan Servlet. Since the code is written, let's look at the actual operating results.

After we send a request to see the page there is a response, at the same time, see the request time spent 10.05s, then we can run the Servlet considered normal friends. Some students will certainly ask, this is not an asynchronous servlet it? Your response time and did not speed up, what use is it? Yes, we can not speed up the response time, or will it depend on our business logic, but after our request for asynchronous servlet, asynchronous execution depends on the business, we can return immediately, that is to say, Tomcat threads can be recovered immediately, By default, Tomcat core thread is 10, the maximum number of threads is 200, we can promptly recover the thread, which means that we can handle more requests, we can increase the throughput, which is the main role of asynchronous Servlet.

Internal principle of asynchronous Servlet

Once you understand the role of asynchronous Servlet, we have to look at, Tomcat Servlet how is the first asynchronous. In fact, the above code, the main core logic is in two parts, final AsyncContext ctx = req.startAsync();and ctx.complete();then we take a look at what they actually do?

   public AsyncContext startAsync(ServletRequest request,
            ServletResponse response) {
        if (!isAsyncSupported()) {
            IllegalStateException ise =
                    new IllegalStateException(sm.getString("request.asyncNotSupported"));
            log.warn(sm.getString("coyoteRequest.noAsync",
                    StringUtils.join(getNonAsyncClassNames())), ise);
            throw ise;
        }

        if (asyncContext == null) {
            asyncContext = new AsyncContextImpl(this);
        }

        asyncContext.setStarted(getContext(), request, response,
                request==getRequest() && response==getResponse().getResponse());
        asyncContext.setTimeout(getConnector().getAsyncTimeout());

        return asyncContext;
    }
复制代码

We found req.startAsync();just saved an asynchronous context, while setting some basic information, such as Timeout, by the way, the default timeout set here is 30S, which means your asynchronous processing logic after the error will exceed 30S, perform this time ctx.complete();will IllegalStateException is thrown.

Let's look at ctx.complete();the logic

  public void complete() {
        if (log.isDebugEnabled()) {
            logDebug("complete   ");
        }
        check();
        request.getCoyoteRequest().action(ActionCode.ASYNC_COMPLETE, null);
    }
//类：AbstractProcessor 
 public final void action(ActionCode actionCode, Object param) {
    case ASYNC_COMPLETE: {
            clearDispatches();
            if (asyncStateMachine.asyncComplete()) {
                processSocketEvent(SocketEvent.OPEN_READ, true);
            }
            break;
        } 
    }
    //类：AbstractProcessor 
protected void processSocketEvent(SocketEvent event, boolean dispatch) {
        SocketWrapperBase<?> socketWrapper = getSocketWrapper();
        if (socketWrapper != null) {
            socketWrapper.processSocket(event, dispatch);
        }
    }
    //类：AbstractEndpoint
public boolean processSocket(SocketWrapperBase<S> socketWrapper,
            SocketEvent event, boolean dispatch) {
        //省略部分代码
            SocketProcessorBase<S> sc = null;
            if (processorCache != null) {
                sc = processorCache.pop();
            }
            if (sc == null) {
                sc = createSocketProcessor(socketWrapper, event);
            } else {
                sc.reset(socketWrapper, event);
            }
            Executor executor = getExecutor();
            if (dispatch && executor != null) {
                executor.execute(sc);
            } else {
                sc.run();
            }
   
        return true;
    }
复制代码

So, eventually I will call here AbstractEndpointthe processSocketmethod, read my earlier blog before students should have the impression that EndPointis used to receive and process the request, the next will be handed over Processorto protocol processing.

类：AbstractProcessorLight
public SocketState process(SocketWrapperBase<?> socketWrapper, SocketEvent status)
            throws IOException {
        //省略部分diam
        SocketState state = SocketState.CLOSED;
        Iterator<DispatchType> dispatches = null;
        do {
            if (dispatches != null) {
                DispatchType nextDispatch = dispatches.next();
                state = dispatch(nextDispatch.getSocketStatus());
            } else if (status == SocketEvent.DISCONNECT) {
            
            } else if (isAsync() || isUpgrade() || state == SocketState.ASYNC_END) {
                state = dispatch(status);
                if (state == SocketState.OPEN) {
                    state = service(socketWrapper);
                }
            } else if (status == SocketEvent.OPEN_WRITE) {
                state = SocketState.LONG;
            } else if (status == SocketEvent.OPEN_READ){
                state = service(socketWrapper);
            } else {
                state = SocketState.CLOSED;
            }

        } while (state == SocketState.ASYNC_END ||
                dispatches != null && state != SocketState.CLOSED);

        return state;
    }
复制代码

This part is the key, AbstractProcessorLightbased on SocketEventthe state judge is not going to call service(socketWrapper), call the method will eventually go to the container, thus completing the call business logic, our request is executed after completion of the call, certainly not into the container, or else is the cycle of death, here by isAsync()judge, will enter dispatch(status), will eventually call CoyoteAdapterthe asyncDispatchmethod

public boolean asyncDispatch(org.apache.coyote.Request req, org.apache.coyote.Response res,
            SocketEvent status) throws Exception {
        //省略部分代码
        Request request = (Request) req.getNote(ADAPTER_NOTES);
        Response response = (Response) res.getNote(ADAPTER_NOTES);
        boolean success = true;
        AsyncContextImpl asyncConImpl = request.getAsyncContextInternal();
        try {
            if (!request.isAsync()) {
                response.setSuspended(false);
            }

            if (status==SocketEvent.TIMEOUT) {
                if (!asyncConImpl.timeout()) {
                    asyncConImpl.setErrorState(null, false);
                }
            } else if (status==SocketEvent.ERROR) {
                
            }

            if (!request.isAsyncDispatching() && request.isAsync()) {
                WriteListener writeListener = res.getWriteListener();
                ReadListener readListener = req.getReadListener();
                if (writeListener != null && status == SocketEvent.OPEN_WRITE) {
                    ClassLoader oldCL = null;
                    try {
                        oldCL = request.getContext().bind(false, null);
                        res.onWritePossible();//这里执行浏览器响应，写入数据
                        if (request.isFinished() && req.sendAllDataReadEvent() &&
                                readListener != null) {
                            readListener.onAllDataRead();
                        }
                    } catch (Throwable t) {
                       
                    } finally {
                        request.getContext().unbind(false, oldCL);
                    }
                } 
                }
            }
            //这里判断异步正在进行，说明这不是一个完成方法的回调，是一个正常异步请求，继续调用容器。
            if (request.isAsyncDispatching()) {
                connector.getService().getContainer().getPipeline().getFirst().invoke(
                        request, response);
                Throwable t = (Throwable) request.getAttribute(RequestDispatcher.ERROR_EXCEPTION);
                if (t != null) {
                    asyncConImpl.setErrorState(t, true);
                }
            }
            //注意，这里，如果超时或者出错，request.isAsync()会返回false，这里是为了尽快的输出错误给客户端。
            if (!request.isAsync()) {
                //这里也是输出逻辑
                request.finishRequest();
                response.finishResponse();
            }
            //销毁request和response
            if (!success || !request.isAsync()) {
                updateWrapperErrorCount(request, response);
                request.recycle();
                response.recycle();
            }
        }
        return success;
    }
复制代码

The above code is the ctx.complete()execution of a final method (of course, many details are omitted), the output data is completed, the final output to the browser.

Here some students might say, I know that after the implementation of asynchronous call ctx.complete()will be output to the browser, but, for the first time doGet request execution is completed, Tomcat is not know how to return it to the client? The key code CoyoteAdapterof servicethe method, part of the code as follows:

  postParseSuccess = postParseRequest(req, request, res, response);
            //省略部分代码
            if (postParseSuccess) {
                request.setAsyncSupported(
                        connector.getService().getContainer().getPipeline().isAsyncSupported());
                connector.getService().getContainer().getPipeline().getFirst().invoke(
                        request, response);
            }
            if (request.isAsync()) {
                async = true;
               } else {
               //输出数据到客户端
                request.finishRequest();
                response.finishResponse();
            if (!async) {
                updateWrapperErrorCount(request, response);
                //销毁request和response
                request.recycle();
                response.recycle();
            }

复制代码

After calling this code Servletafter, it will pass request.isAsync()to determine whether the asynchronous request if an asynchronous request is set async = true. If the request is a non-asynchronous execution logic outputs data to the client, while the destruction requestand response. Here the operation is completed does not respond to a client request ends.

Why Spring Boot of @EnableAsync not comment asynchronous Servlet

Because until you are ready to write this article when he inquired about a lot of information, find a lot of information written SpringBoot asynchronous programming is dependent on the @EnableAsyncnotes, and then Controlleruse multiple threads to complete the business logic, the final summary results, complete return output. Here Nuggets take a big brother to the article, for example, " SpringBoot Asynchronous Programming Guide novice can understand ," the article written very easy to understand, very good, from a business perspective, indeed asynchronous programming, but there are one problem, set aside for parallel processing operations, for the entire request, not asynchronous, meaning not immediately release the Tomcat threads, which can not achieve the effect of asynchronous Servlet. Here I refer to above also wrote a demo, we have to verify at why it is not asynchronous.

@RestController
@Slf4j
public class TestController {
    @Autowired
    private TestService service;

    @GetMapping("/hello")
    public String test() {
        try {
            log.info("testAsynch Start");
            CompletableFuture<String> test1 = service.test1();
            CompletableFuture<String> test2 = service.test2();
            CompletableFuture<String> test3 = service.test3();
            CompletableFuture.allOf(test1, test2, test3);
            log.info("test1=====" + test1.get());
            log.info("test2=====" + test2.get());
            log.info("test3=====" + test3.get());
        } catch (InterruptedException e) {
            e.printStackTrace();
        } catch (ExecutionException e) {
            e.printStackTrace();
        }
        return "hello";
    }
@Service
public class TestService {
    @Async("asyncExecutor")
    public CompletableFuture<String> test1() throws InterruptedException {
        Thread.sleep(3000L);
        return CompletableFuture.completedFuture("test1");
    }

    @Async("asyncExecutor")
    public CompletableFuture<String> test2() throws InterruptedException {
        Thread.sleep(3000L);
        return CompletableFuture.completedFuture("test2");
    }

    @Async("asyncExecutor")
    public CompletableFuture<String> test3() throws InterruptedException {
        Thread.sleep(3000L);
        return CompletableFuture.completedFuture("test3");
    }
}
@SpringBootApplication
@EnableAsync
public class TomcatdebugApplication {

    public static void main(String[] args) {
        SpringApplication.run(TomcatdebugApplication.class, args);
    }

    @Bean(name = "asyncExecutor")
    public Executor asyncExecutor() {
        ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
        executor.setCorePoolSize(3);
        executor.setMaxPoolSize(3);
        executor.setQueueCapacity(100);
        executor.setThreadNamePrefix("AsynchThread-");
        executor.initialize();
        return executor;
    }

复制代码

Here I run to see results

Here after my request, before calling execute business logic container hit a breakpoint, then also hit a breakpoint after returning in Controllerafter the execution, a request was returned CoyoteAdapter, and the judge request.isAsync(), according to figures seen is false, then the next will be executed request.finishRequest()and response.finishResponse()performed in response to the end, and destroy request and response body. Very interesting thing is that I found the time to experiment, performing request.isAsync()before, on the browser page has emerged in response to the body, which is SpringBoot framework has been through StringHttpMessageConverterclass writeInternalmethod has been output.

The core logic of the above analysis is , the thread of execution Tomcat CoyoteAdapterafter calling the container must wait until the request returns, and then determine whether the request is asynchronous, and then processes the request, and then after the implementation of the thread can be recycled. I Asynchronous Servlet examples of a beginning, after the completion of the implementation of doGet method will return immediately, that is, directly to request.isAsync()the logic, then logic to perform the entire thread is completed, the thread is recovered.

Asynchronous Servlet chat usage scenarios

Analyze so much, then the asynchronous Servlet usage scenarios, what does? In fact, as long as we can hold on to some analysis, that is asynchronous Servlet increase the throughput of the system can accept more requests. Assuming that Tomcat web system thread is not enough, a large number of requests waiting for a time when the Web application level system optimization can no longer optimized, that is not to shorten the response time of the business logic, and this time, if you want to reduce the user the waiting time and improve throughput, lower use of asynchronous Servlet can try.

For a practical example : such as making a messaging system, SMS system for high real-time requirements, it requires waiting time as short as possible, and we are actually sending the operator to send a delegate, that we have to call interface , assuming high concurrency, so this time the business system to send text messages to call our function, it is possible to put our Tomcat thread pool is used up, the remaining requests will wait in the queue, and that this time, the message delay will go up, in order to solve this problem, we can introduce asynchronous Servlet, accept more requests to send text messages, thus reducing latency SMS.

to sum up

I began this article from a handwritten asynchronous Servlet, analyzed the role of asynchronous Servlet and how internal Tomcat Servlet is asynchronous, and then I have to be based on the popular Internet SpringBoot asynchronous programming instructions, which are not within Tomcat an asynchronous Servlet. Finally, I talked about asynchronous Servlet usage scenarios analyzed under what circumstances you can try asynchronous Servlet.