Until now, as it applies to serving http requests, I thought the terms - asynchronous and non-blocking i/o meant the same thing. But apparently, they have been implemented separately in servlet 3.0 and 3.1 respectively. I am struggling to understand the difference here...
Can someone shed more light on this topic, please? Specifically, I am looking for an example of how a servlet 3.0 implementation of a server can be async, yet block on a thread? I think may be if I understand this, it may be easier to understand the exact problem that the non-blocking i/o in servlet 3.1 is trying to solve.
I will try to summarize what I learned. To understand the problem that Servlet 3.0 and Servlet 3.1 solve, let's look at it this way:
Prior to Servlet 3.0:
The problem with synchronous processing of requests is that it resulted in threads (doing heavy-lifting) running for a long time before the response goes out. If this happens at scale, the servlet container eventually runs out of threads - long running threads lead to thread starvation.
Prior to Servlet 3.0, there were container specific solutions for these long running threads where we can spawn a separate worker thread to do the heavy task and then return the response to client. The servlet thread returns to the servlet pool after starting the worker thread. Tomcat’s Comet, WebLogic’s FutureResponseServlet and WebSphere’s Asynchronous Request Dispatcher are some of the example of implementation of asynchronous processing.
(See link 1 for more info.)
Servlet 3.0 Async:
The actual work could be delegated to a thread pool implementation (independent of the container specific solutions). The Runnable implementation will perform the actual processing and will use the AsyncContext to either dispatch the request to another resource or write the response. We can also add AsyncListener implementation to the AsyncContext object to implement callback methods.
(See link 1 for more info.)
Servlet 3.1 NIO:
As described above, Servlet 3.0 allowed asynchronous request processing but only traditional I/O (as opposed to NIO) was permitted. Why is traditional I/O a problem?
In traditional I/O, there are two scenarios to consider:
- If the data coming into the server (I/O) is blocking or streamed slower than the server can read, then the server thread that is trying to read this data has to wait for that data.
- On the other hand, if the response data from the server written to
ServletOutputStreamis slow, the client thread has to wait. In both cases, the server thread doing the traditional I/O (for requests/responses) blocks.
In other words, with Servlet 3.0, only the request processing part became async, but not the I/O for serving the requests and responses. If enough threads block, this results in thread starvation and affects performance.
With Servlet 3.1 NIO, this problem is solved by ReadListener and WriteListener interfaces. These are registered in ServletInputStream and ServletOutputStream. The listeners have callback methods that are invoked when the content is available to be read or can be written without the servlet container blocking on the I/O threads. So these I/O threads are freed up and can now serve other request increasing performance.
(See link 2 for more info.)
Credits