Article Directory
Series Article Directory
Netty core source code analysis (1), Netty’s server-side startup process source code analysis
Netty core source code analysis (2), Netty’s server-side receiving request process source code analysis
Netty core source code analysis (3) The key to business request execution - ChannelPipeline, ChannelHandler, ChannelHandlerContext source code analysis
Netty core source code analysis (4) heartbeat detection source code analysis
Netty core source code analysis (5) core component EventLoop source code analysis
1. EventLoop source code analysis
We briefly analyzed the source code of NioEventLoopGroup before. Today we analyze the source code executed by EventLoop.
1. NioEventLoop source code
First, let's analyze the class inheritance diagram:
(1) The ScheduledExecutorService interface represents a scheduled task interface, and EventLoop can accept scheduled tasks.
(2) EventLoop interface: Once the Channel is registered, all IO operations corresponding to the Channel will be processed.
(3) The SingleThreadEventExecutor interface indicates that this is a single-threaded thread pool.
(4) EventLoop is a singleton thread pool, which contains an endless loop of threads that are constantly doing three things: listening to ports, processing port events, and processing queue events. Each EventLoop can be bound to multiple Channels, and each Channel can always be handled by only one EventLoop.
2. SingleThreadEventExecutor, the parent interface of EventLoop
SingleThreadEventExecutor is a single-threaded thread pool, which contains the execute method is the source used by EventLoop:
// io.netty.util.concurrent.SingleThreadEventExecutor#execute
@Override
public void execute(Runnable task) {
if (task == null) {
throw new NullPointerException("task");
}
// 是否是当前线程
boolean inEventLoop = inEventLoop();
addTask(task);
if (!inEventLoop) {
// 如果该EventLoop的线程不是当前线程
startThread(); // 开启线程
if (isShutdown() && removeTask(task)) {
// 如果线程已经停止，并且删除任务失败，执行拒绝策略，默认是抛出异常RejectedExecutionException
reject();
}
}
// 如果addTaskWakesUp是false，并且任务不是NonWakeupRunnable类型的，尝试唤醒selector，这个时候，阻塞在selector的线程会立即返回
if (!addTaskWakesUp && wakesUpForTask(task)) {
wakeup(inEventLoop);
}
}
(1) addTask method
// io.netty.util.concurrent.SingleThreadEventExecutor#addTask
protected void addTask(Runnable task) {
if (task == null) {
throw new NullPointerException("task");
}
if (!offerTask(task)) {
reject(task);
}
}
// io.netty.util.concurrent.SingleThreadEventExecutor#offerTask
final boolean offerTask(Runnable task) {
if (isShutdown()) {
reject();
}
return taskQueue.offer(task);
}
(2) startThread method
First determine whether it has been started, and ensure that EventLoop has only one thread. If it has not been started, try to use CAS to change the state state to ST_STARTED, and then call doStartThread to start
// io.netty.util.concurrent.SingleThreadEventExecutor#startThread
private void startThread() {
if (state == ST_NOT_STARTED) {
if (STATE_UPDATER.compareAndSet(this, ST_NOT_STARTED, ST_STARTED)) {
try {
// 启动
doStartThread();
} catch (Throwable cause) {
// 异常回滚
STATE_UPDATER.set(this, ST_NOT_STARTED);
PlatformDependent.throwException(cause);
}
}
}
}
// io.netty.util.concurrent.SingleThreadEventExecutor#doStartThread
private void doStartThread() {
assert thread == null;
// executor就是创建EventLoopGroup时创建的ThreadPerTaskExecutor类，将runnable包装秤Netty的FastThreadLocalThread
executor.execute(new Runnable() {
@Override
public void run() {
thread = Thread.currentThread();
// 判断中断状态
if (interrupted) {
thread.interrupt();
}
boolean success = false;
// 设置最后一次的执行时间
updateLastExecutionTime();
try {
// this就是NioEventLoop，执行NioEventLoop的run方法，这个方法是个死循环，也是整个EventLoop的核心！
SingleThreadEventExecutor.this.run();
success = true;
} catch (Throwable t) {
logger.warn("Unexpected exception from an event executor: ", t);
} finally {
// 使用CAS不断修改state状态，改成ST_SHUTTING_DOWN
for (;;) {
int oldState = state;
if (oldState >= ST_SHUTTING_DOWN || STATE_UPDATER.compareAndSet(
SingleThreadEventExecutor.this, oldState, ST_SHUTTING_DOWN)) {
break;
}
}
// Check if confirmShutdown() was called at the end of the loop.
if (success && gracefulShutdownStartTime == 0) {
logger.error("Buggy " + EventExecutor.class.getSimpleName() + " implementation; " +
SingleThreadEventExecutor.class.getSimpleName() + ".confirmShutdown() must be called " +
"before run() implementation terminates.");
}
try {
// Run all remaining tasks and shutdown hooks.
for (;;) {
if (confirmShutdown()) {
break;
}
}
} finally {
try {
// cheanup
cleanup();
} finally {
// 修改状态ST_TERMINATED
STATE_UPDATER.set(SingleThreadEventExecutor.this, ST_TERMINATED);
threadLock.release();
if (!taskQueue.isEmpty()) {
logger.warn(
"An event executor terminated with " +
"non-empty task queue (" + taskQueue.size() + ')');
}
// 回调terminationFuture方法
terminationFuture.setSuccess(null);
}
}
}
}
});
}
3. The run method of NioEventLoop (core!)
This method is an endless loop, and it is also the core of the entire NioEventLoop!
From the source code, we can see that the run method does three things in total:
(1) select to get interested events.
(2) processSelectedKeys processing events.
(3) runAllTasks executes the tasks in the queue.
// io.netty.channel.nio.NioEventLoop#run
@Override
protected void run() {
for (;;) {
try {
switch (selectStrategy.calculateStrategy(selectNowSupplier, hasTasks())) {
case SelectStrategy.CONTINUE:
continue;
case SelectStrategy.SELECT:
// select
select(wakenUp.getAndSet(false));
// 'wakenUp.compareAndSet(false, true)' is always evaluated
// before calling 'selector.wakeup()' to reduce the wake-up
// overhead. (Selector.wakeup() is an expensive operation.)
//
// However, there is a race condition in this approach.
// The race condition is triggered when 'wakenUp' is set to
// true too early.
//
// 'wakenUp' is set to true too early if:
// 1) Selector is waken up between 'wakenUp.set(false)' and
// 'selector.select(...)'. (BAD)
// 2) Selector is waken up between 'selector.select(...)' and
// 'if (wakenUp.get()) { ... }'. (OK)
//
// In the first case, 'wakenUp' is set to true and the
// following 'selector.select(...)' will wake up immediately.
// Until 'wakenUp' is set to false again in the next round,
// 'wakenUp.compareAndSet(false, true)' will fail, and therefore
// any attempt to wake up the Selector will fail, too, causing
// the following 'selector.select(...)' call to block
// unnecessarily.
//
// To fix this problem, we wake up the selector again if wakenUp
// is true immediately after selector.select(...).
// It is inefficient in that it wakes up the selector for both
// the first case (BAD - wake-up required) and the second case
// (OK - no wake-up required).
if (wakenUp.get()) {
selector.wakeup();
}
// fall through
default:
}
cancelledKeys = 0;
needsToSelectAgain = false;
final int ioRatio = this.ioRatio;
if (ioRatio == 100) {
try {
// 处理select keys
processSelectedKeys();
} finally {
// Ensure we always run tasks.
runAllTasks();
}
} else {
final long ioStartTime = System.nanoTime();
try {
processSelectedKeys();
} finally {
// Ensure we always run tasks.
final long ioTime = System.nanoTime() - ioStartTime;
runAllTasks(ioTime * (100 - ioRatio) / ioRatio);
}
}
} catch (Throwable t) {
handleLoopException(t);
}
// Always handle shutdown even if the loop processing threw an exception.
try {
if (isShuttingDown()) {
closeAll();
if (confirmShutdown()) {
return;
}
}
} catch (Throwable t) {
handleLoopException(t);
}
}
}
（1）select
The general logic is: call the select method of NIO's selector, and block for one second by default. If there is a scheduled task, add 0.5 seconds to the remaining time of the scheduled task to block. When the execute method is executed, that is, when a task is added, the selector will be woken up to prevent the selector from blocking for too long.
// io.netty.channel.nio.NioEventLoop#select
private void select(boolean oldWakenUp) throws IOException {
Selector selector = this.selector;
try {
int selectCnt = 0;
long currentTimeNanos = System.nanoTime();
long selectDeadLineNanos = currentTimeNanos + delayNanos(currentTimeNanos);
for (;;) {
long timeoutMillis = (selectDeadLineNanos - currentTimeNanos + 500000L) / 1000000L;
if (timeoutMillis <= 0) {
if (selectCnt == 0) {
selector.selectNow();
selectCnt = 1;
}
break;
}
// If a task was submitted when wakenUp value was true, the task didn't get a chance to call
// Selector#wakeup. So we need to check task queue again before executing select operation.
// If we don't, the task might be pended until select operation was timed out.
// It might be pended until idle timeout if IdleStateHandler existed in pipeline.
if (hasTasks() && wakenUp.compareAndSet(false, true)) {
selector.selectNow();
selectCnt = 1;
break;
}
// 阻塞给定时间，默认一秒
int selectedKeys = selector.select(timeoutMillis);
selectCnt ++;
// 如果有返回值||select被用户唤醒||任务队列有任务||有定时任务即将被执行，则跳出循环
if (selectedKeys != 0 || oldWakenUp || wakenUp.get() || hasTasks() || hasScheduledTasks()) {
// - Selected something,
// - waken up by user, or
// - the task queue has a pending task.
// - a scheduled task is ready for processing
break;
}
if (Thread.interrupted()) {
// Thread was interrupted so reset selected keys and break so we not run into a busy loop.
// As this is most likely a bug in the handler of the user or it's client library we will
// also log it.
//
// See https://github.com/netty/netty/issues/2426
if (logger.isDebugEnabled()) {
logger.debug("Selector.select() returned prematurely because " +
"Thread.currentThread().interrupt() was called. Use " +
"NioEventLoop.shutdownGracefully() to shutdown the NioEventLoop.");
}
selectCnt = 1;
break;
}
long time = System.nanoTime();
if (time - TimeUnit.MILLISECONDS.toNanos(timeoutMillis) >= currentTimeNanos) {
// timeoutMillis elapsed without anything selected.
selectCnt = 1;
} else if (SELECTOR_AUTO_REBUILD_THRESHOLD > 0 &&
selectCnt >= SELECTOR_AUTO_REBUILD_THRESHOLD) {
// The selector returned prematurely many times in a row.
// Rebuild the selector to work around the problem.
logger.warn(
"Selector.select() returned prematurely {} times in a row; rebuilding Selector {}.",
selectCnt, selector);
rebuildSelector();
selector = this.selector;
// Select again to populate selectedKeys.
selector.selectNow();
selectCnt = 1;
break;
}
currentTimeNanos = time;
}
if (selectCnt > MIN_PREMATURE_SELECTOR_RETURNS) {
if (logger.isDebugEnabled()) {
logger.debug("Selector.select() returned prematurely {} times in a row for Selector {}.",
selectCnt - 1, selector);
}
}
} catch (CancelledKeyException e) {
if (logger.isDebugEnabled()) {
logger.debug(CancelledKeyException.class.getSimpleName() + " raised by a Selector {} - JDK bug?",
selector, e);
}
// Harmless exception - log anyway
}
}
4. Summary
Each execution of the ececute method adds a task to the queue. When the thread is added for the first time, the run method is executed, and the run method is the core of the entire EventLoop, just like the name of the EventLoop, Loop Loop, non-stop Loop, what does Loop do? Do 3 things.
- Call the select method of the selector. The default is one second. If there is a scheduled task, it will be blocked by adding 0.5 seconds to the remaining time of the scheduled task. When the execute method is executed, that is, when a task is added, the selecor is woken up to prevent the selector from being blocked for too long.
- When the selector returns, the processSelectedKeys method will be called to process the selectKey.
- When the processSelectedKeys method is executed, the runAlITasks method is executed according to the ratio of ioRatio. By default, the IO task time and non-IO task time are the same, and you can also tune it according to your application characteristics. For example, if there are many non-IO tasks, then you should reduce the ioRatio so that the non-IO tasks can be executed longer. Prevent the queue from accumulating too many tasks.
At this point, we have finished analyzing the source code circled in red in the figure below.
