关于线程池，池化和重用相对于简单地为每个任务都创建和销毁线程是一种进步，但它并不能消除由于上下文切换所带来的开销。下面我们来看看netty的线程模型，它是如何优化和简化多线程的操作。
netty常用时，配置NioEventLoopGroup，为每个channel绑定相应的EventLoop，EventLoop上绑定的Thread处理相应channel上的所有io操作和事件。
NioEventLoopGroup的构造方法具体逻辑实际上是在其超类MultithreadEventExecutorGroup上
protected MultithreadEventExecutorGroup(int nThreads, ThreadFactory threadFactory, Object... args) {
this.terminationFuture = new DefaultPromise(GlobalEventExecutor.INSTANCE);
if(nThreads <= 0) {
throw new IllegalArgumentException(String.format("nThreads: %d (expected: > 0)", new Object[]{Integer.valueOf(nThreads)}));
} else {
if(threadFactory == null) {
threadFactory = this.newDefaultThreadFactory();
}
this.children = new SingleThreadEventExecutor[nThreads];
int j;
for(int i = 0; i < nThreads; ++i) {
boolean success = false;
boolean var17 = false;
try {
var17 = true;
this.children[i] = this.newChild(threadFactory, args);
success = true;
var17 = false;
} catch (Exception var18) {
throw new IllegalStateException("failed to create a child event loop", var18);
} finally {
if(var17) {
if(!success) {
int j;
for(j = 0; j < i; ++j) {
this.children[j].shutdownGracefully();
}
for(j = 0; j < i; ++j) {
EventExecutor e = this.children[j];
try {
while(!e.isTerminated()) {
e.awaitTermination(2147483647L, TimeUnit.SECONDS);
}
} catch (InterruptedException var19) {
Thread.currentThread().interrupt();
break;
}
}
}
}
}
if(!success) {
for(j = 0; j < i; ++j) {
this.children[j].shutdownGracefully();
}
for(j = 0; j < i; ++j) {
EventExecutor e = this.children[j];
try {
while(!e.isTerminated()) {
e.awaitTermination(2147483647L, TimeUnit.SECONDS);
}
} catch (InterruptedException var21) {
Thread.currentThread().interrupt();
break;
}
}
}
}
FutureListener<Object> terminationListener = new FutureListener<Object>() {
public void operationComplete(Future<Object> future) throws Exception {
if(MultithreadEventExecutorGroup.this.terminatedChildren.incrementAndGet() == MultithreadEventExecutorGroup.this.children.length) {
MultithreadEventExecutorGroup.this.terminationFuture.setSuccess((Object)null);
}
}
};
EventExecutor[] arr$ = this.children;
j = arr$.length;
for(int i$ = 0; i$ < j; ++i$) {
EventExecutor e = arr$[i$];
e.terminationFuture().addListener(terminationListener);
}
}
}
在构造方法中，先保证nThread大于0，它就是EventLoopGroup中线程的数量。主要是创建SingleThreadEventExecutor数组，即存放单个线程的容器，它的大小即为nThread，具体赋值是调用抽象方法newChild()，实现在NioEventLoopGroup中
@Override
protected EventExecutor newChild(
ThreadFactory threadFactory, Object... args) throws Exception {
return new NioEventLoop(this, threadFactory, (SelectorProvider) args[0]);
}
NioEventLoop即是继承自SingleThreadEventExecutor。
NioEventLoop(NioEventLoopGroup parent, ThreadFactory threadFactory, SelectorProvider selectorProvider) {
//调用父类的构造函数
super(parent, threadFactory, false);
//参数检查
if (selectorProvider == null) {
throw new NullPointerException("selectorProvider");
}
provider = selectorProvider;
selector = openSelector();
}
调用父类构造函数，再打开selector。通过回溯NioEventLoop到NioEventLoopGroup构造函数中，我们知道这里的provider是WindowsSelectorProvider类的openSelector()
public AbstractSelector openSelector() throws IOException {
return new WindowsSelectorImpl(this);
}
WindowsSelectorImpl是jdk nio部分的选择器。
继承链上的SingleThreadEventExector给出更详细的构造方法
protected SingleThreadEventExecutor(EventExecutorGroup parent, ThreadFactory threadFactory, boolean addTaskWakesUp) {
this.terminationFuture = new DefaultPromise(GlobalEventExecutor.INSTANCE);
if(threadFactory == null) {
throw new NullPointerException("threadFactory");
} else {
this.parent = parent;
this.addTaskWakesUp = addTaskWakesUp;
this.thread = threadFactory.newThread(new Runnable() {
public void run() {
boolean success = false;
SingleThreadEventExecutor.this.updateLastExecutionTime();
boolean var274 = false;
label2893: {
try {
var274 = true;
SingleThreadEventExecutor.this.run();
success = true;
var274 = false;
break label2893;
} catch (Throwable var296) {
SingleThreadEventExecutor.logger.warn("Unexpected exception from an event executor: ", var296);
var274 = false;
} finally {
......
this.taskQueue = this.newTaskQueue();
}
}
实现过长。。也没太看懂。明显可以看到，在这里生成了个线程赋值给thread，也就是每个eventLoop都与一个thread绑定，生命周期同步。每个对应channel的io跟事件处理线程，就是这个thread成员，在这里直接调用了该执行器的抽象方法run()，跟踪代码后发现，就是执行NioEventLoop的run()
最后，创建任务队列LinkedBlockingQueue作为正在等待执行的task队列。
接下来看下eventloop的run方法，算是核心
protected void run() {
for (; ; ) {
//得到旧值赋值给oldWakenUp，并给wakenUp赋值为false
oldWakenUp = wakenUp.getAndSet(false);
try {
if (hasTasks()) {
selectNow();
} else {
select();
if (wakenUp.get()) {
selector.wakeup();
}
}
cancelledKeys = 0;
//得到执行线程执行IO开始时间，单位毫秒
final long ioStartTime = System.nanoTime();
needsToSelectAgain = false;
/**
* IO任务
*/
//selectedKeys不为null，说明在构造函数中调用openSelector()方法进行初始化selector时，使用的是进行优化后的SelectKeySet
//默认情况下都是使用的是优化后的SelectKeySet，所这里可以先着重看processSelectedKeysOptimized()方法的实现
if (selectedKeys != null) {
processSelectedKeysOptimized(selectedKeys.flip());
} else {
processSelectedKeysPlain(selector.selectedKeys());
}
//用系统当前时间-执行IO的开始时间 = 当前线程执行IO的总耗时
final long ioTime = System.nanoTime() - ioStartTime;
/**
* TODO：think about of:这里任务比例的计算方式
* 表示线程在执行任务过程中，IO任务所占的比例，以百分比为单位的，即：
*
*/
final int ioRatio = this.ioRatio;
/**
* 非IO任务
*/
//运行任务队列中的其他任务
runAllTasks(ioTime * (100 - ioRatio) / ioRatio);
//判断当前evnetloop状态是否正在shuttingDown
if (isShuttingDown()) {
closeAll();
if (confirmShutdown()) {
break;
}
}
} catch (Throwable t) {
logger.warn("Unexpected exception in the selector loop.", t);
// Prevent possible consecutive immediate failures that lead to
// excessive CPU consumption.
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
// Ignore.
}
}
}
}
run方法死循环，分别调用io跟非io任务，循环一开始先判断hasTasks()即阻塞队列中有无未执行任务，有的话SelectNow()，否则select()去轮询channel去取得io信息，selectNow()与select()的区别在于selectNow()没有tiemout，即使channel没有已经就绪的信息也会立即返回。若task队列空了则调用select()。
//todo:通过封装后的select方法
private void select() throws IOException {
Selector selector = this.selector;
try {
int selectCnt = 0;
long currentTimeNanos = System.nanoTime();
long selectDeadLineNanos = currentTimeNanos + delayNanos(currentTimeNanos);
for (; ; ) {
long timeoutMillis = (selectDeadLineNanos - currentTimeNanos + 500000L) / 1000000L;
if (timeoutMillis <= 0) {
if (selectCnt == 0) {
selector.selectNow();
selectCnt = 1;
}
break;
}
//此处select超时后，不会抛出异常
int selectedKeys = selector.select(timeoutMillis);
selectCnt++;
if (selectedKeys != 0 || oldWakenUp || wakenUp.get() || hasTasks()) {
// Selected something,
// waken up by user, or
// the task queue has a pending task.
break;
}
if (SELECTOR_AUTO_REBUILD_THRESHOLD > 0 &&
selectCnt >= SELECTOR_AUTO_REBUILD_THRESHOLD) {
// The selector returned prematurely many times in a row.
// Rebuild the selector to work around the problem.
logger.warn(
"Selector.select() returned prematurely {} times in a row; rebuilding selector.",
selectCnt);
rebuildSelector();
selector = this.selector;
// Select again to populate selectedKeys.
selector.selectNow();
selectCnt = 1;
break;
}
currentTimeNanos = System.nanoTime();
}
if (selectCnt > MIN_PREMATURE_SELECTOR_RETURNS) {
if (logger.isDebugEnabled()) {
logger.debug("Selector.select() returned prematurely {} times in a row.", selectCnt - 1);
}
}
} catch (CancelledKeyException e) {
if (logger.isDebugEnabled()) {
logger.debug(CancelledKeyException.class.getSimpleName() + " raised by a Selector - JDK bug?", e);
}
// Harmless exception - log anyway
}
}
有timeout时间去轮询，直到取到任务队列有选定的东西，用户唤醒，或者有一个新的定时任务时，退出select()。
我们继续看run方法。
先着重看processSelectedKeysOptimized()方法的实现
private void processSelectedKeysOptimized(SelectionKey[] selectedKeys) {
for (int i = 0; ; i++) {
final SelectionKey k = selectedKeys[i];
if (k == null) {
break;
}
final Object a = k.attachment();
if (a instanceof AbstractNioChannel) {
processSelectedKey(k, (AbstractNioChannel) a);
} else {
@SuppressWarnings("unchecked")
NioTask<SelectableChannel> task = (NioTask<SelectableChannel>) a;
processSelectedKey(k, task);
}
if (needsToSelectAgain) {
selectAgain();
// Need to flip the optimized selectedKeys to get the right reference to the array
// and reset the index to -1 which will then set to 0 on the for loop
// to start over again.
//
// See https://github.com/netty/netty/issues/1523
selectedKeys = this.selectedKeys.flip();
i = -1;
}
}
}
遍历接收到的selectKey，再得到channel调用processSelectedKey(k, (AbstractNioChannel) a);如果还需要select调用selectAgain。初始化keys跟i。
processSelectedKey是真正处理selector当有io事件，产生的selectionKey
private static void processSelectedKey(SelectionKey k, AbstractNioChannel ch) {
final NioUnsafe unsafe = ch.unsafe();
if (!k.isValid()) {
// close the channel if the key is not valid anymore
unsafe.close(unsafe.voidPromise());
return;
}
try {
int readyOps = k.readyOps();
// Also check for readOps of 0 to workaround possible JDK bug which may otherwise lead
// to a spin loop
if ((readyOps & (SelectionKey.OP_READ | SelectionKey.OP_ACCEPT)) != 0 || readyOps == 0) {
unsafe.read();
if (!ch.isOpen()) {
// Connection already closed - no need to handle write.
return;
}
}
if ((readyOps & SelectionKey.OP_WRITE) != 0) {
// Call forceFlush which will also take care of clear the OP_WRITE once there is nothing left to write
ch.unsafe().forceFlush();
}
if ((readyOps & SelectionKey.OP_CONNECT) != 0) {
// remove OP_CONNECT as otherwise Selector.select(..) will always return without blocking
// See https://github.com/netty/netty/issues/924
int ops = k.interestOps();
ops &= ~SelectionKey.OP_CONNECT;
k.interestOps(ops);
unsafe.finishConnect();
}
} catch (CancelledKeyException e) {
unsafe.close(unsafe.voidPromise());
}
}
通过selectKey的ops来判断io的类型，根据类型调用usafe的具体底层的io操作。
以usafe.read为例，usafe的一个实现类NioByteUnsafe的read操作
@Override
public void read() {
//得到配置信息
final ChannelConfig config = config();
final ChannelPipeline pipeline = pipeline();
final ByteBufAllocator allocator = config.getAllocator();
final int maxMessagesPerRead = config.getMaxMessagesPerRead();
RecvByteBufAllocator.Handle allocHandle = this.allocHandle;
if (allocHandle == null) {
this.allocHandle = allocHandle = config.getRecvByteBufAllocator().newHandle();
}
if (!config.isAutoRead()) {
removeReadOp();
}
ByteBuf byteBuf = null;
int messages = 0;
boolean close = false;
try {
int byteBufCapacity = allocHandle.guess();
int totalReadAmount = 0;
do {
byteBuf = allocator.ioBuffer(byteBufCapacity);
int writable = byteBuf.writableBytes();
int localReadAmount = doReadBytes(byteBuf);
if (localReadAmount <= 0) {
// not was read release the buffer
byteBuf.release();
close = localReadAmount < 0;
break;
}
//触发fireChannelRead事件
pipeline.fireChannelRead(byteBuf);
byteBuf = null;
if (totalReadAmount >= Integer.MAX_VALUE - localReadAmount) {
// Avoid overflow.
totalReadAmount = Integer.MAX_VALUE;
break;
}
totalReadAmount += localReadAmount;
if (localReadAmount < writable) {
// Read less than what the buffer can hold,
// which might mean we drained the recv buffer completely.
break;
}
} while (++ messages < maxMessagesPerRead);
pipeline.fireChannelReadComplete();
allocHandle.record(totalReadAmount);
if (close) {
closeOnRead(pipeline);
close = false;
}
} catch (Throwable t) {
handleReadException(pipeline, byteBuf, t, close);
}
}
}
封装调用了通过pipline编写的自己业务逻辑的部分，eventloop对io的操作到这里基本上很清晰了。
在完成io后，会计算ioRatio参数，表示了执行io时间与task时间的百分比。如果配置了100，那么执行队列中的任务会直到处理完信息之后开始，并直到处理完队列中的task之后才会继续尝试去取得select key，如果不是100，那么将会给执行队伍中task任务的时间设为执行io数据时间的（100- ioRatio）/ioRatio百分比的timeout。
最后就执行非io操作，通过runAllTasks()方法开始处理阻塞队列中的任务
protected boolean runAllTasks(long timeoutNanos) {
this.fetchFromDelayedQueue();
Runnable task = this.pollTask();
if(task == null) {
return false;
} else {
long deadline = ScheduledFutureTask.nanoTime() + timeoutNanos;
long runTasks = 0L;
long lastExecutionTime;
while(true) {
try {
task.run();
} catch (Throwable var11) {
logger.warn("A task raised an exception.", var11);
}
++runTasks;
if((runTasks & 63L) == 0L) {
lastExecutionTime = ScheduledFutureTask.nanoTime();
if(lastExecutionTime >= deadline) {
break;
}
}
task = this.pollTask();
if(task == null) {
lastExecutionTime = ScheduledFutureTask.nanoTime();
break;
}
}
this.lastExecutionTime = lastExecutionTime;
return true;
}
}
首先通过fetchFromDelayedQueue将延时队列中超过diedline的定时任务放入阻塞队列中，准备开始执行。
最后在timeout时间内，不断执行task任务，时间则到退出。