EventLoop and EventLoopGroup 6. Netty source code analysis of the

First, the relationship NioEventLoop and NioEventLoopGroup

Two, NioEventLoop

1. Design Principles

1. Responsible IO read and write

2. Perform the task. By calling NioEventLoop's execute (Runnable task) method. We know that, in order to prevent competition for resources and concurrent operation, we often judge the current operation thread is a thread EventLoop, if not, the operation encapsulated into NioEventLoop task execution queue, thus achieving a partial lock-free technology.

3. regular tasks. (, Long delay, TimeUnit unit Runnable command) method is implemented by calling NioEventLoop of schedule.

2. The class hierarchy diagram

3. member variable initialization and Selector

　　Here you can see, NioEventLoop Selector holds a reference to the charge of polling readiness Channel.

Selector Selector; // multiplexer 
Private SelectedSelectionKeySet the selectedKeys;
 Private  Final SelectorProvider Provider; // Selector producers 

Private Selector the openSelector () {
     Final Selector Selector;
     the try {
         // 1. Open Selector 
        Selector = provider.openSelector (); 
    } the catch (IOException E) {
         the throw  new new ChannelException ( "A new new open failed to Selector" , E); 
    } 

    // 2. SelectionKey optimization is turned off by default, returns directly Selector 
    iF (DISABLE_KEYSET_OPTIMIZATION) {
        return selector;
    }

    // 3. 用自己的SelectedSelectionKeySet代替Java自带的selectedKeys
    try {
        SelectedSelectionKeySet selectedKeySet = new SelectedSelectionKeySet();

        Class<?> selectorImplClass =
                Class.forName("sun.nio.ch.SelectorImpl", false, ClassLoader.getSystemClassLoader());

        // Ensure the current selector implementation is what we can instrument.
        if (!selectorImplClass.isAssignableFrom(selector.getClass())) {
            return selector;
        }

        Field selectedKeysField = selectorImplClass.getDeclaredField("selectedKeys");
        Field publicSelectedKeysField = selectorImplClass.getDeclaredField("publicSelectedKeys");

        selectedKeysField.setAccessible(true);
        publicSelectedKeysField.setAccessible(true);

        selectedKeysField.set(selector, selectedKeySet);
        publicSelectedKeysField.set(selector, selectedKeySet);

        selectedKeys = selectedKeySet;
        logger.trace("Instrumented an optimized java.util.Set into: {}", selector);
    } catch (Throwable t) {
        selectedKeys = null;
        logger.trace("Failed to instrument an optimized java.util.Set into: {}", selector, t);
    }

    return selector;
}

4. run method

　　NioEventLoop the most important methods, unlimited poll prepared Channel and processed.

　　First wakenUp reduced to false, and save the state before. By hasTash () to determine whether the current message queue pending message if there is called selectNow () method to conduct a select operation immediately to see if there is readiness Channel.

protected void run() {
    for (;;) {
        oldWakenUp = wakenUp.getAndSet(false);
        try {
            if (hasTasks()) {
                selectNow();
            } else {
                select();

                if (wakenUp.get()) {
                    selector.wakeup();
                }
            }
...

　　selectorNow () method will trigger an immediate selector select operation, if the readiness Channel, Channel-ready collection is returned, otherwise 0. After the operation is complete, again to determine whether the user invokes the Selector's wakenUp () method, if the call is executed selector.wakenUp () operation.

    void selectNow() throws IOException {
        try {
            selector.selectNow();
        } finally {
            // restore wakup state if needed
            if (wakenUp.get()) {
                selector.wakeup();
            }
        }
    }

　　If the message queue is not pending messages, select the method is performed by the selector polling to see if the readiness Channel.

Private  void SELECT () throws IOException { 
    Selector Selector = the this .selector;
     the try {
         int selectCnt = 0 ;
         Long currentTimeNanos = to System.nanoTime ();
         // current time = time + time delay timed task task trigger event 
        Long selectDeadLineNanos = currentTimeNanos + delayNanos (currentTimeNanos);
         for (;;) {
             // timeout timer task trigger time = - + 0.5 ms current time adjustment value -> converted to milliseconds 
            Long timeoutMillis = (selectDeadLineNanos - currentTimeNanos + 500000L) / 1000000L ;
             //If immediate execution (timed out), then the polling 
            IF (timeoutMillis <= 0 ) {
                 IF (selectCnt == 0 ) { 
                    selector.selectNow (); 
                    selectCnt =. 1 ; 
                } 
                BREAK ; 
            } 
            
            // blocking the timeout time, wait for a timer task, execute the SELECT 
            int selectedKeys = selector.select (timeoutMillis); 
            selectCnt ++ ; 

            IF (! = 0 || oldWakenUp selectedKeys wakenUp.get || () || hasTasks ()) {
                 // there is a new event to be processed || the user calls wakenUp () || multiplexer wakeup message queue new task 
                BREAK ; 
            }
            
            // If this is an empty polling, it is possible to start the epoll bug JDK, it will lead to empty polling selector, the IO thread has been in a state of 100%
             // judges the empty polling, continuously if within a certain period empty polling occurred N times, described triggered bugs
             // need to rebuild the selector, the selector Channel original register selector to the new, old and closed selector 
            IF (SELECTOR_AUTO_REBUILD_THRESHOLD> 0 && 
                    selectCnt > = SELECTOR_AUTO_REBUILD_THRESHOLD) {
                 // . MANY Times of The Selector returned prematurely in Row A
                 // the Rebuild The Selector The problem to Work around. 
                logger.warn (
                         "Selector.select () {} Times returned prematurely in Row A;. Rebuilding Selector" , 
                        selectCnt);

                rebuildSelector();
                selector = this.selector;

                // Select again to populate selectedKeys.
                selector.selectNow();
                selectCnt = 1;
                break;
            }

            currentTimeNanos = System.nanoTime();
        }

        if (selectCnt > MIN_PREMATURE_SELECTOR_RETURNS) {
            if (logger.isDebugEnabled()) {
                logger.debug("Selector.select() returned prematurely {} times in a row.", selectCnt - 1);
            }
        }
    } catch (CancelledKeyException e) {
        if (logger.isDebugEnabled()) {
            logger.debug(CancelledKeyException.class.getSimpleName() + " raised by a Selector - JDK bug?", e);
        }
        // Harmless exception - log anyway
    }
}

　　If the polling SocketChannel to a state of readiness, we need to deal with IO event.

　　After processing the IO event, NioEventLoop also need to handle the task and timing of non-IO task. In order to ensure IO event and task execution has enough CPU event, where calculations are performed non-event event with IO IO event accounting (default 50%), while performing task, if more than this time, then directly back in the queue after leaving the task to execute (here Netty order to improve performance, each judge to perform 60 times a timeout).

Final  Long ioStartTime = to System.nanoTime (); 
needsToSelectAgain = to false ;
 // performs different processing depending on whether optimization methods SelectionKey (IO is handled event) 
IF (the selectedKeys =! null ) { 
    processSelectedKeysOptimized (selectedKeys.flip ()); 
} the else { 
    processSelectedKeysPlain (selector.selectedKeys ()); 
} 
Final  Long IOTime to System.nanoTime = () - ioStartTime; 

Final  int ioRatio = the this .ioRatio;
 // get a non IO IO execution time according to the execution time, execution then taking Task 
runAllTasks ( ioTime * (100 - ioRatio) / ioRatio);

protected  Boolean runAllTasks ( Long timeoutNanos) {
     // put in the scheduled tasks from the task queue in tskQueue (where holding a task is overdue) 
    fetchFromDelayedQueue ();
     // get the taskQueue the task 
    the Runnable task = pollTask ();
     IF (Task == null ) {
         return  to false ; 
    } 

    Final  Long DEADLINE ScheduledFutureTask.nanoTime = () + timeoutNanos;
     Long runTasks = 0 ;
     Long lastExecutionTime;
     for (;;) {
         the try { 
            task.run ();
        } The catch(The Throwable T) { 
            logger.warn ( "A Task raised AN Exception." , T); 
        } 

        runTasks ++ ; 

        // To improve the performance, each performed 60 times, a timeout is determined, if the execution time has exceeded a given system time, exit 
        IF ((runTasks & 0x3F) == 0 ) { 
            lastExecutionTime = ScheduledFutureTask.nanoTime ();
             IF (lastExecutionTime> = DEADLINE) {
                 BREAK ; 
            } 
        } 
        
        // loop processing task, the task is exited if no 
        task = pollTask ();
         IF (Task == null ) {
            lastExecutionTime = ScheduledFutureTask.nanoTime();
            break;
        }
    }

    this.lastExecutionTime = lastExecutionTime;
    return true;
}

　　Finally, to determine whether the system is at a standstill, and if so, call closeAll method to release resources, so NioEventLoop exit the loop, close the thread.

    if (isShuttingDown()) {
        closeAll();
        if (confirmShutdown()) {
            break;
        }
    }

　　closeAll () method to traverse to get all Channel, calling it Unsafe.close () method to close all the links, the release of resources (specifically close () method can see in front of Unsafe source code analysis, close () method end up calling the close or javaChannel ()method).

private void closeAll() {
    selectAgain();
    Set<SelectionKey> keys = selector.keys();
    Collection<AbstractNioChannel> channels = new ArrayList<AbstractNioChannel>(keys.size());
    for (SelectionKey k: keys) {
        Object a = k.attachment();
        if (a instanceof AbstractNioChannel) {
            channels.add((AbstractNioChannel) a);
        } else {
            k.cancel();
            @SuppressWarnings("unchecked")
            NioTask<SelectableChannel> task = (NioTask<SelectableChannel>) a;
            invokeChannelUnregistered(task, k, null);
        }
    }

    for (AbstractNioChannel ch: channels) {
        ch.unsafe().close(ch.unsafe().voidPromise());
    }
}

Three, NioEventLoopGroup

1. Constructor (NioEventLoop creation)

　　Let's look at EventLoopGroup construction method, where by the constructor, create a NioEventLoop specified number of threads.

    public NioEventLoopGroup(int nThreads) {
        this(nThreads, (Executor) null);
    }
    
    public NioEventLoopGroup(int nThreads, Executor executor) {
        this(nThreads, executor, SelectorProvider.provider());
    }
    
    public NioEventLoopGroup(
            int nThreads, Executor executor, final SelectorProvider selectorProvider) {
        super(nThreads, executor, selectorProvider);
    }
    
    // DEFAULT_EVENT_LOOP_THREADS = CPU个数*2
    protected MultithreadEventLoopGroup(int nThreads, Executor executor, Object... args) {
        super(nThreads == 0 ? DEFAULT_EVENT_LOOP_THREADS : nThreads, executor, args);
    }
    
    protected MultithreadEventExecutorGroup(int nThreads, Executor executor, Object... args) {
    if (nThreads <= 0) {
        throw new IllegalArgumentException(String.format("nThreads: %d (expected: > 0)", nThreads));
    }

    if (executor == null) {
        executor = new ThreadPerTaskExecutor(newDefaultThreadFactory());
    }
    
    // EventLoop数组
    children = new EventExecutor[nThreads];
    for (int i = 0; i < nThreads; i ++) {
        boolean success = false;
        try {
            children[i] = newChild(executor, args);
            success = true;
        } catch (Exception e) {
            // TODO: Think about if this is a good exception type
            throw new IllegalStateException("failed to create a child event loop", e);
        } finally {
            if (!success) {
                for (int j = 0; j < i; j ++) {
                    children[j].shutdownGracefully();
                }

                for (int j = 0; j < i; j ++) {
                    EventExecutor e = children[j];
                    try {
                        while (!e.isTerminated()) {
                            e.awaitTermination(Integer.MAX_VALUE, TimeUnit.SECONDS);
                        }
                    } catch (InterruptedException interrupted) {
                        Thread.currentThread().interrupt();
                        break;
                    }
                }
            }
        }
    }
    
    protected EventLoop newChild(Executor executor, Object... args) throws Exception {
        return new NioEventLoop(this, executor, (SelectorProvider) args[0]);
    }

2. NioEventLoop distribution

　　When a new Channel connection, NioEventLoopGroup need to come up with a NioEventLoop let Channel binding, IO operation after the Channel are operating on this NioEventLoop.

    public EventExecutor next() {
        return children[Math.abs(childIndex.getAndIncrement() % children.length)];
    }