Past recommendation

Actuator example

The following are xxl-jobexample codes for integrating executors in different ways:

The most commonly used springbootmethod is of course the integration method. Let's use this example to study xxl-jobthe startup process of the client executor.

start process

The client executor startup process entry is in XxlJobSpringExecutorthe class, by implementing the spring extension SmartInitializingSingleton, when the IOC singleton bean is loaded, the method is called afterSingletonsInstantiated():

@Override
public void afterSingletonsInstantiated() {

    // init JobHandler Repository (for method)
    initJobHandlerMethodRepository(applicationContext);

    // refresh GlueFactory
    GlueFactory.refreshInstance(1);

    // super start
    try {
        super.start();
    } catch (Exception e) {
        throw new RuntimeException(e);
    }
}

Bean type task analysis

First, let's analyze initJobHandlerMethodRepository(applicationContext)the method, which

private void initJobHandlerMethodRepository(ApplicationContext applicationContext) {

    // init job handler from method
    String[] beanDefinitionNames = applicationContext.getBeanNamesForType(Object.class, false, true);
    for (String beanDefinitionName : beanDefinitionNames) {
        Object bean = applicationContext.getBean(beanDefinitionName);

        Map<Method, XxlJob> annotatedMethods = null;   // referred to ：org.springframework.context.event.EventListenerMethodProcessor.processBean
        try {
            annotatedMethods = MethodIntrospector.selectMethods(bean.getClass(),
                    new MethodIntrospector.MetadataLookup<XxlJob>() {
                        @Override
                        public XxlJob inspect(Method method) {
                            return AnnotatedElementUtils.findMergedAnnotation(method, XxlJob.class);
                        }
                    });
        } catch (Throwable ex) {
            logger.error("xxl-job method-jobhandler resolve error for bean[" + beanDefinitionName + "].", ex);
        }
        if (annotatedMethods==null || annotatedMethods.isEmpty()) {
           continue;
        }

        ......

    }

}

The above logic is to traverse IoCthe container Bean, obtain and analyze @XxlJobthe method with annotations, and finally return Map<Method, XxlJob>the result type, keywhich is @XxlJobannotated Method, which is the annotation information valueparsed on the method .@XxlJob

After parsing and obtaining the information with @XxlJobannotations Method, let's see how to proceed further:

// 遍历
for (Map.Entry<Method, XxlJob> methodXxlJobEntry : annotatedMethods.entrySet()) {
    Method method = methodXxlJobEntry.getKey();
    XxlJob xxlJob = methodXxlJobEntry.getValue();
    if (xxlJob == null) {
     continue;
    }

    // 获取@XxlJob注解value配置
    String name = xxlJob.value();
    if (name.trim().length() == 0) {
     throw new RuntimeException("xxl-job method-jobhandler name invalid, for[" + bean.getClass() + "#" + method.getName() + "] .");
    }
    
    // 判断是否同名的已经加载，有则抛异常
    if (loadJobHandler(name) != null) {
     throw new RuntimeException("xxl-job jobhandler[" + name + "] naming conflicts.");
    }

    // @XxlJob注解方法参数校验：必须只有一个参数，且参数类型是String，否则抛出异常
    if (!(method.getParameterTypes().length == 1 && method.getParameterTypes()[0].isAssignableFrom(String.class))) {
     throw new RuntimeException("xxl-job method-jobhandler param-classtype invalid, for[" + bean.getClass() + "#" + method.getName() + "] , " + "The correct method format like \" public ReturnT<String> execute(String param) \" .");
    }
    // @XxlJob注解方法返回值校验，必须返回ReturnT类型
    if (!method.getReturnType().isAssignableFrom(ReturnT.class)) {
     throw new RuntimeException("xxl-job method-jobhandler return-classtype invalid, for[" + bean.getClass() + "#" + method.getName() + "] , " + "The correct method format like \" public ReturnT<String> execute(String param) \" .");
    }
    method.setAccessible(true);

    // init and destory
    Method initMethod = null;
    Method destroyMethod = null;

    // 解析@XxlJob注解init配置
    if (xxlJob.init().trim().length() > 0) {
        try {
            initMethod = bean.getClass().getDeclaredMethod(xxlJob.init());
            initMethod.setAccessible(true);
        } catch (NoSuchMethodException e) {
            throw new RuntimeException("xxl-job method-jobhandler initMethod invalid, for[" + bean.getClass() + "#" + method.getName() + "] .");
        }
    }
    
    // 解析@XxlJob注解destroy配置
    if (xxlJob.destroy().trim().length() > 0) {
        try {
            destroyMethod = bean.getClass().getDeclaredMethod(xxlJob.destroy());
            destroyMethod.setAccessible(true);
        } catch (NoSuchMethodException e) {
            throw new RuntimeException("xxl-job method-jobhandler destroyMethod invalid, for[" + bean.getClass() + "#" + method.getName() + "] .");
        }
    }

    // 将@XxlJob注解的Method、initMethod和destroyMethod封装成MethodJobHandler，并放入到Map中完成注册，[email protected]
    registJobHandler(name, new MethodJobHandler(bean, method, initMethod, destroyMethod));
}

GlueFactory initialization

GlueFactoryIt mainly deals with GLUE(Java)type jobs, GLUE(Java)compiles type job source code, creates instances and calls, and can support spring dependency injection, such as support in source code @Autowired, , @Resource, @Qualifieretc.

Executor startup process

super.start()This sentence will actually enter the executor startup process XxlJobExecutor#start:

public void start() throws Exception {

        // 初始化日志路径
        XxlJobFileAppender.initLogPath(logPath);

        // 初始化AdminBizClient，用于和admin远程交互
        initAdminBizList(adminAddresses, accessToken);

        // 初始化日志清理线程，用于清理日志目录下过期日志文件
        JobLogFileCleanThread.getInstance().start(logRetentionDays);

        // 初始化回调线程，triggerCallbackThread
        TriggerCallbackThread.getInstance().start();

        // init executor-server
        initEmbedServer(address, ip, port, appname, accessToken);
}

The main thing here is the latter two statements, TriggerCallbackThread.getInstance().start();which are mainly used to call back after the job execution is completed and pass the result to the admin module. For details, see the analysis of the job execution process of the client executor in the next section.initEmbedServer(address, ip, port, appname, accessToken);

private void initEmbedServer(String address, String ip, int port, String appname, String accessToken) throws Exception {

    // fill ip port
    port = port>0?port: NetUtil.findAvailablePort(9999);
    ip = (ip!=null&&ip.trim().length()>0)?ip: IpUtil.getIp();

    // generate address
    if (address==null || address.trim().length()==0) {
        String ip_port_address = IpUtil.getIpPort(ip, port);
        address = "http://{ip_port}/".replace("{ip_port}", ip_port_address);
    }

    // start
    embedServer = new EmbedServer();
    embedServer.start(address, port, appname, accessToken);
}

The logic above is for processing ip:portand parsing. The key is the following two sentences, EmbedServer#startwhich create a thread Thread internally and start it:

public void start(final String address, final int port, final String appname, final String accessToken) {
        executorBiz = new ExecutorBizImpl();
        thread = new Thread(new Runnable() {

            @Override
            public void run() {
    ......

            }
        });
        thread.setDaemon(true); // daemon, service jvm, user thread leave >>> daemon leave >>> jvm leave
        thread.start();
    }

Let's take a look at what this thread does internally:

EventLoopGroup bossGroup = new NioEventLoopGroup();
EventLoopGroup workerGroup = new NioEventLoopGroup();
ThreadPoolExecutor bizThreadPool = new ThreadPoolExecutor(
                        0,
                        200,
                        60L,
                        TimeUnit.SECONDS,
                        new LinkedBlockingQueue<Runnable>(2000),
                        new ThreadFactory() {
                            @Override
                            public Thread newThread(Runnable r) {
                                return new Thread(r, "xxl-rpc, EmbedServer bizThreadPool-" + r.hashCode());
                            }
                        },
                        new RejectedExecutionHandler() {
                            @Override
                            public void rejectedExecution(Runnable r, ThreadPoolExecutor executor) {
                                throw new RuntimeException("xxl-job, EmbedServer bizThreadPool is EXHAUSTED!");
                            }
});


try {
 // start server
 ServerBootstrap bootstrap = new ServerBootstrap();
    bootstrap.group(bossGroup, workerGroup)
       .channel(NioServerSocketChannel.class)
                .childHandler(new ChannelInitializer<SocketChannel>() {
                 @Override
                    public void initChannel(SocketChannel channel) throws Exception {
                     channel.pipeline()
                               .addLast(new IdleStateHandler(0, 0, 30 * 3, TimeUnit.SECONDS))  // beat 3N, close if idle
                               .addLast(new HttpServerCodec())
                               .addLast(new HttpObjectAggregator(5 * 1024 * 1024))  // merge request & reponse to FULL
                               .addLast(new EmbedHttpServerHandler(executorBiz, accessToken, bizThreadPool));
                    }})
               .childOption(ChannelOption.SO_KEEPALIVE, true);

 // bind
 ChannelFuture future = bootstrap.bind(port).sync();

 logger.info(">>>>>>>>>>> xxl-job remoting server start success, nettype = {}, port = {}", EmbedServer.class, port);

 // start registry
 startRegistry(appname, address);

    // wait util stop
    future.channel().closeFuture().sync();

}

There are a lot of codes above, but the logic is actually very simple. It mainly does two things:

nettyOne is started using initialization http server, which is mainly used to receive admin模块and send instructions to the executor, such as issuing execution job instructions and kill job instructions, and the main processing logic is encapsulated in EmbedHttpServerHandler;
startRegistry(appname, address)Start the client executor to regularly admin模块register threads, the logic code is in ExecutorRegistryThread#startthe method, which is relatively simple;

xxl-jobThe client executor registration process is roughly as follows:

1. The client uses adminBiz.registry(registryParam)the timing cycle to admin模块send registration information to the client;

2. admin模块After receiving the client registration information, insert|update the field time value xxl_job_registryof the table update_time;

3. admin模块Start JobRegistryMonitorHelperthe thread to scan xxl_job_registrythe table regularly, remove the timeout, and splice the online instance collection together and update it to the executor address_listfield information whose executor address is automatically registered.

Summarize

xxl-jobThe startup process of the client executor is relatively simple, and there are two core points:

The application starts nettya http servercontainer and IP:PORTbrings the application registration information to adminthe module, so adminthat the executor can issue commands such as running jobs and killing jobs;
The executor adminregisters once at regular intervals (30 seconds by default). When adminthe executor registration information is not received for more than 90 seconds, the executor is deemed to have timed out and will be removed offline;

Long press the QR code to identify attention

02 xxl-job executor startup process