canal 源码解析（1）-启动篇（1）

1 .源码地址：https://github.com/alibaba/canal 看下是否是这个canal 若本地安装了git 右键bash ，然后窗口输入以下命令 git clone https://github.com/alibaba/canal.git

2. 启动方式，在GitHub上已经有了windos 和服务器的方式，我就不具体介绍了，现在介绍一种本地debug版本的

3. 本地目录结构为

1）修改下参数，也修改下日志级别和输出路径，后期容易维护

2）启动类：在CanalLauncher 的main方法里

public class CanalLauncher {

    private static final String CLASSPATH_URL_PREFIX = "classpath:";
    private static final Logger logger               = LoggerFactory.getLogger(CanalLauncher.class);

    public static void main(String[] args) throws Throwable {
        try {
            logger.info("## set default uncaught exception handler");
            setGlobalUncaughtExceptionHandler();

            logger.info("## load canal configurations");
            String conf = System.getProperty("canal.conf", "classpath:canal.properties");
            Properties properties = new Properties();
            //文件名检验。
            if (conf.startsWith(CLASSPATH_URL_PREFIX)) {
                conf = StringUtils.substringAfter(conf, CLASSPATH_URL_PREFIX);
                properties.load(CanalLauncher.class.getClassLoader().getResourceAsStream(conf));
            } else {
                /**
                 * 把配置文件canal.properties的配置加载到properties
                 */
                properties.load(new FileInputStream(conf));
            }

            logger.info("## start the canal server.");
            //new对象，加载其构造方法，
            final CanalController controller = new CanalController(properties);
            controller.start();
            logger.info("## the canal server is running now ......");
            Runtime.getRuntime().addShutdownHook(new Thread() {

                public void run() {
                    try {
                        logger.info("## stop the canal server");
                        controller.stop();
                    } catch (Throwable e) {
                        logger.warn("##something goes wrong when stopping canal Server:", e);
                    } finally {
                        logger.info("## canal server is down.");
                    }
                }

            });
        } catch (Throwable e) {
            logger.error("## Something goes wrong when starting up the canal Server:", e);
            System.exit(0);
        }
    }

    private static void setGlobalUncaughtExceptionHandler() {
        Thread.setDefaultUncaughtExceptionHandler(new Thread.UncaughtExceptionHandler() {
            @Override
            public void uncaughtException(Thread t, Throwable e) {
                logger.error("UnCaughtException", e);
            }
        });
    }

以上代码解析：就是从配置文件里加载属性最后加载到properties里。然后new对象，调用controller里的start方法

运行结果如下：

扫描二维码关注公众号，回复： 1497595 查看本文章

3）canalcontroller 类 new对象，调用其构造方法若有集成，优先加载父类构造方法，（若还有静态块，类则优先于构造方法加载）

public CanalController(final Properties properties) {
    /**
     * map做为缓存，在第一次get的时候，存放get的values。
     */
    managerClients = MigrateMap.makeComputingMap(new Function<String, CanalConfigClient>() {

        public CanalConfigClient apply(String managerAddress) {
            return getManagerClient(managerAddress);
        }
    });

    /**
     * 初始化全局参数设置
     */
    globalInstanceConfig = initGlobalConfig(properties);
    instanceConfigs = new MapMaker().makeMap();

    /**初始化instance config
     *
     */
    initInstanceConfig(properties);

    /**
     * init socketChannel
     */
    String socketChannel = getProperty(properties, CanalConstants.CANAL_SOCKETCHANNEL);
    if (StringUtils.isNotEmpty(socketChannel)) {
        System.setProperty(CanalConstants.CANAL_SOCKETCHANNEL, socketChannel);
    }
    /**
     * 准备canal server
     */
    cid = Long.valueOf(getProperty(properties, CanalConstants.CANAL_ID));
    ip = getProperty(properties, CanalConstants.CANAL_IP);
    port = Integer.valueOf(getProperty(properties, CanalConstants.CANAL_PORT));
    embededCanalServer = CanalServerWithEmbedded.instance();

    /**
     * 设置自定义的instanceGenerator
     */
    embededCanalServer.setCanalInstanceGenerator(instanceGenerator);
    canalServer = CanalServerWithNetty.instance();
    canalServer.setIp(ip);
    canalServer.setPort(port);


    // 处理下ip为空，默认使用hostIp暴露到zk中
    if (StringUtils.isEmpty(ip)) {
        ip = AddressUtils.getHostIp();
    }
    final String zkServers = getProperty(properties, CanalConstants.CANAL_ZKSERVERS);
    if (StringUtils.isNotEmpty(zkServers)) {
        //Fixme zk的创建 初始化。
    }

    final ServerRunningData serverData = new ServerRunningData(cid, ip + ":" + port);
    ServerRunningMonitors.setServerData(serverData);
    /**
     *  canal会为每一个destination创建一个CanalInstance，每个CanalInstance都会由一个ServerRunningMonitor来进行监控。
     *  而ServerRunningMonitor统一由ServerRunningMonitors进行管理。
     */
    ServerRunningMonitors.setRunningMonitors(MigrateMap.makeComputingMap(new Function<String, ServerRunningMonitor>() {

        public ServerRunningMonitor apply(final String destination) {
            ServerRunningMonitor runningMonitor = new ServerRunningMonitor(serverData);
            runningMonitor.setDestination(destination);
            runningMonitor.setListener(new ServerRunningListener() {
                /**
                 * 内部调用了embededCanalServer的start(destination)方法。
                 * 每个destination对应的CanalInstance是通过embededCanalServer的start方法启动的
                 * embededCanalServer负责调用instanceGenerator生成CanalInstance实例，并负责其启动。
                 */
                public void processActiveEnter() {
                    try {
                        MDC.put(CanalConstants.MDC_DESTINATION, String.valueOf(destination));
                        embededCanalServer.start(destination);
                    } finally {
                        MDC.remove(CanalConstants.MDC_DESTINATION);
                    }
                }
                /**
                 * 内部调用embededCanalServer的stop(destination)方法。与上start方法类似，
                 * 只不过是停止CanalInstance
                 */
                public void processActiveExit() {
                    try {
                        MDC.put(CanalConstants.MDC_DESTINATION, String.valueOf(destination));
                        embededCanalServer.stop(destination);
                    } finally {
                        MDC.remove(CanalConstants.MDC_DESTINATION);
                    }
                }
                /**
                 * 处理存在zk的情况下，在Canalinstance启动之前，在zk中创建节点。
                 路径为：/otter/canal/destinations/{0}/cluster/{1}，其0会被destination替换，1会被ip:port替换。
                 此方法会在processActiveEnter()之前被调用*
                 */
                public void processStart() {
                    try {
                        if (zkclientx != null) {
                            final String path = ZookeeperPathUtils.getDestinationClusterNode(destination, ip + ":"
                                    + port);
                            initCid(path);
                            zkclientx.subscribeStateChanges(new IZkStateListener() {

                                public void handleStateChanged(Watcher.Event.KeeperState state) throws Exception {

                                }
                                public void handleNewSession() throws Exception {
                                    initCid(path);
                                }
                                @Override
                                public void handleSessionEstablishmentError(Throwable error) throws Exception {
                                    logger.error("failed to connect to zookeeper", error);
                                }
                            });
                        }
                    } finally {
                        MDC.remove(CanalConstants.MDC_DESTINATION);
                    }
                }
                /**
                 * 处理存在zk的情况下，在Canalinstance停止前，释放zk节点，路径为/otter/canal/destinations/{0}/cluster/{1}，
                 其0会被destination替换，1会被ip:port替换。此方法会在processActiveExit()之前被调用
                 */
                public void processStop() {
                    try {
                        MDC.put(CanalConstants.MDC_DESTINATION, String.valueOf(destination));
                        if (zkclientx != null) {
                            final String path = ZookeeperPathUtils.getDestinationClusterNode(destination, ip + ":"
                                    + port);
                            releaseCid(path);
                        }
                    } finally {
                        MDC.remove(CanalConstants.MDC_DESTINATION);
                    }
                }

            });
            if (zkclientx != null) {
                runningMonitor.setZkClient(zkclientx);
            }
            // 触发创建一下cid节点
            runningMonitor.init();
            return runningMonitor;
        }
    }));


    /**
     * 初始化monitor机制
     */
    autoScan = BooleanUtils.toBoolean(getProperty(properties, CanalConstants.CANAL_AUTO_SCAN));
    if (autoScan) {
        defaultAction = new InstanceAction() {

            public void start(String destination) {
                InstanceConfig config = instanceConfigs.get(destination);
                if (config == null) {
                    // 重新读取一下instance config
                    config = parseInstanceConfig(properties, destination);
                    instanceConfigs.put(destination, config);
                }

                if (!embededCanalServer.isStart(destination)) {
                    // HA机制启动
                    ServerRunningMonitor runningMonitor = ServerRunningMonitors.getRunningMonitor(destination);
                    if (!config.getLazy() && !runningMonitor.isStart()) {
                        runningMonitor.start();
                    }
                }
            }

            public void stop(String destination) {
                // 此处的stop，代表强制退出，非HA机制，所以需要退出HA的monitor和配置信息
                InstanceConfig config = instanceConfigs.remove(destination);
                if (config != null) {
                    embededCanalServer.stop(destination);
                    ServerRunningMonitor runningMonitor = ServerRunningMonitors.getRunningMonitor(destination);
                    if (runningMonitor.isStart()) {
                        runningMonitor.stop();
                    }
                }
            }

            public void reload(String destination) {
                // 目前任何配置变化，直接重启，简单处理
                stop(destination);
                start(destination);
            }
        };

        instanceConfigMonitors = MigrateMap.makeComputingMap(new Function<InstanceMode, InstanceConfigMonitor>() {

            public InstanceConfigMonitor apply(InstanceMode mode) {
                int scanInterval = Integer.valueOf(getProperty(properties, CanalConstants.CANAL_AUTO_SCAN_INTERVAL));

                if (mode.isSpring()) {
                    SpringInstanceConfigMonitor monitor = new SpringInstanceConfigMonitor();
                    monitor.setScanIntervalInSecond(scanInterval);
                    monitor.setDefaultAction(defaultAction);
                    // 设置conf目录，默认是user.dir + conf目录组成
                    String rootDir = getProperty(properties, CanalConstants.CANAL_CONF_DIR);
                    if (StringUtils.isEmpty(rootDir)) {
                        rootDir = "../conf";
                    }

                    if (StringUtils.equals("otter-canal", System.getProperty("appName"))) {
                        monitor.setRootConf(rootDir);
                    } else {
                        // eclipse debug模式
                        monitor.setRootConf("src/main/resources/");
                    }
                    return monitor;
                } else if (mode.isManager()) {
                    return new ManagerInstanceConfigMonitor();
                } else {
                    throw new UnsupportedOperationException("unknow mode :" + mode + " for monitor");
                }
            }
        });}

}

解析以上的构造器类：

3.1）

/**
 * 初始化全局参数设置
 */
globalInstanceConfig = initGlobalConfig(properties);
instanceConfigs = new MapMaker().makeMap();

private InstanceConfig initGlobalConfig(Properties properties) {
    InstanceConfig globalConfig = new InstanceConfig();
    /**
     * 从配置文件里获取，canal.instance.global.mode的值，lazy spring
     */
    String modeStr = getProperty(properties, CanalConstants.getInstanceModeKey(CanalConstants.GLOBAL_NAME));
    if (StringUtils.isNotEmpty(modeStr)) {
        globalConfig.setMode(InstanceMode.valueOf(StringUtils.upperCase(modeStr)));
    }

    String lazyStr = getProperty(properties, CanalConstants.getInstancLazyKey(CanalConstants.GLOBAL_NAME));
    if (StringUtils.isNotEmpty(lazyStr)) {
        globalConfig.setLazy(Boolean.valueOf(lazyStr));
    }

    String managerAddress = getProperty(properties,
            CanalConstants.getInstanceManagerAddressKey(CanalConstants.GLOBAL_NAME));
    if (StringUtils.isNotEmpty(managerAddress)) {
        globalConfig.setManagerAddress(managerAddress);
    }

    String springXml = getProperty(properties, CanalConstants.getInstancSpringXmlKey(CanalConstants.GLOBAL_NAME));
    if (StringUtils.isNotEmpty(springXml)) {
        globalConfig.setSpringXml(springXml);

    }
    instanceGenerator = new CanalInstanceGenerator() {
        @Override
        public CanalInstance generate(String destination) {
            InstanceConfig config = instanceConfigs.get(destination);
            if (config == null) {
                throw new CanalServerException("can't find destination:{}");
            }
            if (config.getMode().isManager()) {

 InstanceConfig[globalConfig=<null>,mode=SPRING,lazy=false,managerAddress<null>,springXml=classpath:spring/file-instance.xml]

3.2）注意

Map<String, InstanceConfig> instanceConfigs = new MapMaker().makeMap(); 其中

instanceConfigs 对象为一个缓存map ，初始化时候为null，当第一次调用（就是获取其中值得时候），会调用其apply里的方法

第二次就，直接从map里获取values

/**初始化instance config
 *
 */
initInstanceConfig(properties);

3.3）

private void initInstanceConfig(Properties properties) {
    /**
     * 从配置文件里获取canal.destinations的值，值若多个，则遍历放入instanceConfigs的map里，key为配置项，
     * value为InstanceConfig
     */
    String destinationStr = getProperty(properties, CanalConstants.CANAL_DESTINATIONS);
    String[] destinations = StringUtils.split(destinationStr, CanalConstants.CANAL_DESTINATION_SPLIT);
    for (String destination : destinations) {
        InstanceConfig config = parseInstanceConfig(properties, destination);
        InstanceConfig oldConfig = instanceConfigs.put(destination, config);
        if (oldConfig != null) {
            logger.warn("destination:{} old config:{} has replace by new config:{}", new Object[]{destination,
                    oldConfig, config});
        }
    }

}

3.4）因为本地debug启动，所以没有zookeeper ，凡是zookeeper的判断都为null，目前数据为单实例消费单数据库，后期再进行模拟zookeeper场景

加载完构造器最终运行结果为：

最重要的是启动方法，下一篇进行解释

controller.start();

canal 源码解析（1）-启动篇（1）

猜你喜欢