由于要在组内使用canal，但公司的中间件团队并不能提供相应支持，因此只能自己读懂源码，防止生产出现问题，前期先在边缘模块进行使用，稳定后，可以进行推广。
一、读源码的策略
看书，看github的相关材料永远是能让你最快入门，但只有源码级的了解，在一个商业软件中， 才能真正放心进行使用。读源码最快的方式是本地，调试。
先了解整体的流程，在了解细分的流程。不要求一开始就了解每个细节，这也不现实。
读canal源码过程中，参考了https://blog.csdn.net/skycanf/article/details/80497675
二、导入工程并运行
IDE环境： IDEA
git clone 工程导入后，点开pom文件，右键变为Maven项目。
-
找到启动类：deploy工程的CanalLauncher 进行流程梳理
image.png 找到启动类后，先使工程跑起来
参考canal的使用手册，配置数据库等相关信息，和日志信息，先使他跑起来，完整运行一次
期间发现github由于默认clone了最新的alpha版本，跑不起来。回退到正式版1.1.2后程序正常运行无报错信息，至此
git tag
git checkout 版本号
三、通过启动的日志，初步了解
2019-01-08 14:45:23.388 [main] INFO com.alibaba.otter.canal.prometheus.PrometheusService - Start prometheus HTTPServer on port 11112.
2019-01-08 14:45:24.040 [main] INFO o.s.context.support.ClassPathXmlApplicationContext - Refreshing org.springframework.context.support.ClassPathXmlApplicationContext@7e0e6aa2: startup date [Tue Jan 08 14:45:24 CST 2019]; root of context hierarchy
2019-01-08 14:45:24.136 [main] INFO o.s.beans.factory.xml.XmlBeanDefinitionReader - Loading XML bean definitions from class path resource [spring/file-instance.xml]
2019-01-08 14:45:24.330 [main] INFO o.s.beans.factory.xml.XmlBeanDefinitionReader - Loading XML bean definitions from class path resource [spring/base-instance.xml]
2019-01-08 14:45:24.759 [main] INFO o.s.beans.factory.support.DefaultListableBeanFactory - Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@2667f029: defining beans [com.alibaba.otter.canal.instance.spring.support.PropertyPlaceholderConfigurer#0,socketAddressEditor,org.springframework.beans.factory.config.CustomEditorConfigurer#0,baseEventParser,instance,alarmHandler,metaManager,eventStore,eventSink,eventParser,mqConfig]; root of factory hierarchy
2019-01-08 14:45:25.068 [main] WARN o.s.beans.GenericTypeAwarePropertyDescriptor - Invalid JavaBean property 'connectionCharset' being accessed! Ambiguous write methods found next to actually used [public void com.alibaba.otter.canal.parse.inbound.mysql.AbstractMysqlEventParser.setConnectionCharset(java.nio.charset.Charset)]: [public void com.alibaba.otter.canal.parse.inbound.mysql.AbstractMysqlEventParser.setConnectionCharset(java.lang.String)]
2019-01-08 14:45:25.080 [main] INFO o.s.context.support.ClassPathXmlApplicationContext - Refreshing org.springframework.context.support.ClassPathXmlApplicationContext@4e50c791: startup date [Tue Jan 08 14:45:25 CST 2019]; root of context hierarchy
2019-01-08 14:45:25.082 [main] INFO o.s.beans.factory.xml.XmlBeanDefinitionReader - Loading XML bean definitions from class path resource [spring/tsdb/h2-tsdb.xml]
2019-01-08 14:45:25.159 [main] INFO o.s.beans.factory.support.DefaultListableBeanFactory - Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory@117e949d: defining beans [com.alibaba.otter.canal.instance.spring.support.PropertyPlaceholderConfigurer#0,tableMetaTSDB,dataSource,sqlMapClient,metaHistoryDAO,metaSnapshotDAO]; root of factory hierarchy
2019-01-08 14:45:25.463 [main] ERROR com.alibaba.druid.pool.DruidDataSource - testWhileIdle is true, validationQuery not set
2019-01-08 14:45:25.478 [main] INFO com.alibaba.druid.pool.DruidDataSource - {dataSource-1} inited
2019-01-08 14:45:25.798 [main] INFO c.a.o.c.p.inbound.mysql.tsdb.DefaultTableMetaTSDBFactory - example init TableMetaTSDB with classpath:spring/tsdb/h2-tsdb.xml
2019-01-08 14:45:25.808 [main] INFO com.alibaba.otter.canal.prometheus.CanalInstanceExports - Successfully register metrics for instance example.
2019-01-08 14:45:25.808 [main] INFO com.alibaba.otter.canal.prometheus.PrometheusService - Register metrics for destination example.
* 此处可以看到instance启动成功
2019-01-08 14:45:25.827 [main] INFO c.a.otter.canal.server.embedded.CanalServerWithEmbedded - start CanalInstances[example] successfully
* 心跳
2019-01-08 14:45:25.828 [destination = example , address = /45.76.154.119:8080 , EventParser] INFO c.a.o.c.p.inbound.mysql.rds.RdsBinlogEventParserProxy - start heart beat....
2019-01-08 14:45:26.112 [destination = example , address = /45.76.154.119:8080 , EventParser] INFO c.alibaba.otter.canal.parse.driver.mysql.MysqlConnector - connect MysqlConnection to /45.76.154.119:8080...
* 权限验证
2019-01-08 14:45:26.401 [destination = example , address = /45.76.154.119:8080 , EventParser] INFO c.alibaba.otter.canal.parse.driver.mysql.MysqlConnector - handshake initialization packet received, prepare the client authentication packet to send
2019-01-08 14:45:26.409 [destination = example , address = /45.76.154.119:8080 , EventParser] INFO c.alibaba.otter.canal.parse.driver.mysql.MysqlConnector - client authentication packet is sent out.
2019-01-08 14:45:27.945 [destination = example , address = /45.76.154.119:8080 , EventParser] INFO c.alibaba.otter.canal.parse.driver.mysql.MysqlConnector - connect MysqlConnection to /45.76.154.119:8080...
2019-01-08 14:45:28.225 [destination = example , address = /45.76.154.119:8080 , EventParser] INFO c.alibaba.otter.canal.parse.driver.mysql.MysqlConnector - handshake initialization packet received, prepare the client authentication packet to send
2019-01-08 14:45:28.225 [destination = example , address = /45.76.154.119:8080 , EventParser] INFO c.alibaba.otter.canal.parse.driver.mysql.MysqlConnector - client authentication packet is sent out.
* 通过show master status 去找binlog目前的文件和position
2019-01-08 14:45:28.805 [destination = example , address = /45.76.154.119:8080 , EventParser] WARN c.a.o.c.p.inbound.mysql.rds.RdsBinlogEventParserProxy - ---> begin to find start position, it will be long time for reset or first position
2019-01-08 14:45:28.806 [destination = example , address = /45.76.154.119:8080 , EventParser] WARN c.a.o.c.p.inbound.mysql.rds.RdsBinlogEventParserProxy - prepare to find start position just show master status
2019-01-08 14:45:29.112 [destination = example , address = /45.76.154.119:8080 , EventParser] INFO c.alibaba.otter.canal.parse.driver.mysql.MysqlConnector - disConnect MysqlConnection to /45.76.154.119:8080...
2019-01-08 14:45:29.344 [destination = example , address = /45.76.154.119:8080 , EventParser] INFO c.alibaba.otter.canal.parse.driver.mysql.MysqlConnector - connect MysqlConnection to /45.76.154.119:8080...
2019-01-08 14:45:29.572 [destination = example , address = /45.76.154.119:8080 , EventParser] INFO c.alibaba.otter.canal.parse.driver.mysql.MysqlConnector - handshake initialization packet received, prepare the client authentication packet to send
2019-01-08 14:45:29.573 [destination = example , address = /45.76.154.119:8080 , EventParser] INFO c.alibaba.otter.canal.parse.driver.mysql.MysqlConnector - client authentication packet is sent out.
2019-01-08 14:45:31.822 [destination = example , address = /45.76.154.119:8080 , EventParser] INFO c.a.otter.canal.parse.inbound.mysql.MysqlConnection - COM_BINLOG_DUMP with position:BinlogDumpCommandPacket[binlogPosition=4,slaveServerId=1779723102,binlogFileName=mysql-bin.000003,command=18]
2019-01-08 14:45:32.045 [destination = example , address = /45.76.154.119:8080 , EventParser] INFO com.taobao.tddl.dbsync.binlog.LogEvent - common_header_len= 19, number_of_event_types= 38
2019-01-08 14:46:11.826 [destination = example , address = /45.76.154.119:8080 , EventParser] WARN c.a.o.c.p.inbound.mysql.rds.RdsBinlogEventParserProxy - ---> find start position successfully, EntryPosition[included=false,journalName=mysql-bin.000003,position=793,serverId=1,gtid=<null>,timestamp=1546846649000] cost : 43020ms , the next step is binlog dump
2019-01-08 14:46:11.826 [destination = example , address = /45.76.154.119:8080 , EventParser] INFO c.alibaba.otter.canal.parse.driver.mysql.MysqlConnector - disConnect MysqlConnection to /45.76.154.119:8080...
2019-01-08 14:46:12.120 [destination = example , address = /45.76.154.119:8080 , EventParser] INFO c.alibaba.otter.canal.parse.driver.mysql.MysqlConnector - connect MysqlConnection to /45.76.154.119:8080...
2019-01-08 14:46:12.416 [destination = example , address = /45.76.154.119:8080 , EventParser] INFO c.alibaba.otter.canal.parse.driver.mysql.MysqlConnector - handshake initialization packet received, prepare the client authentication packet to send
2019-01-08 14:46:12.416 [destination = example , address = /45.76.154.119:8080 , EventParser] INFO c.alibaba.otter.canal.parse.driver.mysql.MysqlConnector - client authentication packet is sent out.
2019-01-08 14:46:12.969 [destination = example , address = /45.76.154.119:8080 , EventParser] INFO c.alibaba.otter.canal.parse.driver.mysql.MysqlConnector - disConnect MysqlConnection to /45.76.154.119:8080...
2019-01-08 14:46:13.178 [destination = example , address = /45.76.154.119:8080 , EventParser] INFO c.alibaba.otter.canal.parse.driver.mysql.MysqlConnector - connect MysqlConnection to /45.76.154.119:8080...
2019-01-08 14:46:13.386 [destination = example , address = /45.76.154.119:8080 , EventParser] INFO c.alibaba.otter.canal.parse.driver.mysql.MysqlConnector - handshake initialization packet received, prepare the client authentication packet to send
2019-01-08 14:46:13.387 [destination = example , address = /45.76.154.119:8080 , EventParser] INFO c.alibaba.otter.canal.parse.driver.mysql.MysqlConnector - client authentication packet is sent out.
* 最终的数据库订阅信息
2019-01-08 14:46:15.587 [destination = example , address = /45.76.154.119:8080 , EventParser] INFO c.a.otter.canal.parse.inbound.mysql.MysqlConnection - Register slave RegisterSlaveCommandPacket[reportHost=172.20.107.94,reportPort=5700,reportUser=canal,reportPasswd=OStem@00,serverId=1779723102,command=21]
2019-01-08 14:46:15.819 [destination = example , address = /45.76.154.119:8080 , EventParser] INFO c.a.otter.canal.parse.inbound.mysql.MysqlConnection - COM_BINLOG_DUMP with position:BinlogDumpCommandPacket[binlogPosition=793,slaveServerId=1779723102,binlogFileName=mysql-bin.000003,command=18]
2019-01-08 14:46:16.466 [MultiStageCoprocessor-other-example-0] INFO com.taobao.tddl.dbsync.binlog.LogEvent - common_header_len= 19, number_of_event_types= 38
四、结合源码，了解总流程
public static void main(String[] args) throws Throwable {
try {
logger.info("## set default uncaught exception handler");
设置全局为未捕获异常，当发生未捕获异常时，会调用uncaughtException方法
setGlobalUncaughtExceptionHandler();
加载本地配置文件
logger.info("## load canal configurations");
String conf = System.getProperty("canal.conf", "classpath:canal.properties");
Properties properties = new Properties();
if (conf.startsWith(CLASSPATH_URL_PREFIX)) {
conf = StringUtils.substringAfter(conf, CLASSPATH_URL_PREFIX);
默认情况下，加载进了canal.conf的配置文件
properties.load(CanalLauncher.class.getClassLoader().getResourceAsStream(conf));
} else {
properties.load(new FileInputStream(conf));
}
CanalMQProducer canalMQProducer = null;
String serverMode = CanalController.getProperty(properties, CanalConstants.CANAL_SERVER_MODE);
canal对于kafka和rocketmq的支持，不用可忽略
if (serverMode.equalsIgnoreCase("kafka")) {
canalMQProducer = new CanalKafkaProducer();
} else if (serverMode.equalsIgnoreCase("rocketmq")) {
canalMQProducer = new CanalRocketMQProducer();
}
if (canalMQProducer != null) {
// disable netty
System.setProperty(CanalConstants.CANAL_WITHOUT_NETTY, "true");
}
创建核心的Controller
logger.info("## start the canal server.");
final CanalController controller = new CanalController(properties);
controller.start();
logger.info("## the canal server is running now ......");
hook函数，为了在JVM停止时优雅停止，但在linux中kill -9 等操作时，无法触发
Runtime.getRuntime().addShutdownHook(new Thread() {
public void run() {
try {
logger.info("## stop the canal server");
controller.stop();
} catch (Throwable e) {
logger.warn("##something goes wrong when stopping canal Server:", e);
} finally {
logger.info("## canal server is down.");
}
}
});
if (canalMQProducer != null) {
CanalMQStarter canalMQStarter = new CanalMQStarter(canalMQProducer);
MQProperties mqProperties = buildMQPosition(properties);
canalMQStarter.start(mqProperties);
controller.setCanalMQStarter(canalMQStarter);
}
} catch (Throwable e) {
logger.error("## Something goes wrong when starting up the canal Server:", e);
System.exit(0);
}
}
下一篇看这块核心代码的逻辑