异构数据处理(1).Canal入门示例

Canal

最近有使用到Alibaba的mysql数据库binlog的增量订阅&消费组件“Canal”,抱着试试看的心情打开简(jian)约(lou)的官网,下了源代码,并安装调试。1.0.25和1.0.26两个Canal版本反复调试参数发现跑不起来！按照之前遇(bei)到(keng)的经验,直接丢个issue发到Canal官网,然后该干嘛干嘛去了。
终于收到回复给了一个下载地址,还是canal.deployer-1.0.26-SNAPSHOT.tar.gz，md5sum确认是新的tar文件，下载重新配置启动了下，成功启动，demo代码也调试成功！

数据库配置

[mysqld]
log_bin=mysql-bin
binlog_format='ROW'
server_id=1

datadir=/var/lib/mysql
socket=/var/lib/mysql/mysql.sock
symbolic-links=0
sql_mode=NO_ENGINE_SUBSTITUTION,STRICT_TRANS_TABLES

[mysqld_safe]
log-error=/var/log/mysqld.log
pid-file=/var/run/mysqld/mysqld.pid

mysql canal用户的权限配置(可以看到权限没有问题)

mysql> select host,user,password from mysql.user  where user like '%canal%';
+-----------+-------+-------------------------------------------+
| host      | user  | password                                  |
+-----------+-------+-------------------------------------------+
| %         | canal | *E3619321C1A937C46A0D8BD1DAC39F93B27D4458 |
| 127.0.0.1 | canal | *E3619321C1A937C46A0D8BD1DAC39F93B27D4458 |
| localhost | canal | *E3619321C1A937C46A0D8BD1DAC39F93B27D4458 |
+-----------+-------+-------------------------------------------+
3 rows in set (0.05 sec)

canal 配置

## mysql serverId
canal.instance.mysql.slaveId=0
# position info
canal.instance.master.address=127.0.0.1:3306
canal.instance.master.journal.name=
canal.instance.master.position=
canal.instance.master.timestamp=


# table meta tsdb info
canal.instance.tsdb.enable=true
canal.instance.tsdb.dir=${canal.file.data.dir:../conf}/${canal.instance.destination:}
canal.instance.tsdb.url=jdbc:h2:${canal.instance.tsdb.dir}/h2;CACHE_SIZE=1000;MODE=MYSQL;
#canal.instance.tsdb.url=jdbc:mysql://127.0.0.1:3306/canal_tsdb
canal.instance.tsdb.dbUsername=canal
canal.instance.tsdb.dbPassword=canal


#canal.instance.standby.address =
#canal.instance.standby.journal.name =
#canal.instance.standby.position = 
#canal.instance.standby.timestamp = 
# username/password
canal.instance.dbUsername=canal
canal.instance.dbPassword=canal
canal.instance.defaultDatabaseName=canal_test
canal.instance.connectionCharset=UTF-8
# table regex
canal.instance.filter.regex=.*\\..*
# table black regex
canal.instance.filter.black.regex=

---

[1]: https://github.com/alibaba/canal/wiki/ClientExample clientExample
[2]:https://github.com/alibaba/canal CanleHome
[3]:https://github.com/alibaba/canal/issues/542 issue
[4]:https://github.com/WilliamGai/s-samples/tree/master/sample-canal Spring Boot CanalSample