Canal download, deployment and getting started (details)

1 Overview

canal [kə'næl], translated as waterway/pipe/ditch.
It is an open source project under Alibaba, developed in pure Java.
Based on database incremental log analysis, the main purpose of providing incremental data subscription & consumption is based on MySQL database incremental log analysis, currently mainly supports mysql.

2 Advantages and disadvantages

advantage:与业务代码完全解耦，API完全解耦，异步处理，且可以做到准实时，用着很舒服；
shortcoming:
- After opening Binlog, there will be additional overhead on the database, and the performance will be affected;
- Canal is implemented by a third party, which requires learning costs (the learning is endless, and the skills are not overwhelming);

2 common scenarios

Canal is part of Ali Otter middleware, and Otter is Ali's synchronization framework between remote databases;
Used to update the cache and reduce the DB pressure of real-time data synchronization;
Capture business data and add a change table to make a zipper table;
Grab the newly added change data of the business table for making real-time statistics;

3 working principle

Mysql's Binlog records all DDL and DML (except data query statements) statements in the form of events, and also includes the time consumed by the statement execution. Mainly used for backup and data synchronization. Canal is mainly a technology based on Binlog.

在mysql中开启binlog日志记录，完成主从复制。
canal 会伪装(实现了mysql的binlog日志的传输协议)为 mysql的一个从机实例！向mysql的主机发送dump请求。
主机将binlog日志发送给canal,有canal解析，解析后，可以发送到任意地方。

Binlog has three modes: STATEMENT, ROW, MIXED
STATEMENT records the executed sql statement.
ROW records are real row data records.
MIXED records 1+2, and records in the mode of 1 first.

(The part about Binlog will not be elaborated too much. If you are interested, you can download it on Baidu, and enter the text below)

4 Quick Start

4.1 Preparation

4.1.1 Open mysql Binlog

For details, see another article I wrote: "Mysql database opens binlog"

4.1.2 Download Canal file package

Official download address: https://github.com/alibaba/canal/releases
（本次下载的 canal 是版本号为：1.1.6）
insert image description here
After decompression, you get the following folder

4.1.3 Modify configuration file

Go to the conf\example file and modify the instance.properties configuration

#position info，需要改成自己的数据库信息
canal.instance.master.address = 127.0.0.1:3306 
canal.instance.master.journal.name = 
canal.instance.master.position = 
canal.instance.master.timestamp = 
#canal.instance.standby.address = 
#canal.instance.standby.journal.name =
#canal.instance.standby.position = 
#canal.instance.standby.timestamp = 
# username/password，需要改成自己的数据库信息
canal.instance.dbUsername=root
canal.instance.dbPassword=root
canal.instance.connectionCharset = UTF-8
#table regex 数据库表的过滤
canal.instance.filter.regex=canal_test.*

4.1.4 Start Canal

./bin/startup.sh

insert image description here