Configuration process before preparation --otter
otter配置比较繁琐,并不会特别复杂,我这里先大概说下流程和几个关键名词解释
**otter名词解释:**
数据源:读取的源实例信息,和写入的目标实例信息
数据表:配置映射用的,用于配置,源实例,什么库,什么表,同步到目标什么库,什么表
canal:otter是做增量同步的,增量同步基于mysql的binlog日志,并且是row格式。这里需要配置你读取binlog的信息,和数据源里面的源实例信息可以说是同一个。
通道配置: otter采用一个实例一个通道方式。一个实例可以多个配置多个库
pipeline:主要核心功能如下
(1)选择你的canal配置,读取哪个实例的binlog。
(2)选择整个同步是在哪个节点上进行,例如我们部署了三个node节点,可以由node1进行读取的操作,可以由node2进行目标实例写入操作。也可以同时放到一个节点上。
(3)binlog位置,默认不写就读最新位置的。
(4)高级配置里面有是否跳过DDL,传输模式,负载均衡算法等,一般保持默认即可。
**流程:**
(1)新增数据源,一般最少配置2个,一个读取的源库,一个目标
(2)数据表,配置映射关系,从哪里同步到哪里。
(3)Canal,配置读取binlog的信息
上面步骤创建好后,我们就可以正式开始配置通道了
(4)创建通道
(5)创建pipeline
(6)创建表映射关系
(7)启动通道
Examples 1. Configuration Synchronization
I have here two new data sources, called a read (read source) example, called write instance (target written).
(1) follows a detailed configuration edit
list after (2) the configuration succeeds
source_read_testing_01: Examples of data to be read
target_write_testing_01: Examples of data to be written2. Table Configuration Synchronization
(1) Reading Library table configuration - Details edit
here is mainly the configuration database name you want to read the table name.
schema name: configuration repository name
Table name: because we are all synchronized libraries following table, so I filled out a regular expression;
Data Source: Select the source library (above step instance configuration has been configured, you can select)
(2) write library table configuration - Details edit
here is mainly the configuration database name you want to write, table name.
Schema name: configuration repository name
Table name: because we are all synchronized libraries following table, so I filled out a regular expression;
Data Source: Select (step above instance configuration has been configured, you can select) the target library
3.Canal Configuration
Canal configuration, mainly binlog log source instance read with
Canal Name: Custom
Zookeeper: The default will automatically select, after a previously configured
database type: Mysql
database address: binlog is what you want to get the address of the source database [ the same database instance]
account password: the need to obtain permission binlog account Oh.
Other content not fill
4.Channel channel configuration
Creating Canal
selection based on the current change log, select rows; the name can be customized;5.Pipeline Configuration
Step 1: Click on the name of the Channel, the next stepStep 2: Add Pipleline, click Add
to select the node select and load. A read, a write. Choose a node, reducing the transmission between the networks.
Canal Name: Select the canal just add configuration, the other remains unchanged6. Configure the mapping relationship
Means that you have to on this passage, what rules synchronization is configured between the table and the table
Step 1: Click Pipeline into the mapping between configurationStep 2: Add the mapping relationship
Step 3: Find data table
source data table: select readonly source database table
target table: select a target database table
after storage, as follows, Home return passage7. Start channel
8. Acceptance Testing
New data in the source library, verify that the target database table is normal