Canal (2): Canal deployment and configuration

1 mysql database configuration

1.1 Create a new database

1.2 Create a new test table

CREATE TABLE user_info(
    `id` VARCHAR(255),
    `name` VARCHAR(255),
    `sex` VARCHAR(255)
);

1.3 Modify the configuration file to enable Binlog

vim /etc/my.cnf

server-id=4
log-bin=mysql-bin
binlog_format=row
binlog-do-db=caneltestdb

1.4 Restart MySQL to make the configuration take effect

systemctl restart mysqld

Go to the /var/lib/mysql directory to view the initial file size 154

1.5 Test whether Binlog is enabled

insert data

INSERT INTO user_info VALUES('1001','zhangsan','male');

Go to the /var/lib/mysql directory again to check the size of the index file

1.6 Authorization

Execute in MySQL

mysql> set global validate_password_length=4;
mysql> set global validate_password_policy=0;
mysql> GRANT SELECT, REPLICATION SLAVE, REPLICATION CLIENT ON *.* TO 'canal'@'%' IDENTIFIED BY 'canal' ;

2 canel deployment and configuration

2.1 Download and decompress the Jar package

https://github.com/alibaba/canal/releases

The canal.deployer-1.1.6.tar.gz version is used here

Copy it to the /opt/sortware directory, and then extract it to the /opt/module/canal package (the directory is determined by yourself)

Note: canal is scattered after decompression, we need to specify canal when specifying the decompression directory

2.2 Modify the configuration of canal.properties

vim  conf/canal.properties

The modification is as follows:

 

Description: This file is the basic general configuration of canal, the default port number of canal is 11111, modify the output model of canal, default tcp, change to output to kafka

Multi-instance configuration If you create multiple instances, through the previous canal architecture, we can know that there can be multiple instances in a canal service, each example under conf/ is an instance, and each instance has an independent configuration file . By default, there is only one instance example. If multiple instances are required to process different MySQL data, copy multiple examples directly and rename them. The name is consistent with the name specified in the configuration file, and then modify the canal in canal.properties. destinations=instance1, instance2, instance3.

 

2.3 modify instance.properties

We only read one MySQL data here, so there is only one instance, and the configuration file of this instance is in the conf/example directory

vim conf/example/instance.properties

(1) Configure the MySQL server address

Notice:

        The value of canal.instance.mysql.slaveId cannot be the same as the server-id configured in mysql. 

(2) Configure the username and password to connect to MySQL. The default is the canal we authorized earlier

 

 2.4 start canal

sh bin/startup.sh

The startup is successful as follows