Article directory
Concept overview
binlog
MySQL's binary log binlog can be said to be the most important log of MySQL. It records all DDL and DML statements (except the data query statement select) in the form of events, and also includes the time consumed by the statement execution. MySQL's binary log It is transaction safe.
Canal
Translated as waterway/pipeline/ditch, its main purpose is based on MySQL database incremental log analysis, providing incremental data subscription and consumption
canal, which can be used to monitor changes in database data to obtain new data or modified data.
The working principle of canal is to disguise itself as a MySQL slave, simulate the interaction protocol of MySQL slave and send the dump protocol to MySQL Mater. MySQL mater receives the dump request sent by canal and starts to push the binary log to canal. Then canal parses the binary log and then Sent to storage destination.
MySQL configuration
First, mysql needs to be installed in docker and the configuration file directory has been mounted.
Open the mysql configuration file directory mounted on the host.
vim my.conf
Add the following configuration to the configuration file to enable binlog
[mysqld]
skip-name-resolve
character_set_server=utf8
datadir=/var/lib/mysql
server-id=1000
log-bin=/var/lib/mysql/mysql-bin
binlog-do-db=canal
After completion, save and exit.
Restart mysql in docker.
Canal configuration
Create mounting directory
Create a new /opt/docker/canal/conf directory. We will place two configuration files later and mount them. Create a
new /opt/docker/canal/log directory for mounting log files.
mkdir -p /opt/docker/canal/{conf,log}
Setting permissions
chmod -R 777 /opt/docker/canal/conf
chmod -R 777 /opt/docker/canal/log
Create a MySQl Canal account
I configure it directly in navicat here. The username and password are both canal.
create user canal@'%' IDENTIFIED by 'canal';
GRANT SELECT, REPLICATION SLAVE, REPLICATION CLIENT,SUPER ON *.* TO 'canal'@'%' identified by 'canal';
FLUSH PRIVILEGES;
Pull image
docker pull canal/canal-admin:v1.1.5
Run container
Simple operation
Let’s run it briefly first to obtain the configuration file.
docker run --name canal -d canal/canal-server:v1.1.5
Copy the configuration file to the host machine
Copy the two configuration files in the container to the conf directory we created
docker cp canal:/home/admin/canal-server/conf/canal.properties /opt/docker/canal/conf
docker cp canal:/home/admin/canal-server/conf/example/instance.properties /opt/docker/canal/conf
Modify configuration file
Modify the instance.properties configuration file.
The first page looks like this. Here we will modify the parts in the box. The first one is the slaveId we set in the mysql configuration file. The following is the database address, IP: port number. The second page is long
. In this way, you only need to change the part in the box.
The above is the account and password we created for canal before, both of which are canal. The
following is the library table that needs to be monitored. The picture is the default state, monitoring all, and can be changed according to actual needs.
Delete the previously running canal container
Pause canal
docker stop canal
Delete canal
docker rm canal
Officially run the Canal container
Pay attention to modifying the mysql address
docker run -d -it -h 127.0.0.1 -e server.port=8089 \
-e canal.adminUser=admin -e canal.adminPasswd=admin \
-e spring.datasource.address=mysql地址:3306 \
-e spring.datasource.database= canal_manager \
-e spring.datasource.username= canal \
-e spring.datasource.password= canal \
--name=canal-admin -p 8089:8089 \
-m 1024m canal/canal-admin:v1.1.5
View running status
docker ps -a
success
Troubleshooting
Enter the created mounting log file directory log, enter the canal directory, there are two log files in it
If you encounter problems during operation, you can check these logs to troubleshoot the problem.
Finish