Canal combat | SpringBoot integrates Canal + RabbitMQ to monitor MySQL database and update Redis cache synchronously

Foreword: How to ensure data consistency between redis and mysql? Most of the answers on the Internet are delayed double deletion. Now I will share another solution with you. I should also come up with it in a recent project.

1. Open the biglog log (change according to the location of your mysql configuration file)

vim /etc/my.cnf

add configuration

[mysqld]
log-bin=mysql-bin # 开启binlog
binlog-format=ROW # 选择ROW模式
server_id=1 # 配置MySQL replaction需要定义，不和Canal的slaveId重复即可

2. Restart MySQL to check whether the configuration takes effect

show variables like 'log_bin';

 3. RabbitMQ queue creation 

• Add exchange canal_exchange

• Add queue canal_queue

• Queue Binding Exchange

4. Canal configuration and startup

Canal ServerDownload

• Official documentation: Home · alibaba/canal Wiki · GitHub
• Project address: GitHub - alibaba/canal: Alibaba MySQL binlog incremental subscription & consumption component
• Download address: Releases alibaba/canal GitHub

Enter the download address and select  canal.deployer-1.1.5.tar.gz

Canal Server configuration

There are only two things that need to be configured, one is the monitoring database configuration, and the other is the RabbitMQ connection configuration.

The two changed files are the Canal configuration file  canal.properties and the instance configuration file instance.properties

㊙️: A Server can be configured with multiple instance monitors. The Canal function comes with an example instance by default. This article uses the example instance. If adding an instance, copy the content of the example folder to the same level directory, and then  canal.properties specify the name of the added instance.

The extracted directory information:

canal.properties

Configure Canal service mode as RabbitMQ and connection configuration

Go to the conf file and open canal.properties

Change the principle tcp mode to rabbitMQ

 

Here we will complete the canal connection rabbitMQ configuration

instance.properties

Monitor database configuration

cd /example directory

 

 5. SpringBoot integrates Canal + RabbitMQ

Introduce maven dependency

<dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-amqp</artifactId>
</dependency>

RabbitMQ connection configuration

spring:
  rabbitmq:
    host: ip
    port: 端口
    username: 用户名
    password: 密码

RabbitMQ monitors synchronous cache

 Returns json mapped bean class CanalMessage

/**
 * @author yueF_L
 * @version 1.0
 * @date 2022-04-10 0:30
 * Canal消息接收实体类
 */
@NoArgsConstructor
@Data
public class CanalMessage<T> {
    @JsonProperty("type")
    private String type;

    @JsonProperty("table")
    private String table;

    @JsonProperty("data")
    private List<T> data;

    @JsonProperty("database")
    private String database;

    @JsonProperty("es")
    private Long es;

    @JsonProperty("id")
    private Integer id;

    @JsonProperty("isDdl")
    private Boolean isDdl;

    @JsonProperty("old")
    private List<T> old;

    @JsonProperty("pkNames")
    private List<String> pkNames;

    @JsonProperty("sql")
    private String sql;

    @JsonProperty("ts")
    private Long ts;

}