Binlog4j 1.2.0 Released, Java Lightweight Binary Log Client

Binlog4j

Lightweight Mysql Binlog client, providing features such as continuous reading after downtime, high-availability clusters, etc.

project characteristics

• Cluster mode ensures high availability of services through cluster deployment.

• Continue reading after downtime to avoid data loss during downtime.

• Data conversion, serialized data based on generic encapsulation binlog Event.

• Compatible with traditional projects and Spring Boot / Cloud projects.

• Compatible with Spring Boot 2.x and Spring Boot 3.x versions. project characteristics

Changelog (1.2.0)

[New] BinlogClient class connect method log printing.
[New] Binlog4jException.class replaces RuntimeException.class exception handling.
[Add] binlog4j-spring-boot-starter package spring.binlog4j.enabled configuration, the default is true.
[Add] GsonUtils utility class to utils package, providing Gson methods.
[Upgrade] spring-boot to version 2.7.14.
[Upgrade] redisson to version 3.23.3.
[Upgrade] gson to version 2.10.1.
[Upgrade] jedis to version 4.4.3

Download and install

<dependency>
   <groupId>com.gitee.Jmysy</groupId>
   <artifactId>binlog4j-core</artifactId>
   <version>latest.version</version>
</dependency>

or

implementation group: 'com.gitee.Jmysy', name: 'binlog4j-core', version: 'latest.version'

easy to use

Create a binlog client through BinlogClient, and IBinlogEventHandler is used to accept binlog event notifications. This interface allows the use of generics, and the data will be encapsulated according to the hump rule.

public class BootStrap {

    public static void main(String[] args) {
        
        BinlogClientConfig clientConfig = new BinlogClientConfig();
        clientConfig.setHost("127.0.0.1");
        clientConfig.setPort(3306);
        clientConfig.setUsername("root");
        clientConfig.setPassword("taoren@123");
        clientConfig.setServerId(1990);
  
        IBinlogClient binlogClient = new BinlogClient(clientConfig);

        binlogClient.registerEventHandler("database", "table", new IBinlogEventHandler() {
            
            @Override
            public void onInsert(BinlogEvent event) {
                System.out.println("插入数据:{}", event.getData());
            }

            @Override
            public void onUpdate(BinlogEvent event) {
                System.out.println("修改数据:{}", event.getData());
            }

            @Override
            public void onDelete(BinlogEvent event) {
                System.out.println("删除数据:{}", event.getData());
            }
        });

        binlogClient.connect();
    }
}

advanced features

By configuring Persistence to true to enable the function of continuous reading after downtime, Binlog4j will record the filename and position of binlog to redis, so you need to set the Redis configuration at the same time.

public class BootStrap {

    public static void main(String[] args) {

        RedisConfig redisConfig = new RedisConfig();
        redisConfig.setHost("127.0.0.1");
        redisConfig.setPort(6379);
        redisConfig.setPassword("taoren@123");

        BinlogClientConfig clientConfig = new BinlogClientConfig();
        clientConfig.setHost("127.0.0.1");
        clientConfig.setPort(3306);
        clientConfig.setUsername("root");
        clientConfig.setPassword("taoren@123");
        clientConfig.setServerId(1990); // Client 编号
        clientConfig.setRedisConfig(redisConfig); // Redis 配置
        clientConfig.setPersistence(true); // 启用持久化 (宕机重启后, 从上次读取的位置开始)
        clientConfig.setMode(BinlogClientMode.cluster); // 高可用集群

        BinlogClient binlogClient = new BinlogClient(clientConfig);

        binlogClient.registerEventHandler("database", "table", new IBinlogEventHandler<User>() {

            @Override
            public void onInsert(BinlogEvent<User> event) {
                System.out.println("插入数据:{}", event.getData());
            }

            @Override
            public void onUpdate(BinlogEvent<User> event) {
                System.out.println("修改数据:{}", event.getData());
            }

            @Override
            public void onDelete(BinlogEvent<User> event) {
                System.out.println("删除数据:{}", event.getData());
            }
        });

        binlogClient.connect();
    }
}

Spring Boot Starter

<dependency>
    <groupId>com.gitee.Jmysy</groupId>
    <artifactId>binlog4j-spring-boot-starter</artifactId>
    <version>latest.version</version>
</dependency>

or

implementation group: 'com.gitee.Jmysy', name: 'binlog4j-spring-boot-starter', version: 'latest.version'

First, fill in the BinlogClient configuration in application.yml

spring:
  binlog4j:
    redis-config:
      host: 127.0.0.1
      port: 6379
      password: taoren@123
    client-configs:
      master:
        username: root
        password: taoren@123
        host: 127.0.0.1
        port: 3306
        serverId: 1990

Single table monitoring

Use the @BinlogSubscriber annotation to specify which client IBinlogEventHandler needs to register to, and specify the database and table to monitor.

@BinlogSubscriber(clientName = "master", database = "pear-admin", table ="sys_user")
public class UserEventHandler implements IBinlogEventHandler<User> {

    @Override
    public void onInsert(BinlogEvent<User> event) {
        System.out.println("插入数据：" + event.getData());
    }

    @Override
    public void onUpdate(BinlogEvent<User> event) {
        System.out.println("修改数据:" + event.getData());
    }

    @Override
    public void onDelete(BinlogEvent<User> event) {
        System.out.println("删除数据:" + event.getData());
    }

}

Multi-table monitoring

The @BinlogSubscriber annotation database and table properties support Pattern matching. In the case of IBinlogEventHandler without specifying generics, event.getData() is of type Map<String, Object>, which is used to be compatible with data structures of different tables.

@BinlogSubscriber(clientName = "master", database = ".*", table ="sys.*")
public class UserEventHandler implements IBinlogEventHandler {

    @Override
    public void onInsert(BinlogEvent event) {
        System.out.println("数据库：" + event.getDatabase());
        System.out.println("数据表：" + event.getTable());
        System.out.println("新数据：" + event.getData());
    }

    @Override
    public void onUpdate(BinlogEvent event) {
        System.out.println("数据库：" + event.getDatabase());
        System.out.println("数据表：" + event.getTable());
        System.out.println("原数据：" + event.getOriginalData());
        System.out.println("新数据：" + event.getData());
    }

    @Override
    public void onDelete(BinlogEvent event) {
        System.out.println("数据库：" + event.getDatabase());
        System.out.println("数据表：" + event.getTable());
        System.out.println("新数据：" + event.getData());
    }

}

Monitor the data changes of all tables in all databases of the master data source [the most flexible]

@BinlogSubscriber(clientName = "master")

Monitor the data changes of all tables in the master data source pear-admin database

@BinlogSubscriber(clientName = "master", database="pear-admin")

Monitor data changes in the user table in the master data source pear-admin database

@BinlogSubscriber(clientName = "master", database="pear-admin", table="user")

Monitor data changes in user tables in all databases of the master data source

@BinlogSubscriber(clientName = "master", table="user")

Monitor data changes in tables starting with user in all databases of the master data source

@BinlogSubscriber(clientName = "master", table="user.*")