Maxwell 介绍

Maxwell是一个读取MySQL binlog并将行更新作为JSON写入Kafka，Kinesis或其他流平台的应用程序。

Maxwell快速开始

1.下载Maxwell（以下三种方式任选一种即可）

tar

curl -sLo - https://github.com/zendesk/maxwell/releases/download/v1.20.0/maxwell-1.20.0.tar.gz \ | tar zxvf - cd maxwell-1.20.0

Docker

docker pull zendesk/maxwell

macOS homebrew

brew install maxwell

配置MySQL

修改my.cnf

vim /etc/mysql/my.cnf

[mysqld]
server_id=1
log-bin=master binlog_format=row

创建相应的用户和数据库

mysql> CREATE USER 'maxwell'@'%' IDENTIFIED BY 'maxwell';
mysql> CREATE DATABASE maxwell; mysql> GRANT ALL ON maxwell.* TO 'maxwell'@'%'; mysql> GRANT SELECT, REPLICATION CLIENT, REPLICATION SLAVE ON *.* TO 'maxwell'@'%'; mysql> flush privileges;

配置完成之后，重启MySQL

运行Maxwell

命令行

maxwell --user='maxwell' --password='maxwell' --host='127.0.0.1' --producer=stdout

创建测试数据库和表进行测试

create database test;
use test;
create table maxwell(id int,daemon varchar(100)); insert into `test`.`maxwell` set id = 1, daemon = 'Stanislaw Lem'; update test.maxwell set daemon = 'firebus! firebus!' where id = 1;

控制台输出

{
	"database": "test",
	"table": "maxwell", "type": "insert", "ts": 1551862585, "xid": 381, "commit": true, "data": { "id": 1, "daemon": "Stanislaw Lem" } } { "database": "test", "table": "maxwell", "type": "update", "ts": 1551862800, "xid": 450, "commit": true, "data": { "id": 1, "daemon": "firebus! firebus!" }, "old": { "daemon": "Stanislaw Lem" } }

Kafka

启动Zookeeper

zkServer.sh start

启动kafka server

kafka-server-start.sh /opt/kafka/config/server-1.properties

启动Maxwell

maxwell --user='maxwell' --password='maxwell' --host='127.0.0.1' --producer=kafka --kafka.bootstrap.servers=localhost:9092 --kafka_topic=maxwell

启动kafka-console-consumer消费消息

kafka-console-consumer.sh  --bootstrap-server localhost:9092 --topic maxwell

创建测试数据进行测试

insert into `test`.`maxwell` set id = 1, daemon = 'Stanislaw Lem'; update test.maxwell set daemon = 'firebus! firebus!!' where id = 1;

实时MySQL数据源同步中间件-Maxwell