Maxwell introduced

Maxwell is a read MySQL binlog row updates as JSON and write applications Kafka, Kinesis or other streaming platforms.

Maxwell Quick Start

1. Download the Maxwell (can choose one of three ways)

takes

curl -sLo - https://github.com/zendesk/maxwell/releases/download/v1.20.0/maxwell-1.20.0.tar.gz \ | tar zxvf - cd maxwell-1.20.0

Docker

docker pull zendesk/maxwell

macOS homebrew

brew install maxwell

MySQL Configuration

Modify my.cnf

vim /etc/mysql/my.cnf

[mysqld]
server_id=1
log-bin=master binlog_format=row

Create a corresponding user and the database

mysql> CREATE USER 'maxwell'@'%' IDENTIFIED BY 'maxwell';
mysql> CREATE DATABASE maxwell; mysql> GRANT ALL ON maxwell.* TO 'maxwell'@'%'; mysql> GRANT SELECT, REPLICATION CLIENT, REPLICATION SLAVE ON *.* TO 'maxwell'@'%'; mysql> flush privileges;

After configuration is complete, restart MySQL

Run Maxwell

Command Line

maxwell --user='maxwell' --password='maxwell' --host='127.0.0.1' --producer=stdout

Create a test database and test table

create database test;
use test;
create table maxwell(id int,daemon varchar(100)); insert into `test`.`maxwell` set id = 1, daemon = 'Stanislaw Lem'; update test.maxwell set daemon = 'firebus! firebus!' where id = 1;

Console output

{
	"database": "test",
	"table": "maxwell", "type": "insert", "ts": 1551862585, "xid": 381, "commit": true, "data": { "id": 1, "daemon": "Stanislaw Lem" } } { "database": "test", "table": "maxwell", "type": "update", "ts": 1551862800, "xid": 450, "commit": true, "data": { "id": 1, "daemon": "firebus! firebus!" }, "old": { "daemon": "Stanislaw Lem" } }

Kafka

Start Zookeeper

zkServer.sh start

Start kafka server

kafka-server-start.sh /opt/kafka/config/server-1.properties

Start Maxwell

maxwell --user='maxwell' --password='maxwell' --host='127.0.0.1' --producer=kafka --kafka.bootstrap.servers=localhost:9092 --kafka_topic=maxwell

Start kafka-console-consumer news consumption

kafka-console-consumer.sh  --bootstrap-server localhost:9092 --topic maxwell

Create test data for testing

insert into `test`.`maxwell` set id = 1, daemon = 'Stanislaw Lem'; update test.maxwell set daemon = 'firebus! firebus!!' where id = 1;

MySQL real-time data source synchronization middleware -Maxwell