MirrorMaker Kafka cross-cluster synchronization solution

The following is the cluster synchronization scheme I configured before.

 

Script path

/usr/local/kafka/bin/kafka-mirror-maker.sh

 

consumer.properties

bootstrap.servers=10.255.30.152:9092

group.id=mirrorGroupTest

auto.offset.reset=earliest

partition.assignment.strategy=org.apache.kafka.clients.consumer.RoundRobinAssignor

producer.properties

bootstrap.servers=10.255.30.132:9092 compression.type=none retries=3

Start script: start-mirror-maker.sh

nohup bin/kafka-mirror-maker.sh --consumer.config MirrorMaker/consumer.properties --producer.config MirrorMaker/producer.properties --whitelist ' test' &

 

If you need to debug, you can modify it by viewing the debug log

/usr/local/kafka/config/tools-log4j.properties

Next, test:

Issues that need attention :

consumer.properties (10.255.30.152:9092) The message source cluster that needs to be synchronized

producer.properties (10.255.30.132:9092) is the message target cluster

Result: The message is written to the 10.255.30.152:9092 cluster, and the message will be copied to the target cluster 10.255.30.132:9092.

 

 Thousands of coupons every day on Huihui Street

 

Guess you like

Origin blog.csdn.net/xiaoping117/article/details/109649620