Kafka Basics

1, every partition has a leader and one or more followers. Leader selection is done with the help of the Zookeeper.

2, Zookeeper helps coordinate the cluster state of the Kafka brokers. Consumers and producers of the Kafka queues consult the Zookeeper about the Kafka brokers, and leaders and followers of all the partitions of the topics they subscribe (or publish) to.

3, producers search for the elected leader (broker) of the partition it is pushing data to and then write at the end of the partition log files. Producers first find the topic, is assigned a partition according to some metric of the message (e.g., hashing of the message id), and then search for the elected leader of the partition.

4, consumers store message offsets at the Zookeeper. Once it sets its offset, it acknowledges that it has consumed all the previous messages before the offset.

猜你喜欢

转载自blog.csdn.net/qq_25527791/article/details/88421349