Kafka basic concept learning

Basic concept learning

1. Basic concepts

Kafka 基本概念:一种分布式的,基于发布/订阅的消息系统
project Value
Broker Kafka server (that is, the machine where Kafka is deployed)
Producer Producer writes data to primary replica
Consumer Consumers read data from the primary copy
Consumer Group Consumer group, multiple topics can be subscribed under the same group, each topic partition does not allow repeated consumption; multiple different groups can consume the same partition of the same topic
Broker One instance (one machine) of the Kafka server, and each instance has a master copy or a slave copy
Topic Topics, all data is isolated and interactive based on topics and partitions (combination of multiple message queues)
Partition Partition, corresponding to a data partition (message queue) of a certain topic, improves concurrent read and write capabilities
Offset Offset, the offset position of a certain consumption under a certain Partition of a certain Topic
Leader The master copy interacts with producers and consumers to complete data reading and writing
Follower From the copy, synchronize data from the master copy to ensure high availability of the system
ZooKeeper/KRaft Registry, used to manage the binding relationship of topics, partitions, offsets, producers, and consumers
rebalance Rebalancing, the consumer heartbeat thread communicates with the Coordinator, and the heartbeat timeout caused by abnormalities such as consumer downtime will trigger rebalancing
ISR (IN-SYNC REPLICAS) Synchronize the progress from the replica. Brokers that are considered out of sync will be removed from the list and cannot participate in the election of the master node

2. Structural diagram

insert image description here

Guess you like

Origin blog.csdn.net/weixin_42176639/article/details/131647885