《Kafka源码解析与实战》Broker的控制管理模块

Kafka broker 相关结合实战，待补充细致

KakfaController的选举

在这里插入图片描述
创建一个 control 瞬时节点（Ephemeral Node），创建该瞬时节点的原子性由zookeeper保证

bin/zkCli.sh -server 172.16.227.250:2181,172.16.227.129:2181,172.16.227.135:2181

在这里插入图片描述

KakfaController 初始化

初始化Kafka集群内部的时钟，其时钟的具体数值是存在zookeeper上，路径为/contoller_epoch, Broker Server 利用此值区分请求的实效性
注册各种监听函数，Kafka把元数据持久化在Zookeeper上, 所以在zookeeper不同路径注册各种监听函数

/broker/seqid

在这里插入图片描述
自动生成broker.id的原理是先往/brokers/seqid节点中写入一个空字符串，然后获取返回的Stat信息中的version的值，然后将version的值和reserved.broker.max.id参数配置的值相加可得。之所以是先往节点中写入数据再获取Stat信息，这样可以确保返回的version值大于0，进而就可以确保生成的broker.id值大于reserved.broker.max.id参数配置的值，符合非自动生成的broker.id的值在[0, reserved.broker.max.id]区间的设定。

作者：朱小厮
来源：CSDN
原文：https://blog.csdn.net/u013256816/article/details/80546337
版权声明：本文为博主原创文章，转载请附上博文链接！

Topic分区状态转换

在这里插入图片描述

topic的创建，删除
topic的分区变化: PartitionStateMachine

分区状态

/**
 * This class represents the state machine for partitions. It defines the states that a partition can be in, and
 * transitions to move the partition to another legal state. The different states that a partition can be in are -
 * 1. NonExistentPartition: This state indicates that the partition was either never created or was created and then
 *                          deleted. Valid previous state, if one exists, is OfflinePartition
 * 2. NewPartition        : After creation, the partition is in the NewPartition state. In this state, the partition should have
 *                          replicas assigned to it, but no leader/isr yet. Valid previous states are NonExistentPartition
 * 3. OnlinePartition     : Once a leader is elected for a partition, it is in the OnlinePartition state.
 *                          Valid previous states are NewPartition/OfflinePartition
 * 4. OfflinePartition    : If, after successful leader election, the leader for partition dies, then the partition
 *                          moves to the OfflinePartition state. Valid previous states are NewPartition/OnlinePartition
 */
class PartitionStateMachine(config: KafkaConfig,
                            stateChangeLogger: StateChangeLogger,
                            controllerContext: ControllerContext,
                            topicDeletionManager: TopicDeletionManager,
                            zkClient: KafkaZkClient,
                            partitionState: mutable.Map[TopicPartition, PartitionState],
                            controllerBrokerRequestBatch: ControllerBrokerRequestBatch) extends Logging {

分区Leader

在这里插入图片描述

KafkaController内部各种监视器

TopicChangeListener

AddPartitionsListener

PartitionsReassignedListener

ReassignedPartitionsIsr-ChangeListener

PreferredReplicationElection-Listener

BrokerChangerListener

DeleteTopicsListener

Kafka集群的负载均衡流程

Partiton的AR列表中的第一个Replica称为Preferred Replica,并均匀分布在整个Kafka集群中；每个Partition只有Leader replica对外提供读写服务，一般Preferred Replica是Leader replica

某个broker宕机导致负载不均衡

在这里插入图片描述

broker 宕机

broker恢复，`partition-rebalance-thread`线程负责重新负载均衡

基本术语

分区中的所有副本统称为AR（Assigned Repllicas）。

所有与leader副本保持一定程度同步的副本（包括Leader）组成ISR（In-Sync Replicas）

master-dragon

发布了441 篇原创文章 · 获赞 110 · 访问量 57万+

他的留言板关注