Information stored by Kafka in zookeeper

Kafka storage structure in zookeeper

1.topic registration information

/brokers/topics/[topic] :

Store all allocation information of partitions of a topic

Information for each topic

{
    
    
    "version":1,  // 版本编号目前固定为数字1
    "partitions":{
    
      // partitionId编号
        "1":[  //同步副本组brokerId列表
            101,
            103
        ],
        "0":[
            103,
            102
        ]
    }
}

Partition status information

/brokers/topics/[topic]/partitions/[0…N] where [0…N] represents the partition index number
/brokers/topics/[topic]/partitions/[partitionId]/state

{
    
    
    "controller_epoch":33,  //表示kafka集群中的中央控制器选举次数
    "leader":102,    //表示该partition选举leader的brokerId
    "version":1,   //版本编号默认为1
    "leader_epoch":8,   // 该partition leader选举次数,
    "isr":[    // 同步副本组brokerId列表
        102,
        103
    ]
}

2. Broker registration information

/brokers/ids/[0…N]

Each broker's configuration file needs to specify a numeric id (globally non-repeatable), and this node is a temporary znode (EPHEMERAL)

{
    
    
    "listener_security_protocol_map":{
    
    
        "PLAINTEXT":"PLAINTEXT"
    },
    "endpoints":[
        "PLAINTEXT://zjj101:9092"
    ],
    "jmx_port":-1,  // jmx端口号
    "host":"zjj101",  // 主机名或ip地址,
    "timestamp":"1614570663881",  //kafka broker初始启动时的时间戳,
    "port":9092, // kafka broker的服务端端口号,由server.properties中参数port确定
    "version":4  // 版本编号默认为1,
}

3.Controller epoch

/controller_epoch --> int (epoch)

This value is a number. The first broker in the Kafka cluster is 1 when it is started for the first time. In the future, as long as the broker where the center controller of the center controller in the cluster changes or hangs, a new center controller will be re-elected, each time the center controller Change the controller_epoch value and it will be + 1;

4. Controller registration information

/controller -> int (broker id of the controller) stores the information of the kafka broker where the center controller is located

{
    
    
    "version":1,   // 版本编号默认为1,
    "brokerid":101,  // kafka集群中broker唯一编号,
    "timestamp":"1614570663255"  // kafka broker中央控制器变更时的时间戳
}