Kafka操作命令大全

启动

nohup ./kafka-server-start.sh ../config/server.properties > /dev/null 2>&1 &

停止

./kafka-server-stop.sh 

生产消息

./kafka-console-producer.sh --broker-list *:9092 --topic test

查看Topics状态

./kafka-topics.sh --describe --zookeeper */kafka --topic test

查看topic列表

./kafka-topics.sh --zookeeper *:2181 --list

查看指定Topic的配置

./kafka-configs.sh --zookeeper */kafka --entity-type topics --entity-name __consumer_offsets --describe

更改指定Topic的配置

例如修改__consumer_offsets的保留策略为删除(默认压缩)

./kafka-configs.sh --zookeeper */kafka --entity-type topics --entity-name __consumer_offsets --alter --delete-config cleanup.policy=delete

创建Topic

./kafka-topics.sh --create --topic test0 --zookeeper */kafka --config max.message.bytes=12800000 --config flush.messages=1 --partitions 5 --replication-factor 1

删除Topic

./kafka-topics.sh --delete --zookeeper */kafka --topic test

消费指定Topic,查找关键字

./kafka-console-consumer.sh  --bootstrap-server *:9092 --topic test --offset 424942891 --group kh_test --partition 0|grep  227471233

修改Kafka集群配置

./kafka-configs.sh --bootstrap-server *:9092 --entity-type brokers --entity-default --alter --add-config min.insync.replicas=2

重新分配Partition和副本

./kafka-reassign-partitions.sh -zookeeper */kafka --reassignment-json-file topic-reassignment.json --execute

例如:__consumer_offsets是单副本,需要修改成多副本,需指定json:

{
    "version": 1, 
    "partitions": [
        {
            "topic": "__consumer_offsets", 
            "partition": 0, 
            "replicas": [
                0, 
                1, 
                3
            ]
        },
        {
            "topic": "__consumer_offsets", 
            "partition": 1, 
            "replicas": [
                1, 
                3, 
                0
            ]
        },
        {
            "topic": "__consumer_offsets", 
            "partition": 2, 
            "replicas": [
                3, 
                0, 
                1
            ]
        },
        {
            "topic": "__consumer_offsets", 
            "partition": 3, 
            "replicas": [
                0, 
                1, 
                3
            ]
        },
        {
            "topic": "__consumer_offsets", 
            "partition": 4, 
            "replicas": [
                1, 
                3, 
                0
            ]
        },
        {
            "topic": "__consumer_offsets", 
            "partition": 5, 
            "replicas": [
                3, 
                0, 
                1
            ]
        },
        {
            "topic": "__consumer_offsets", 
            "partition": 6, 
            "replicas": [
                0, 
                1, 
                3
            ]
        },
        {
            "topic": "__consumer_offsets", 
            "partition": 7, 
            "replicas": [
                1, 
                3, 
                0
            ]
        },
        {
            "topic": "__consumer_offsets", 
            "partition": 8, 
            "replicas": [
                3, 
                0, 
                1
            ]
        },
        {
            "topic": "__consumer_offsets", 
            "partition": 9, 
            "replicas": [
                0, 
                1, 
                3
            ]
        },
        {
            "topic": "__consumer_offsets", 
            "partition": 10, 
            "replicas": [
                1, 
                3, 
                0
            ]
        },
        {
            "topic": "__consumer_offsets", 
            "partition": 11, 
            "replicas": [
                3, 
                0, 
                1
            ]
        },
        {
            "topic": "__consumer_offsets", 
            "partition": 12, 
            "replicas": [
                0, 
                1, 
                3
            ]
        },
        {
            "topic": "__consumer_offsets", 
            "partition": 13, 
            "replicas": [
                1, 
                3, 
                0
            ]
        },
        {
            "topic": "__consumer_offsets", 
            "partition": 14, 
            "replicas": [
                3, 
                0, 
                1
            ]
        },
        {
            "topic": "__consumer_offsets", 
            "partition": 15, 
            "replicas": [
                0, 
                1, 
                3
            ]
        },
        {
            "topic": "__consumer_offsets", 
            "partition": 16, 
            "replicas": [
                1, 
                3, 
                0
            ]
        },
        {
            "topic": "__consumer_offsets", 
            "partition": 17, 
            "replicas": [
                3, 
                0, 
                1
            ]
        },
        {
            "topic": "__consumer_offsets", 
            "partition": 18, 
            "replicas": [
                0, 
                1, 
                3
            ]
        },
        {
            "topic": "__consumer_offsets", 
            "partition": 19, 
            "replicas": [
                1, 
                3, 
                0
            ]
        },
        {
            "topic": "__consumer_offsets", 
            "partition": 20, 
            "replicas": [
                3, 
                0, 
                1
            ]
        },
        {
            "topic": "__consumer_offsets", 
            "partition": 21, 
            "replicas": [
                0, 
                1, 
                3
            ]
        },
        {
            "topic": "__consumer_offsets", 
            "partition": 22, 
            "replicas": [
                1, 
                3, 
                0
            ]
        },
        {
            "topic": "__consumer_offsets", 
            "partition": 23, 
            "replicas": [
                3, 
                0, 
                1
            ]
        },
        {
            "topic": "__consumer_offsets", 
            "partition": 24, 
            "replicas": [
                0, 
                1, 
                3 
            ]
        },
        {
            "topic": "__consumer_offsets", 
            "partition": 25, 
            "replicas": [
                0, 
                1, 
                3
            ]
        },
        {
            "topic": "__consumer_offsets", 
            "partition": 26, 
            "replicas": [
                0, 
                1, 
                3
            ]
        },
        {
            "topic": "__consumer_offsets", 
            "partition": 27, 
            "replicas": [
                0, 
                1, 
                3
            ]
        },
        {
            "topic": "__consumer_offsets", 
            "partition": 28, 
            "replicas": [
                0, 
                1, 
                3
            ]
        },
        {
            "topic": "__consumer_offsets", 
            "partition": 29, 
            "replicas": [
                0, 
                1, 
                3
            ]
        },
        {
            "topic": "__consumer_offsets", 
            "partition": 30, 
            "replicas": [
                0, 
                1, 
                3
            ]
        },
        {
            "topic": "__consumer_offsets", 
            "partition": 31, 
            "replicas": [
                0, 
                1, 
                3
            ]
        },
        {
            "topic": "__consumer_offsets", 
            "partition": 32, 
            "replicas": [
                0, 
                1, 
                3
            ]
        },
        {
            "topic": "__consumer_offsets", 
            "partition": 33, 
            "replicas": [
                0, 
                1, 
                3
            ]
        },
        {
            "topic": "__consumer_offsets", 
            "partition": 34, 
            "replicas": [
                0, 
                1, 
                3
            ]
        },
        {
            "topic": "__consumer_offsets", 
            "partition": 35, 
            "replicas": [
                0, 
                1, 
                3
            ]
        },
        {
            "topic": "__consumer_offsets", 
            "partition": 36, 
            "replicas": [
                1, 
                3, 
                0
            ]
        },
        {
            "topic": "__consumer_offsets", 
            "partition": 37, 
            "replicas": [
                1, 
                3, 
                0
            ]
        },
        {
            "topic": "__consumer_offsets", 
            "partition": 38, 
            "replicas": [
                1, 
                3, 
                0
            ]
        },
        {
            "topic": "__consumer_offsets", 
            "partition": 39, 
            "replicas": [
                1, 
                3, 
                0
            ]
        },
        {
            "topic": "__consumer_offsets", 
            "partition": 40, 
            "replicas": [
                1, 
                3, 
                0
            ]
        },
        {
            "topic": "__consumer_offsets", 
            "partition": 41, 
            "replicas": [
                1, 
                3, 
                0
            ]
        },
        {
            "topic": "__consumer_offsets", 
            "partition": 42, 
            "replicas": [
                1, 
                3, 
                0
            ]
        },
        {
            "topic": "__consumer_offsets", 
            "partition": 43, 
            "replicas": [
                1, 
                3, 
                0
            ]
        },
        {
            "topic": "__consumer_offsets", 
            "partition": 44, 
            "replicas": [
                3, 
                0, 
                1
            ]
        },
        {
            "topic": "__consumer_offsets", 
            "partition": 45, 
            "replicas": [
                3, 
                0, 
                1
            ]
        },
        {
            "topic": "__consumer_offsets", 
            "partition": 46, 
            "replicas": [
                3, 
                0, 
                1
            ]
        },
        {
            "topic": "__consumer_offsets", 
            "partition": 47, 
            "replicas": [
                3, 
                0, 
                1
            ]
        },
        {
            "topic": "__consumer_offsets", 
            "partition": 48, 
            "replicas": [
                3, 
                0, 
                1
            ]
        },
        {
            "topic": "__consumer_offsets", 
            "partition": 49, 
            "replicas": [
                3, 
                0, 
                1
            ]
        }
    ]
}

发布了55 篇原创文章 · 获赞 14 · 访问量 2万+

猜你喜欢

转载自blog.csdn.net/qq422243639/article/details/100118984