kafka初识(三)常用命令

1、启动

kafka和zk一样,都是启动命令调用配置文件,一般命令加&,放在后台执行。

first:bin/zookeeper-server-start.sh config/zookeeper.properties &

Second:bin/kafka-server-start.sh config/server.properties &

2、创建topic

创建topic时需要指定副本和分区,副本数不能大于Broker数量,分区数根据实际需求,不是越多越好。

bin/kafka-topic.sh --create --zookeeper zk1:2181,zk2:2181,zk3:2181 --replication-factor 3 --partitions 3 --topic test

3、查看topic

可以查看topic列表及描述,describe中能看到分区、副本及ISR信息。

bin/kafka-topic.sh --describe/list --zookeeper zk1:2181,zk2:2181,zk3:2181 --topic test

4、删除topic

直接删除topic,包含数据文件、zk节点,注意server.properties中要开启 delete.topic.enable=true。

bin/kafka-topic.sh --delete --zookeeper zk1:2181,zk2:2181,zk3:2181 --topic test

5、生产消息

生产可以命令中指定参数,也可以调用配置文件。

bin/kafka-console-producer.sh --broker-list broker1:9092,broker2:9092,broker3:9092 --topic test --producer.config config/producer.properties

6、消费消息

消费可以命令中指定参数,也可以调用配置文件。

bin/kafka-console-consumer.sh --zookeeper zk1:2181,zk2:2181,zk3:2181 --topic test --from-beginning --consumer.config config/consumer.properties

7、查看offset

0.8之后的版本命令做了修改,可以查看消费的offset值。

bin/kafka-consumer-offset-checker.sh --zookeeper zk1:2181,zk2:2181,zk3:2181 --topic test --group TestGroup   (——0.8)

bin/kafka-run-class.sh kafka.tools.GetOffsetShell --topic test --broker-list broker1:9092, broker2:9092, broker3:9092 --time -1   (0.9——)

8、查看ConsumerGroup

显示当前所有消费组。

bin/kafka-consumer-groups.sh --list --zookeeper zk1:2181,zk2:2181,zk3:2181

9、查看组实时消费情况

显示topic、分区、消息量、offset、消费延迟、消费id等信息。

bin/kafka-consumer-groups.sh --describe --zookeeper zk1:2181,zk2:2181,zk3:2181 --group console-consumer-xxxxx

10、增加分区

动态增加分区,分区数字为增加到的值。

bin/kafka-topics.sh --alter --zookeeper zk1:2181,zk2:2181,zk3:2181 --topic test --partitions 7 

11、重新分配分区

先创建一个json文件,格式如下,指定分区,执行--topics-to-move-json-file test.json,会显示当前分区情况和推荐分区方式。新建一个重新分区的json文件,可以参考推荐分区方式,执行--reassignment-json-file,完成分区,可以把分区调整到任意副本,一般用于Broker迁移,这属于强制指定。执行--verify 验证重分区结果。

[root@SZB-L0047709 kafka_2.10-0.10.2.0]# less test.json

{"topics": [{"topic": "test3"}], "version":1 }

[root@SZB-L0047709 kafka_2.10-0.10.2.0]# bin/kafka-reassign-partitions.sh --topics-to-move-json-file test.json  --zookeeper zk1:2181,zk2:2181,zk3:2181 --broker-list “0,1,2" --generate

Current partition replica assignment

{"version":1,"partitions":[{"topic":"test3","partition":0,"replicas":[2]},{"topic":"test3","partition":1,"replicas":[0]},{"topic":"test3","partition":2,"replicas":[1]}]}

Proposed partition reassignment configuration

{"version":1,"partitions":[{"topic":"test3","partition":0,"replicas":[0]},{"topic":"test3","partition":1,"replicas":[0]},{"topic":"test3","partition":2,"replicas":[0]}]}

[root@SZB-L0047709 kafka_2.10-0.10.2.0]# bin/kafka-reassign-partitions.sh --execute --reassignment-json-file reassign.json --zookeeper zk1:2181,zk2:2181,zk3:2181

Current partition replica assignment

{"version":1,"partitions":[{"topic":"test3","partition":0,"replicas":[2]},{"topic":"test3","partition":1,"replicas":[0]},{"topic":"test3","partition":2,"replicas":[1]}]}

Save this to use as the --reassignment-json-file option during rollback

Successfully started reassignment of partitions.

[root@SZB-L0047709 kafka_2.10-0.10.2.0]# bin/kafka-reassign-partitions.sh --verify --reassignment-json-file reassign.json --zookeeper zk1:2181,zk2:2181,zk3:2181

Status of partition reassignment:

Reassignment of partition [test3,0] completed successfully

Reassignment of partition [test3,1] completed successfully

Reassignment of partition [test3,2] completed successfully

12、Producer压测命令

bin/kafka-producer-perf-test.sh --topic well --num-records 100 --record-size 1 --throughput 100  --producer-props bootstrap.servers=broker1:9092,broker2:9092,broker3:9092

13、Consumer压测命令

bin/kafka-consumer-perf-test.sh  --topic well --messages 1000 --broker-list  broker1:9092,broker2:9092,broker3:9092

14、Zookeeper常用命令

zkCli.sh –server 127.0.0.1:2181 —— 登录客户端

      ls /znode、get /znode 、create /znode 、set /znode  ……

echo stat |nc 127.0.0.1 2181 —— 四字命令

猜你喜欢

转载自blog.csdn.net/qq_15051497/article/details/88892766
今日推荐