Kafka common commands
I feel that I haven't played Kafka for a while, and I'm a little rusty. Today, we will explain how to use commands to operate Kafka:
- start command
- Create Topic
- View topic list
- delete topic
- Producer and Consumer write and consume data
- other commands
According to the previous article How to build a Kafka system on Alibaba Cloud , everyone must already know how to configure Kafka, but what should be the next step after Kafka? This is the main content of this section.
1. Start command
Starting kafka is very simple. After configuring the pre-operation, first enter the directory of zookeeper and start zookeeper
First go to the ../Software/Zookeeper/zookeeper-3.4.6 directory
cd zookeeper-3.4.6
//先看下当前目录
pwd
结果：/Users/XXX/Software/Zookeeper/zookeeper-3.4.6
//启动Zookeeper
./bin/zkServer.sh start
//关闭Zookeeper 当然我们这边就不用关闭了，大家知道就好
./bin/zkServer.sh stop- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
The result after startup is as follows, which means that the command was executed successfully:
sh-3.2# ./bin/zkServer.sh start
JMX enabled by default
Using config: /Users/Sean/Software/Zookeeper/zookeeper-3.4.6/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED- 1
- 2
- 3
- 4
PS: Because kafka needs to be registered on zookeeper and managed and scheduled through zookeeper, we must start zookeeper before starting kafka.
Then we enter the Kafka directory and start kafka. The specific operation commands are as follows:
cd kafka_2.11-0.10.1.0
//查看当前目录
pwd
/Users/xxx/Software/Kafka/kafka_2.11-0.10.1.0
//启动kafka
./bin/kafka-server-start.sh ./config/server.properties- 1
- 2
- 3
- 4
- 5
- 6
After seeing no error, kafka has been successfully run. Of course, you can check it with jps -lm. The operation command is as follows:
jps -lm
// 结果如下
7111 org.apache.zookeeper.server.quorum.QuorumPeerMain /Users/Sean/Software/Zookeeper/zookeeper-3.4.6/bin/../conf/zoo.cfg
7357 sun.tools.jps.Jps -lm
7119 kafka.Kafka ./config/server.properties
sh-3.2#- 1
- 2
- 3
- 4
- 5
- 6
You can see that both zookeeper and kafka are already running. Of course, this is a single-machine command, and the cluster command will be discussed later.
Another disadvantage of starting this way is that after kafka is started, the terminal cannot be closed. For this, we can run this command:
nohup ./bin/kafka-server-start.sh config/server.properties > /dev/null 2>&1 &- 1
运行结果：
[1] 7404- 1
- 2
If there are multiple kafkas, you can run the kafka startup command multiple times on each virtual machine.
2. Create Topic
The command to create a topic is as follows:
Create a topic for test
./bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test- 1
Multi-cluster creation, executing this requires building a multi-machine kafka cluster environment, zkq1/zkq2/zkq3 represent three machines of three zookeeper clusters respectively
./bin/kafka-topics.sh —create —zookeeper zkq1:2181,zkq2:2181,zkq3:2181 -replication-factor 6 —partition 6 —topic test- 1
3. View Topics
After the creation is complete, we need to check whether the topic has been created successfully. You can execute the following command:
./bin/kafka-topics.sh --list --zookeeper 127.0.0.1:2181- 1
Similarly, in the case of a zookeeper cluster, just write the addresses of the three machines in the cluster.
The steps of operation are as follows, you will know at a glance:
//创建Topic test test2 test3
sh-3.2# ./bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test
//结果
Created topic "test".
sh-3.2# ./bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test2
//结果
Created topic "test2".
sh-3.2# ./bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic test3
//结果
Created topic "test3".
//查询
sh-3.2# ./bin/kafka-topics.sh --list --zookeeper 127.0.0.1:2181
//结果
test
test2
test3- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
4. Delete Topic
Deleting a Topic page is actually very simple. The execution command is as follows:
./bin/kafka-topics.sh --zookeeper 127.0.0.1:2181 --delete --topic test- 1
Of course, if there is no configuration related configuration in your server.properties, the following error will occur:
Topic test is marked for deletion.
Note: This will have no impact if delete.topic.enable is not set to true.- 1
- 2
This means that your topic has been marked as a topic to be deleted, but the switch of your configuration file is not turned on, so you just add a mark to it. In fact, the topic has not been deleted. Only, after you turn on the switch, the Topic marked for deletion will be automatically deleted.
Solution:
Set "delete.topic.enable=true" in the server.properties file and restart Kafka.
Reference:
kafka delete operation
5. Send and consume data
send data
./bin/kafka-console-producer.sh --broker-list 127.0.0.1:9092 --topic test
//输入内容
sdfsdfsdf
sdfsdfsdfsdf
woshi yanxu
dfsdfsdfsdf
sdgdfgdfgdfg
dfgdfgdfg- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
consumption data
./bin/kafka-console-consumer.sh --zookeeper 127.0.0.1:2181 --topic test --from-beginning
//内容
c
ls
sdfsdfsdf
sdfsdfsdfsdf
woshi yanxu
dfsdfsdfsdf
sdgdfgdfgdfg
dfgdfgdfg
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11