Background and stop kafka services
Note: the production environment recommended -daemonparameters backstage start kafka, then with the use nohupand&
If you do not want kafka-server-stop.sh to close all kafka broker on a machine, you can use another way, the first jpsview pid kafka, and then run ps ax | grep -i 'kafka/Kafka' | grep java | grep -v grep | awk '${print $1}'on their own to find PID Kafka finally run kill -s TERM $PIDclose broker.
Topic management / command line script
Creating topic
- Automatic partition allocation
bin/kafka-topics.sh --create --zookeeper localhost:2181 --partitions 6 --replication-factor 3
--topic topic-manager-test --config delete.retention.ms=259200000
The last parameter is the number of days log age 3 days
- Manual allocated partition
bin/kafka-topics.sh --create --zookeeper localhost:2181 --topic topic-manager-test2 --replica-assignment 0:1,1:2,0:2,1:2
Suppose there are three clusters broker Serial No. 0,1,2, four partitions, two copies designated by replica-assignment
Delete topic
bin/kafka-topics.sh --delete -zookeeper localhost:2181 --topic topic-manager-test
Topic topic-manager-test is marked for deletion. |
Need to set delete.topic.enable = true will really deleted, marked here only marked for deletion
Queries topic list
bin/kafka-topics.sh --zookeeper localhost:2181 --list
Queries topic details
bin/kafka-topics.sh --zookeeper localhost:2181 --describe --topic topic-manager-test2
Topic:topic-manager-test2 PartitionCount:4 ReplicationFactor:2 Configs: |
Use kafka-configs.sh script View
bin/kafka-configs.sh --describe --zookeeper localhost:2181 --entity-type topics --entity-name topic-manager-test2
See all topics details
bin/kafka-topics.sh --zookeeper localhost:2181 --describe
Modify toipc
kafka allows the user to certain parameters such as the number of partitions of the topic, and topic level replica factor parameter set
- Topic partitions increases from 4 to 10 partitions
bin/kafka-topics.sh --alter --zookeeper localhost:2181 --partitions 10 --topic topic-manager-test2
It does not support the reduction of partition
- Recommended
kafka-configs.shconfiguration, the following is added in the topic parametercleanup.policy=compactconfiguration
bin/kafka-configs.sh --zookeeper localhost:2181 --alter --entity-type topics
--entity-name topic-manager-test2 --add-config cleanup.policy=compact
- Confirm the parameters success
bin/kafka-configs.sh --zookeeper localhost:2181 --describe --entity-type topics --entity-name topic-manager-test2
Delete topic Configuration
bin/kafka-configs.sh --zookeeper localhost:2181 --alter --entity-type topics --entity-name topic-manager-test2 --delete-config prealloocate
appendix
- Common topic level parameters and meaning
| parameter name | Parameter Meaning |
|---|---|
| cleanup.policy | topic retention policy |
| compression.type | Compression type specified topic |
| max.message.bytes | topic message broker can receive the maximum length |
| min.insync.replicas | Specifies the number of broker ISR required minimum received message topic, and acks producer with the use of end = 1 |
| preallocate | Whether to allocate in advance for the topic log file storage space |
| retention.ms | The topic designated to hold the maximum time a single partition message |
| segment.bytes | Specifies the size of the log segment files topic |
| unclean.leader.election.enable | Whether to enable unclean elected topic |
Open method JMX port
For real-time monitoring of the health of kafka, need to open JMX port
Reception starts broker: JMX_PORT=9997 bin/kafka-server-start.sh config/server.properties
Background boot broker: export JMX_PORT=9997 bin/kafka-server-start.sh -daemon config/server.properties