一、前言
严格来讲，kafka删除topic并不是很难，但是步骤比较麻烦，稍有不慎就会失败。本文就kafka删除topic的步骤做出详细解析。
二、详细步骤
①停止kafka服务
停止kafka的所有的消费者和生产者，关闭kafka，防止在删除topic之后，程序发现topic不存在而自动创建topic
②修改配置文件
在kafka安装目录的conf目录下，修改server.properties文件，在文件的末尾添加如下配置：
[root@hadoop01 config]# pwd
/home/software/kafka/config
[root@hadoop01 config]# vim server.properties
[root@hadoop01 config]# |
auto.create.topics.enable = false
delete.topic.enable=true |
这一步配置很重要。
auto.create.topics.enable = false。这个配置的默认值为true，此时topic不存在时会自动创建，这样会给删除topic带来意想不到的问题。
delete.topic.enable=true。不设置这个参数，则topic无法删除。
kafka集群中所有的机器都要执行这一步操作。
③删除topic
进入kafka安装目录的bin目录，执行指令：./kafka-topics --delete --zookeeper 【zookeeper server:port】 --topic 【topic name】
[root@hadoop01 bin]# pwd
/home/software/kafka/bin
[root@hadoop01 bin]# ./kafka-topics.sh --delete --zookeeper hadoop01:2181 --topic park
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/software/hbase/lib/phoenix-4.8.1-HBase-0.98-client.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/software/kafka/libs/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
Topic park is marked for deletion.
Note: This will have no impact if delete.topic.enable is not set to true.
[root@hadoop01 bin]# |
注意，这一步执行完毕，topic可能并没有真正删除，只是被标记为删除：“Topic park is marked for deletion.”
④删除kafka存储目录
topic相关的存储目录为在server.properties文件log.dirs配置的目录（本案例中为：/home/software/kafka/tmp）
| [root@hadoop01 tmp]# pwd
/home/software/kafka/tmp [root@hadoop01 tmp]# ll
total 4
drwxr-xr-x 26 root root 4096 Dec 11 17:08 kafka-logs [root@hadoop01 tmp]# cd kafka-logs/
[root@hadoop01 kafka-logs]# ll
total 112
-rw-r--r-- 1 root root 0 Nov 28 08:55 cleaner-offset-checkpoint
drwxr-xr-x 2 root root 4096 Dec 11 09:02 __consumer_offsets-5
drwxr-xr-x 2 root root 4096 Dec 11 09:02 __consumer_offsets-8
drwxr-xr-x 2 root root 4096 Dec 11 16:13 park
-rw-r--r-- 1 root root 4 Dec 11 17:08 log-start-offset-checkpoint
-rw-r--r-- 1 root root 54 Nov 28 08:55 meta.properties
-rw-r--r-- 1 root root 548 Dec 11 17:08 recovery-point-offset-checkpoint
-rw-r--r-- 1 root root 553 Dec 11 17:09 replication-offset-checkpoint
[root@hadoop01 kafka-logs]# |
删除跟topic相关的目录
| [root@hadoop01 kafka-logs]# rm -rf park |
注意：kafka集群中的每台机器都需要执行这一步的操作
⑤删除zookeeper服务器上的topic
在其中一台zookeeper服务器上，进入zookeeper安装目录的bin目录，登录到shell
[root@hadoop01 bin]# pwd
/home/software/zookeeper/bin [root@hadoop01 bin]# ./zkCli.sh -server hadoop01:2181
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/software/zookeeper/lib/slf4j-log4j12-1.6.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/software/hbase/lib/phoenix-4.8.1-HBase-0.98-client.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
Connecting to hadoop01:2181
2018-12-11 17:16:40,319 [myid:] - INFO [main:Environment@100] - Client environment:zookeeper.version=3.4.7-1713338, built on 11/09/2015 04:32 GMT
2018-12-11 17:16:40,323 [myid:] - INFO [main:Environment@100] - Client environment:host.name=hadoop01
2018-12-11 17:16:40,323 [myid:] - INFO [main:Environment@100] - Client environment:java.version=1.8.0_131
2018-12-11 17:16:40,328 [myid:] - INFO [main:Environment@100] - Client environment:java.vendor=Oracle Corporation
2018-12-11 17:16:40,330 [myid:] - INFO [main:Environment@100] - Client environment:java.home=/home/software/jdk1.8/jre
2018-12-11 17:16:40,330 [myid:] - INFO [main:Environment@100] - Client environment:java.class.path=/home/software/zookeeper/bin/../build/classes:/home/software/zookeeper/bin/../build/lib/*.jar:/home/software/zookeeper/bin/../lib/slf4j-log4j12-1.6.1.jar:/home/software/zookeeper/bin/../lib/slf4j-api-1.6.1.jar:/home/software/zookeeper/bin/../lib/netty-3.7.0.Final.jar:/home/software/zookeeper/bin/../lib/log4j-1.2.16.jar:/home/software/zookeeper/bin/../lib/jline-0.9.94.jar:/home/software/zookeeper/bin/../zookeeper-3.4.7.jar:/home/software/zookeeper/bin/../src/java/lib/*.jar:/home/software/zookeeper/bin/../conf:.:/home/software/jdk1.8/lib/dt.jar:/home/software/jdk1.8/lib/tools.jar:/home/software/hbase/lib/phoenix-4.8.1-HBase-0.98-client.jar:/home/software/hbase/lib/phoenix-4.8.1-HBase-0.98-server.jar
2018-12-11 17:16:40,330 [myid:] - INFO [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2018-12-11 17:16:40,330 [myid:] - INFO [main:Environment@100] - Client environment:java.io.tmpdir=/tmp
2018-12-11 17:16:40,330 [myid:] - INFO [main:Environment@100] - Client environment:java.compiler=<NA>
2018-12-11 17:16:40,330 [myid:] - INFO [main:Environment@100] - Client environment:os.name=Linux
2018-12-11 17:16:40,330 [myid:] - INFO [main:Environment@100] - Client environment:os.arch=amd64
2018-12-11 17:16:40,331 [myid:] - INFO [main:Environment@100] - Client environment:os.version=2.6.32-431.el6.x86_64
2018-12-11 17:16:40,332 [myid:] - INFO [main:Environment@100] - Client environment:user.name=root
2018-12-11 17:16:40,332 [myid:] - INFO [main:Environment@100] - Client environment:user.home=/root
2018-12-11 17:16:40,332 [myid:] - INFO [main:Environment@100] - Client environment:user.dir=/home/software/zookeeper/bin
2018-12-11 17:16:40,335 [myid:] - INFO [main:ZooKeeper@438] - Initiating client connection, connectString=hadoop01:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@2cdf8d8a
Welcome to ZooKeeper!
2018-12-11 17:16:40,387 [myid:] - INFO [main-SendThread(hadoop01:2181):ClientCnxn$SendThread@1032] - Opening socket connection to server hadoop01/192.168.205.141:2181. Will not attempt to authenticate using SASL (unknown error)
JLine support is enabled
2018-12-11 17:16:40,537 [myid:] - INFO [main-SendThread(hadoop01:2181):ClientCnxn$SendThread@876] - Socket connection established to hadoop01/192.168.205.141:2181, initiating session
2018-12-11 17:16:40,556 [myid:] - INFO [main-SendThread(hadoop01:2181):ClientCnxn$SendThread@1299] - Session establishment complete on server hadoop01/192.168.205.141:2181, sessionid = 0x1679abd61be0004, negotiated timeout = 30000 WATCHER:: WatchedEvent state:SyncConnected type:None path:null
[zk: hadoop01:2181(CONNECTED) 0] |
查看所有的topic
[zk: hadoop01:2181(CONNECTED) 0] ls /brokers/topics
[park]
[zk: hadoop01:2181(CONNECTED) 1] |
执行删除指令，删除topic
| [zk: hadoop01:2181(CONNECTED) 1] rmr /brokers/topics/park |
此时，topic被彻底删除。
如果topic还存在，或者仍然被标记为“Topic park is marked for deletion.”，则通过指令ls /admin/delete_topics，找到需要删除的topic，然后执行指令rmr /admin/delete_topics/【topic name】
至此，topic被彻底删除！