【疑难杂症】kafka安装过程中遇到的坑

        在Windows系统安装kafka的过程中,真是举步维艰呀,现在记录一下所遇到的坑

问题1:运行kafka集成的zookeeper时,命令:bin\windows\zookeeper-server-start.bat config\zookeeper.properties,出现报错:Classpath is empty. Please build the project first e.g. by running 'gradlew jarAll'

原因:因为从官方下载的是kafka-3.0.0-src.tgz,这是源码应该下载正常的压缩包,即:kafka_2.13-3.0.0.tgz。也就是没有src的压缩包。


问题2:运行kafka集成的zookeeper时,命令:bin\windows\zookeeper-server-start.bat config\zookeeper.properties,出现报错:命令语法不正确。

原因:解压kafka的压缩包的时候,有多个相同压缩包名重复了,例如压缩包是kafka_2.13-3.0.0.tgz,解压后kafka_2.13-3.0.0/kafka_2.13-3.0.0/...,出现了两个kafka_2.13-3.0.0,把两个文件名任意一个换个名字即可解决,解决后运行如下图所示:


问题3:运行kafka集成的zookeeper时,命令:bin\windows\zookeeper-server-start.bat config\zookeeper.properties,出现:ZooKeeper audit is disabled. 

原因:因为kafka集成了zookeeper,在网上找的资料都说要把zkServer.cmd中加一行:"-Dzookeeper.audit.enable=true",即:

call %JAVA% "-Dzookeeper.audit.enable=true" "-Dzookeeper.log.dir=%ZOO_LOG_DIR%" "-Dzookeeper.root.logger=%ZOO_LOG4J_PROP%" "-Dzookeeper.log.file=%ZOO_LOG_FILE%" "-XX:+HeapDumpOnOutOfMemoryError" "-XX:OnOutOfMemoryError=cmd /c taskkill /pid %%%%p /t /f" -cp "%CLASSPATH%" %ZOOMAIN% "%ZOOCFG%" %*

 但是,这个是单独下载的zookeeper,好家伙,kafka集成的zookeeper压根就没有这个文件,这个问题怕是搞不定了,决定单独下载zookeeper进行解决。


问题4:基于问题3,单独下载了zookeeper,问题又来啦。下载的zookeeper是:apache-zookeeper-3.7.0.tar.gz,点击./apache-zookeeper-3.7.0/bin/zkServer.cmd进行运行,直接闪退,都不知道咋了。那就尝试在cmd命令运行,错误如下:

错误: 找不到或无法加载主类 org.apache.zookeeper.server.quorum.QuorumPeerMain
原因: java.lang.ClassNotFoundException: org.apache.zookeeper.server.quorum.QuorumPeerMain

原因:因为我下载的是未编译的 jar 包,即:apache-zookeeper-3.7.0.tar.gz,这种命名的就是未编译的,需要重新下载,即:apache-zookeeper-3.7.0-bin.tar.gz,这种带bin命名的即可。


问题5:运行zookeeper中zkServer.cmd,出现:ZooKeeper audit is disabled.

 原因:zkServer.cmd中加一行:"-Dzookeeper.audit.enable=true",即:

call %JAVA% "-Dzookeeper.audit.enable=true" "-Dzookeeper.log.dir=%ZOO_LOG_DIR%" "-Dzookeeper.root.logger=%ZOO_LOG4J_PROP%" "-Dzookeeper.log.file=%ZOO_LOG_FILE%" "-XX:+HeapDumpOnOutOfMemoryError" "-XX:OnOutOfMemoryError=cmd /c taskkill /pid %%%%p /t /f" -cp "%CLASSPATH%" %ZOOMAIN% "%ZOOCFG%" %*


问题6:通过命令运行bin/zkServer.cmd,报错信息如下所示:

ERROR [main:ZooKeeperServerMain@86] - Unable to start AdminServer, exiting abnormally
org.apache.zookeeper.server.admin.AdminServer$AdminServerException: Problem starting AdminServer on address 0.0.0.0, port 8080 and command URL /commands

ERROR [main:ZooKeeperServerMain@86] - Unable to start AdminServer, exiting abnormally
org.apache.zookeeper.server.admin.AdminServer$AdminServerException: Problem starting AdminServer on address 0.0.0.0, port 8080 and command URL /commands

原因:运行方式不对,运行zkServer.cmd,不能在cmd中通过命令运行,直接双击运行zkServer.cmd即可。


问题7:zookeeper也安装了,那么接下来就是运行kafka了,这个时候不需要在运行kafka中的集成zookeeper,其运行如下:

1)要先启动zookeeper,双击bin/zkServer.cmd

2) 然后再启动kafka,启动命令为./kafka-server-start.bat ../config/server.properties

问题来了,找到cmd中的第一个错误,即:

ERROR Failed to write meta.properties due to (kafka.server.BrokerMetadataCheckpoint)
java.nio.file.AccessDeniedException: D:\tmp\kafka-logs
        at java.base/sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:89)
        at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:103)
        at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:108)
        at java.base/sun.nio.fs.WindowsFileSystemProvider.newFileChannel(WindowsFileSystemProvider.java:121)
        at java.base/java.nio.channels.FileChannel.open(FileChannel.java:298)
        at java.base/java.nio.channels.FileChannel.open(FileChannel.java:357)
        at org.apache.kafka.common.utils.Utils.flushDir(Utils.java:953)
        at org.apache.kafka.common.utils.Utils.atomicMoveWithFallback(Utils.java:941)
        at kafka.server.BrokerMetadataCheckpoint.liftedTree1$1(BrokerMetadataCheckpoint.scala:214)
        at kafka.server.BrokerMetadataCheckpoint.write(BrokerMetadataCheckpoint.scala:204)
        at kafka.server.KafkaServer.$anonfun$checkpointBrokerMetadata$2(KafkaServer.scala:772)
        at kafka.server.KafkaServer.$anonfun$checkpointBrokerMetadata$2$adapted(KafkaServer.scala:770)
        at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:563)
        at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:561)
        at scala.collection.AbstractIterable.foreach(Iterable.scala:919)
        at scala.collection.IterableOps$WithFilter.foreach(Iterable.scala:889)
        at kafka.server.KafkaServer.checkpointBrokerMetadata(KafkaServer.scala:770)
        at kafka.server.KafkaServer.startup(KafkaServer.scala:322)
        at kafka.Kafka$.main(Kafka.scala:109)
        at kafka.Kafka.main(Kafka.scala)

原因:因为安装的kafka版本是kafka_2.13-3.0.0.tgz,版本太高了,重新安装版本kafka_2.12-2.8.1.tgz,问题得以解决,启动后又如下日志,则表示成功。

[2021-12-19 21:52:37,314] INFO Kafka version: 2.8.1 (org.apache.kafka.common.utils.AppInfoParser)
[2021-12-19 21:52:37,316] INFO Kafka commitId: 839b886f9b732b15 (org.apache.kafka.common.utils.AppInfoParser)
[2021-12-19 21:52:37,318] INFO Kafka startTimeMs: 1639921957299 (org.apache.kafka.common.utils.AppInfoParser)
[2021-12-19 21:52:37,335] INFO [KafkaServer id=0] started (kafka.server.KafkaServer)
[2021-12-19 21:52:37,391] INFO [broker-0-to-controller-send-thread]: Recorded new controller, from now on will use broker NR068VV24TK9IP6:9092 (id: 0 rack: null) (kafka.server.BrokerToControllerRequestThread)

猜你喜欢

转载自blog.csdn.net/qq_33206497/article/details/122029671