Lison <[email protected]>, v1.0.3, 2019.03.27
摘要
解决和排查kafka安装过程中，由于kafka安装版本所使用zookeeper版本与本地zookeeper安装版本不兼容，导致kafka无法连接zookeeper的原因而导致kafka无法启动的问题
文章目录
问题描述和错误展现方式
启动kafka时，kafka一直无法连接到zookeeper，控制台日志出现以下类似的错误
[2019-03-25 15:43:57,740] INFO Opening socket connection to server cundreamc7001/192.168.237.33:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2019-03-25 15:43:57,741] INFO Socket connection established to cundreamc7001/192.168.237.33:2181, initiating session (org.apache.zookeeper.ClientCnxn)
[2019-03-25 15:43:57,748] WARN Connected to an old server; r-o mode will be unavailable (org.apache.zookeeper.ClientCnxnSocket)
[2019-03-25 15:43:57,748] INFO Session establishment complete on server cundreamc7001/192.168.237.33:2181, sessionid = 0x169b3cac2f50002, negotiated timeout = 6000 (org.apache.zookeeper.ClientCnxn)
[2019-03-25 15:43:57,752] WARN Session 0x169b3cac2f50002 for server cundreamc7001/192.168.237.33:2181, unexpected error, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn)
java.io.IOException: Xid out of order. Got Xid 50 with err 0 expected Xid 49 for a packet with details: clientPath:null serverPath:null finished:false header:: 49,14 replyHeader:: 0,0,-4 request:: org.apache.zookeeper.MultiTransactionRecord@fabc5c55 response:: org.apache.zookeeper.MultiResponse@0
at org.apache.zookeeper.ClientCnxn$SendThread.readResponse(ClientCnxn.java:830)
at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:94)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:366)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1145)
问题排查思路和解决步骤
排查思路
遇到上述Kafka启动出现问题时，我们则可以确定是kafka连接zookeeper时出了问题。
所以怀疑出现这样连接问题的原因有以下两种情况
- 本地所使用zookeeper是否已经正常启动和正常连接
- Kafka安装包自带的zookeeper是否能够正常启动
确定本地zookeeper能够正常启动和正常连接
首先打开zookeeper所在目录使用以下命令启动zookeeper
bin/zkServer.sh start
ZooKeeper使用log4j记录消息,会在启动目录生成zookeeper.out 的日志文件
执行玩该命令后 可使用jps命令，会出现如下进程
QuorumPeerMain
然后尝试连接zookeeper
$ bin/zkCli.sh -server 127.0.0.1:2181
这使您可以执行简单的文件操作。连接后，您应该看到类似的内容：
Connecting to localhost:2181
log4j:WARN No appenders could be found for logger (org.apache.zookeeper.ZooKeeper).
log4j:WARN Please initialize the log4j system properly.
Welcome to ZooKeeper!
JLine support is enabled
[zkshell: 0]
确定Kafka能够正常启动
使用kafka一起打包的zookeeper便捷快速启动单节点zookeeper
首先打开kafka所在目录使用以下命令启动kafka自带zookeeper本地节点
bin/zookeeper-server-start.sh config/zookeeper.properties
[2019-03-25 16:08:57,944] INFO Reading configuration from: config/zookeeper.properties (org.apache.zookeeper.server.quorum.QuorumPeerConfig)
...
zookeeper启动成功后，现在启动Kafka服务
bin/kafka-server-start.sh config/server.properties
[2019-03-25 16:29:01,562] INFO Verifying properties (kafka.utils.VerifiableProperties)
[2019-03-25 16:29:01,562] INFO Property socket.send.buffer.bytes is overridden to 1048576 (kafka.utils.VerifiableProperties)
...
[2019-03-25 16:09:02,070] INFO [KafkaServer id=0] started (kafka.server.KafkaServer)
若日志出现 INFO [KafkaServer id=0] started 则表示kafka启动成功。
问题总结
如果Kafka安装包和本地zookeeper都能够正常的启动，所以我们基本可以确定是因为我们所使用的kafka和zookeeper的版本出现了冲突或兼容问题
解决步骤
更换kafka版本
首先查看当前安装的zookeeper版本
使用以下命令查看zookeeper版本
echo stat|nc 127.0.0.1 2181
Zookeeper version: 3.4.13-2d71af4dbe22557fda74f9a9b4309b15a7487f03, built on 06/29/2018 04:05 GMT
Clients:
/192.168.237.33:60380[1](queued=0,recved=236,sent=237)
/127.0.0.1:33288[0](queued=0,recved=1,sent=0)
Latency min/avg/max: 0/1/38
Received: 308
Sent: 312
Connections: 2
Outstanding: 0
Zxid: 0x31
Mode: standalone
Node count: 25
### 如未安装nc,请用root用户执行以下命令
### yum -y install nc
然后下载对应支持的kafka版本
更换zookeeper版本
首先查看kafka支持zookeeper版本，由于kafka没有像别的软件一样有，kafka -version命令，但是可以以查看kafka/libs下的库文件，知道kafka的版本号和zookeeper支持的版本,然后下载对应的zookeeper版本进行安装；