ERROR Failed to send requests for topics

如果是在同一台机器(localhost),Kafka Client工作正常,Producer和Consumer都能正常发送和接收消息,但是一旦部署到两台机器,则默认配置的话不能正常工作。会出现“kafka.common.FailedToSendMessageException: Failed to send messages after 3 tries”的错误。
 

[2020-01-09 12:42:43,841] ERROR Failed to send requests for topics cache-message with correlation ids in [17,24] (kafka.producer.async.DefaultEventHandler)
[2020-01-09 12:42:43,841] ERROR Error in handling batch of 1 events (kafka.producer.async.ProducerSendThread)
kafka.common.FailedToSendMessageException: Failed to send messages after 3 tries.
        at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:90)
        at kafka.utils.Utils$.read(Utils.scala:376)
        at kafka.network.BoundedByteBufferReceive.readFrom(BoundedByteBufferReceive.scala:54)
        at kafka.network.Receive$class.readCompletely(Transmission.scala:56)
        at kafka.network.BoundedByteBufferReceive.readCompletely(BoundedByteBufferReceive.scala:29)
        at kafka.network.BlockingChannel.receive(BlockingChannel.scala:100)
        at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:74)
        at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:71)
        at kafka.producer.SyncProducer.send(SyncProducer.scala:112)
        at kafka.client.ClientUtils$.fetchTopicMetadata(ClientUtils.scala:53)
        at kafka.producer.BrokerPartitionInfo.updateInfo(BrokerPartitionInfo.scala:82)
        at kafka.producer.BrokerPartitionInfo.getBrokerPartitionInfo(BrokerPartitionInfo.scala:49)
        at kafka.producer.async.DefaultEventHandler.kafka$producer$async$DefaultEventHandler$$getPartitionListForTopic(DefaultEventHandler.scala:186)
        at kafka.producer.async.DefaultEventHandler$$anonfun$partitionAndCollate$1.apply(DefaultEventHandler.scala:150)
        at kafka.producer.async.DefaultEventHandler$$anonfun$partitionAndCollate$1.apply(DefaultEventHandler.scala:149)
        at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:60)
        at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
        at kafka.producer.async.DefaultEventHandler.partitionAndCollate(DefaultEventHandler.scala:149)
        at kafka.producer.async.DefaultEventHandler.dispatchSerializedData(DefaultEventHandler.scala:95)
        at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:72)
        at kafka.producer.async.ProducerSendThread.tryToHandle(ProducerSendThread.scala:104)
        at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:87)
        at kafka.producer.async.ProducerSendThread$$anonfun$processEvents$3.apply(ProducerSendThread.scala:67)
        at scala.collection.immutable.Stream.foreach(Stream.scala:526)
        at kafka.producer.async.ProducerSendThread.processEvents(ProducerSendThread.scala:66)
        at kafka.producer.async.ProducerSendThread.run(ProducerSendThread.scala:44)
[2020-01-09 12:42:43,500] WARN Fetching topic metadata with correlation id 19 for topics [Set(cache-message)] from broker [id:2,host:192.168.0.108,port:2181] failed (kafka.client.ClientUtils$)
java.io.EOFException: Received -1 when reading from channel, socket has likely been closed.
        at kafka.utils.Utils$.read(Utils.scala:376)

解决方法其实很简单,只需要在Kafka的配置文件server.properties中,设置好主机名即可:

# Hostname the broker will bind to. If not set, the server will bind to all interfaceshost.name=queue-server1

究其原因,其实从注释里,我们可以知道,这是一个指定broker的地址(严格来说是所监听的网络接口,或者网卡),同时,也可以看出它还和下面的属性相关。

# Hostname the broker will advertise to producers and consumers. If not set, it uses the

# value for “host.name” if configured. Otherwise, it will use the value returned from

# java.net.InetAddress.getCanonicalHostName().

#advertised.host.name=

也就是说,producer和consumer是通过这个主机名(advertised.host.name)来连接broker的,而如果这个值没有设置,则会使用上面的host.name的值,如果上面的host.name也没有设置,则会使用java.net.InetAddress.getCanonicalHostName()获取的值。

从Zookeeper中可以看出,默认的时候该broker是localhost,从其他机器访问当然不可能成功。

扫描二维码关注公众号,回复: 8759794 查看本文章


 
 

发布了41 篇原创文章 · 获赞 20 · 访问量 3万+

猜你喜欢

转载自blog.csdn.net/m0_37598953/article/details/103951936
今日推荐