ERROR Error when sending message to topic test_topic with key: null, value: 11 bytes with error:

problem:

There is an error in producing data to the kafka broker by abbreviated hostname, full hostname, and host IP. The specific errors are as follows:

Error 1

[root@localhost ]# ./bin/kafka-console-producer.sh --broker-list node01:9092 --topic test_topic
[2020-12-1 16:25:33,972] WARN Removing server node01:9092 from bootstrap.servers as DNS resolution failed for node01 (org.apache.kafka.clients.ClientUtils)
org.apache.kafka.common.KafkaException: Failed to construct kafka producer
   at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:415)
   at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:287)
   at kafka.producer.NewShinyProducer.<init>(BaseProducer.scala:40)
   at kafka.tools.ConsoleProducer$.main(ConsoleProducer.scala:48)
   at kafka.tools.ConsoleProducer.main(ConsoleProducer.scala)
Caused by: org.apache.kafka.common.config.ConfigException: No resolvable bootstrap urls given in bootstrap.servers
   at org.apache.kafka.clients.ClientUtils.parseAndValidateAddresses(ClientUtils.java:64)
   at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:372)
   ... 4 more

Error 2

[root@localhost ]# ./bin/kafka-console-producer.sh --broker-list node01.example.com:9092 --topic test_topic
[2020-12-1 16:27:59,444] WARN Removing server node01.example.com:9092 from bootstrap.servers as DNS resolution failed for node01.example.com (org.apache.kafka.clients.ClientUtils)
org.apache.kafka.common.KafkaException: Failed to construct kafka producer
   at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:415)
   at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:287)
   at kafka.producer.NewShinyProducer.<init>(BaseProducer.scala:40)
   at kafka.tools.ConsoleProducer$.main(ConsoleProducer.scala:48)
   at kafka.tools.ConsoleProducer.main(ConsoleProducer.scala)
Caused by: org.apache.kafka.common.config.ConfigException: No resolvable bootstrap urls given in bootstrap.servers
   at org.apache.kafka.clients.ClientUtils.parseAndValidateAddresses(ClientUtils.java:64)
   at org.apache.kafka.clients.producer.KafkaProducer.<init>(KafkaProducer.java:372)
   ... 4 more

Error 3

[root@localhost ]# ./bin/kafka-console-producer.sh --broker-list 192.168.1.1:9092 --topic test_topic
>hello world
>[2020-12-1 16:12:24,583] ERROR Error when sending message to topic test_topic with key: null, value: 11 bytes with error: (org.apache.kafka.clients.producer.internals.ErrorLoggingCallback)
org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for test_topic-0: 40094 ms has passed since batch creation plus linger time

the reason:

This problem is mainly because the Kafka client cannot connect to the broker node of Kafka.

Error 1 and error 2 are because the machine cannot resolve the abbreviated host name and full host name. You need to configure the /etc/hosts file and add the IP address corresponding to the abbreviated host name and full host name to the hosts file.

Error 3 is because the IP and host name of the kafka broker node are not added to the /etc/hosts file, and the IP address corresponding to the host name of the kafka broker needs to be added to the hosts file.

In fact, the reason for the above problem is because the kafka client requests kafka, and kafka returns to the client the host name instead of the IP. If the client host is not equipped with hosts, the host name cannot be resolved and the upload message fails.

solve:

Method 1: Modify Client host hosts

Open the /etc/hosts file and add the following content:

192.168.1.1  node01 node01.example.com
192.168.1.2  node02 node02.example.com
192.168.1.3  node03 node03.example.com

Method 2: Modify Kafka configuration

Modify the server.properties file and set the advertised.host.name field to the ip of the broker

advertised.host.name= 192.168.0.1，192.168.0.2，192.168.0.3

note:

The host name in hosts must include the host name of the Kafka broker, here is node01.example.com, node02.example.com, node03.example.com, otherwise Kafka cannot be used normally.

reference:

1.https://www.cnblogs.com/snifferhu/p/5102629.html

2.https://blog.csdn.net/u013686990/article/details/103026842