Kafka common mistakes finishing (continually updated)

1、UnknownTopicOrPartitionException

org.apache.kafka.common.errors.UnknownTopicOrPartitionException:
This server does not host this topic-partition

Error contents: partition the data is not

Analysis: producer sends a message to the topic does not exist, the user can check whether there is a topic or set parameters auto.create.topics.enable

2、LEADER_NOT_AVAILABLE

WARN Error while fetching metadata with correlation id 0 : {test=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient

Error content: leader unavailable

Analysis: many reasons topic being deleted ongoing leader election using the script checks leader kafka-topics Information

Check the broker's survival and then try to restart solve

3、NotLeaderForPartitionException

org.apache.kafka.common.errors.NotLeaderForPartitionException: This server is not the leader for that topic-partition

Error content: broker is not corresponding to the leader of the partition

Analysis: When the leader switching from one broker to another broker in the event of a change leader, to analyze what causes the switching of leader

4、TimeoutException

org.apache.kafka.common.errors.TimeoutException: Expiring 5 record(s) for test-0: 30040 ms has passe

Error contents: Request timed out

Analysis: Where to see if thrown observation network can pass through if you can consider increasing the value of request.timeout.ms

5、RecordTooLargeException

WARN async.DefaultEventHandler: Produce request with correlation id 92548048 failed due to [TopicName,1]: org.apache.kafka.common.errors.RecordTooLargeException

Content error: Message too large

Analysis: producer, but come the end message processing can increase request.timeout.ms reduce batch.size

6、Closing socket connection

Closing socket connection to/127,0,0,1.(kafka.network.Processor) 

Content error: Connection closed

Analysis: If javaApi producer of high version, low version verification want to start the client consumer, it will not stop error

The client does not recognize the message.

7 ConcurrentModificationException

java.util.ConcurrentModificationException: KafkaConsumer is not safe for multi-threaded access  

Error content: thread safe

Analysis: Kafka consumer non-thread-safe

8、NetWorkException

 [kafka-producer-network-thread | producer-1] o.apache.kafka.common.network.Selector : [Producer clientId=producer-1] Connection with / disconnected

Error: Network anomaly

Analysis: Check the network connection is interrupted network of broker

9、ILLEGAL_GENERATION

ILLEGAL_GENERATION occurred while committing offsets for group  

Content error: Invalid "generation"

Analysis: consumer missed rebalance because consumer spent a lot of time to process the data.

Need to ease max.poll.records value increases max.poll.interval.ms message or find ways to increase the processing speed

To be continued ~ ~ ~

More real-time calculation, Flink, Kafka and other related technologies Bowen, welcome attention to calculate real-time streaming