Spring Kafka listener using DeadLetterPublishingRecoverer and manual ack mode

xstring :

I'm having a hard time to understand how this could be solved so I'm asking it here in the hope someone else already faced the same problems. We are running our @KafkaListener with manual ack mode and a dead letter recoverer with a retry limit of 3. Manual ack mode is needed due to the business logic that we dont ack a message and pause consuming for 5 minutes when certain circumstances are given (external dependencies).

Also we do need the dead letter queue for messages we cannot process for some reason.

Now the problem in manual ack mode is that our listener/consumer does not acknowledge the message when he hits the retry limit and moves it to the dl queue.

If the consumer service will be restarted he tries to consume the messages again and again will move them to the dl queue.

Any ideas how we could solve this issue?

Thanks and greetings from Hamburg!

Gary Russell :

I would try to avoid using manual acks if possible; perhaps by increasing max.poll.interval.ms instead.

If you use AckMode.MANUAL_IMMEDIATE, it will be safe to perform the commit directly on the Consumer in the error handler.

Subclass the SeekToCurrentErrorHandler and override handle(), If super.handle() doesn't throw an exception, it means the retries are exceeded and you can commit the offset on the Consumer.