'org.springframework.kafka.core.ConsumerFactory' that could not be found

Hex Clan :

I need to consume a json object Message from Spring Boot application using Apache Kafka with org.springframework.kafka dependency, but I got the following error:

***************************
APPLICATION FAILED TO START
***************************

Description:

Parameter 1 of method kafkaListenerContainerFactory in org.springframework.boot.autoconfigure.kafka.KafkaAnnotationDrivenConfiguration required a bean of type 'org.springframework.kafka.core.ConsumerFactory' that could not be found.

This is my bean class which has all the required beans created for Kafka configuration.

@Configuration
@EnableKafka
public class KafkaConsumerConfig {

    @Bean
    public ConsumerFactory<String,Message> consumerFactory(){
        Map<String,Object> config = new HashMap<>();
        config.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG,"127.0.0.1:9092");
        config.put(ConsumerConfig.GROUP_ID_CONFIG,"sample-group");
        config.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringSerializer.class);
        config.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, JsonSerializer.class);
        return new DefaultKafkaConsumerFactory<>(config,new StringDeserializer(),
                new JsonDeserializer<>(Message.class));
    }

    @Bean
    public ConcurrentKafkaListenerContainerFactory<String, Message> kafkaListener(){
        ConcurrentKafkaListenerContainerFactory factory = new ConcurrentKafkaListenerContainerFactory();
        factory.setConsumerFactory(consumerFactory());
        return factory;
    }

}

Can anyone help me? What am I doing wrong? Thanks.

Didier Roque Ginebra :

You can use application.properties/application.yml to achive that.
Please notice that Spring Boot allows us to avoid all the boilerplate code that we introduce creating a Java class annotated with @Configuration
application.yml example:

server:
  port: 8085

#Kafka config props:
spring:
  kafka:
    bootstrap-servers: 127.0.0.1:9092
    #Consumer Deserialization:
    consumer:
      key-deserializer: org.apache.kafka.common.serialization.StringDeserializer
      value-deserializer: org.apache.kafka.common.serialization.StringDeserializer
      group-id: myGroupId
      enable-auto-commit: false
    listener:
      missing-topics-fatal: false

bootstrap-servers: List of available Kafka servers.

key-deserializer: Consumer key de-serialization class. to use the StringDeserializer class of Kafka library.

value-deserializer: Consumer value de-serialization class. to use the StringDeserializer class of Kafka library as you are consuming JSON formatted string messages.

group-id: A group id value for the Kafka consumer.

enable-auto-commit: Setting this value to false you can commit the offset messages manually, which avoids crashing of the consumer if new messages are consumed when the currently consumed message is being processed by the consumer.

listener.missing-topics-fatal: By setting the value to false, you can avoid unwanted errors that are displayed during application startup if any of the configured topics are not present on the broker.

Hope this helps.