Kafka learning --spring boot integrate kafka

First, start kafka

　　Before starting kafka must start zookeeper, because you want to use kafka must use zookeeper.

　　Start windows environment, directly kafka comes zookeeper:

　　E:\kafka_2.12-2.4.0\bin\windows  zookeeper-server-start.bat ..\..\config\zookeeper.properties

　　Then start kafka

　　E:\kafka_2.12-2.4.0\bin\windows  kafka-server-start.bat ..\..\config\server.properties

Two, spring boot integrate kafka Projects

1. Import maven

<dependency>
            <groupId>org.springframework.kafka</groupId>
            <artifactId>spring-kafka</artifactId>
        </dependency>

 Profiles:

80 = the server.port 

#kafka address, there may be a plurality of 
spring.kafka.bootstrap-Servers = 127.0.0.1: 9092 

# ------ --------- profile Manufacturer 

# Specify message kafka key body and the encoding format
spring.kafka.producer.key-serializer=org.apache.kafka.common.serialization.StringSerializer
Serializer-org.apache.kafka.common.serialization.StringSerializer = spring.kafka.producer.value 

# acks wait mechanism is provided to return, there are three values
# 0: Do not wait for acks returned (may lose data because there is no failure to send a message retry mechanism, but this is the lowest latency)
# 1: after the message to return kafka partition leader (if no synchronization is complete leader on the follower down, it will lose data)
# 1 (default): After waiting for all the synchronization message before sending follower (never lose data) 
spring.kafka.producer.acks = -1 


# message to the accumulated value of the batch-size, before sending a message, the default to 16384 
spring.kafka.producer.batch-size = 16384 

# If kafka delay in sending the message (in this case the message is not to accumulate a specified number), then after this time (unit: mm) begin sending 
spring.kafka. =. 1 producer.properties.linger.ms 

# set the buffer size, default is 33554432
# This buffer is kafka in two threads in the shared variables
# This is the main thread and two sender, main responsible for sending a message to the shared variable, sender pull data from the shared variables 
spring.kafka.producer.buffer-memory = 33554432 


number # failure retry transmission 
spring.kafka.producer.retries = 2 

# ------ --------- consumer profile 

# key and the message body specifies kafka encoding format
spring.kafka.consumer.key-serializer=org.apache.kafka.common.serialization.StringSerializer
= org.apache.kafka.common.serialization.StringSerializer the Serializer-spring.kafka.consumer.value 

# group_id designated consumer group 
spring.kafka.consumer.group-id = kafka_test 

when #kafka unplanned downtime, re-open the message consumer strategy, there are three strategies
#earliest: When offset has been submitted under the district, from start spending offset submitted; no offset when submitted, scratch consumption
#latest: When offset has been submitted under the district, from start spending offset submitted; when no offset submitted consumption data under the newly created partition,
#none: When all the partitions are present offset submitted, after the offset from the beginning of the consumer; as long as there is a partition offset submitted does not exist, an exception is thrown 
spring.kafka.consumer.auto-offset-the RESET Earliest = 


# Automatic submit offset 
spring.kafka.consumer.enable-Auto-the commit to true = 

# submitting offset interval 
spring.kafka.consumer.auto-commit-interval = 100 

theme # to listen to listen to the consumer does not exist, therefore being given off by default out of this 
spring.kafka.listener.missing-topics-fatal = false

 2. Create topic

import org.apache.kafka.clients.admin.NewTopic;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

/**
 * Use code to create the topic
 * Meaning three parameters: topic name; replication factor the number of partitions, the new theme; if a copy of the assignment is specified, or -1.
 */
@Configuration
public  class KafkaTopic {

     @Bean
    public NewTopic batchTopic() {
        return new NewTopic("testTopic", 8, (short) 1);
    }
}

3. Producer Code

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

/**
 * Kafka Producer Code
 */
@RestController
public class ProductorController {

    @Autowired
    private KafkaTemplate<String, String> kafkaTemplate;

    @RequestMapping("/test")
    public String show() {
        kafkaTemplate.send ( "testTopic", "Hello" );
         return "transmission success" ;
    }

}

4. Consumer Code

import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.springframework.context.annotation.Configuration;
import org.springframework.kafka.annotation.KafkaListener;

import java.util.Optional;

/**
 * Kafka Consumer Code
 */
@Configuration
public  class KafkaConsumer {
    
    @KafkaListener(topics = "testTopic")
    public void consumer(ConsumerRecord consumerRecord){
        Optional<Object> kafkaMassage = Optional.ofNullable(consumerRecord.value());
        if(kafkaMassage.isPresent()){
            Object o = kafkaMassage.get();
            System.out.println ( "the received message is:" + O);
        }

    }

}

Test Results: