【弄nèng - Kafka】应用篇（二） —— Springboot整合Kafka（Java Bean消息）

该篇博客实现Springboot整合kafka ,发送Java Bean消息
更多高级用法请看下篇博客

一. 简介

kafka概念相关的介绍请看官方文档和其他博文
官方中文文档
kafka入门介绍

二. 发送Java Bean消息

2.1 引入依赖

主要是spring-kafka依赖

 <dependencies>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-web</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.kafka</groupId>
            <artifactId>spring-kafka</artifactId>
        </dependency>
        <dependency>
            <groupId>org.springframework.boot</groupId>
            <artifactId>spring-boot-starter-test</artifactId>
            <scope>test</scope>
        </dependency>
        <dependency>
            <groupId>org.springframework.kafka</groupId>
            <artifactId>spring-kafka-test</artifactId>
            <scope>test</scope>
        </dependency>
        
        <!--优化编码-->
        <dependency>
            <groupId>org.projectlombok</groupId>
            <artifactId>lombok</artifactId>
            <optional>true</optional>
        </dependency>
    </dependencies>

application.properties 添加变量参数

设置配置参数，主题，topic等

kafka.bootstrap-servers=localhost:9092

kafka.topic.basic=test_topic
kafka.topic.json=json_topic
kafka.topic.batch=batch_topic
kafka.topic.manual=manual_topic

kafka.topic.transactional=transactional_topic
kafka.topic.reply=reply_topic
kafka.topic.reply.to=reply_to_topic
kafka.topic.filter=filter_topic
kafka.topic.error=error_topic

server.port=9093

2.2 Java Bean

定义一个对象

public class JsonBean {
    private String messageId;
    private String messageContent;
    public JsonBean(){}

    public JsonBean(String messageId , String messageContent){
        this.messageId = messageId;
        this.messageContent = messageContent;
    }

    public String getMessageContent() {
        return messageContent;
    }

    public void setMessageContent(String messageContent) {
        this.messageContent = messageContent;
    }

    public String getMessageId() {
        return messageId;
    }

    public void setMessageId(String messageId) {
        this.messageId = messageId;
    }

    public String toString() {
        return String.format("%s:%s", messageId, messageContent);
    }
}

2.3 Kafka配置

此处我们可以在application.properties中配置，也可以使用Java Config。我使用Java Config，看得更直观。

2.3.1 生产者

配置类 JsonProducerConfig.java

import org.apache.kafka.clients.producer.ProducerConfig;
import org.apache.kafka.common.serialization.StringSerializer;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.kafka.core.DefaultKafkaProducerFactory;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.kafka.support.serializer.JsonSerializer;
import java.util.HashMap;
import java.util.Map;

/**
 * 司马缸砸缸了
 * 生产者配置
 */
@Configuration
public class JsonProducerConfig {
    @Value("${kafka.bootstrap-servers}")
    private String bootstrapServers;


    @Bean
    public KafkaTemplate<String, JsonBean> jsonKafkaTemplate() {
        Map<String, Object> configProps = new HashMap<>();
        configProps.put(ProducerConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapServers);
        configProps.put(ProducerConfig.KEY_SERIALIZER_CLASS_CONFIG, StringSerializer.class);
        // value使用Json序列化
        configProps.put(ProducerConfig.VALUE_SERIALIZER_CLASS_CONFIG, JsonSerializer.class);

        return new KafkaTemplate<>(new DefaultKafkaProducerFactory<>(configProps));
    }
}

注释已经很详细了，详细参数请参考：
http://www.shixinke.com/java/kafka-configuration
https://blog.csdn.net/xiaozhu_you/article/details/91493258

生产者 JsonProducer.java

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.kafka.core.KafkaTemplate;
import org.springframework.stereotype.Component;


@Component
public class JsonProducer {
    @Autowired
    @Qualifier("jsonKafkaTemplate")
    private KafkaTemplate<String, JsonBean> kafkaTemplate;

    @Value("${kafka.topic.json}")
    private String topic;

    public void send(JsonBean json) {
        kafkaTemplate.send(topic, json);
    }

}

2.3.2 消费者

配置类 JsonConsumerConfig.java

import org.apache.kafka.clients.consumer.ConsumerConfig;
import org.apache.kafka.common.serialization.StringDeserializer;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.kafka.annotation.EnableKafka;
import org.springframework.kafka.config.ConcurrentKafkaListenerContainerFactory;
import org.springframework.kafka.core.ConsumerFactory;
import org.springframework.kafka.core.DefaultKafkaConsumerFactory;
import org.springframework.kafka.support.serializer.JsonDeserializer;
import java.util.HashMap;
import java.util.Map;


@Configuration
@EnableKafka
public class JsonConsumerConfig {
    @Value("${kafka.bootstrap-servers}")
    private String bootstrapServers;

    @Value("${kafka.topic.basic}")
    private String topic;

    @Bean
    public ConcurrentKafkaListenerContainerFactory<String, JsonBean> basicKafkaListenerContainerFactory() {
        Map<String, Object> configProps = new HashMap<>();
        configProps.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, bootstrapServers);
        configProps.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, StringDeserializer.class);
        // json反序列化
        configProps.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, JsonDeserializer.class);
        configProps.put(ConsumerConfig.GROUP_ID_CONFIG, topic);

        ConcurrentKafkaListenerContainerFactory<String, JsonBean> factory = new ConcurrentKafkaListenerContainerFactory<>();
        ConsumerFactory<String, JsonBean> basicConsumerFactory =
                new DefaultKafkaConsumerFactory<>(configProps,
                        new StringDeserializer(),
                        new JsonDeserializer<>(JsonBean.class));
        factory.setConsumerFactory(basicConsumerFactory);

        return factory;
    }
}

消费者 JsonConsumer.java

import org.springframework.kafka.annotation.KafkaListener;
import org.springframework.stereotype.Component;


@Component
public class JsonConsumer {

    @KafkaListener(topics = "${kafka.topic.json}", containerFactory = "basicKafkaListenerContainerFactory")
    public void receive(JsonBean basic) {
        System.out.println("receive:" + basic.toString());
    }
}

使用多种方式@KafkaListener进行消费，参考博客：https://www.jianshu.com/p/a64defb44a23

2.3.3 测试

运行

	@Autowired
    private JsonProducer jsonProducer;

	@Test
    public void jsonProducer() {
        JsonBean json = new JsonBean();
        json.setMessageId("1");
        json.setMessageContent("this is message");
        jsonProducer.send(json);

        try {
            Thread.sleep(1000 * 2);
        } catch (InterruptedException e) {
            e.printStackTrace();
        }
    }

结果

源码地址

IT-CLOUD-KAFKA :spring整合kafka教程源码。博文在本CSDN kafka系列中。

---

项目推荐

IT-CLOUD :IT服务管理平台，集成基础服务，中间件服务，监控告警服务等。
IT-CLOUD-ACTIVITI6 :Activiti教程源码。博文在本CSDN Activiti系列中。
IT-CLOUD-ELASTICSEARCH :elasticsearch教程源码。博文在本CSDN elasticsearch系列中。
IT-CLOUD-KAFKA :spring整合kafka教程源码。博文在本CSDN kafka系列中。
IT-CLOUD-KAFKA-CLIENT :kafka client教程源码。博文在本CSDN kafka系列中。

开源项目，持续更新中，喜欢请 Star~