利用Kafka生产消费消息实例

版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。

本文链接： https://blog.csdn.net/weixin_43707160/article/details/100574582

1. 配置相关依赖

ps: 注意artifactId配置项中，kafka_后跟的版本号必须与本机装的kafka对应的scala的版本一致

2. 编码

生产者
该段代码学习
下篇博客中的代码，但本文中init方法是根据官方文档写的
https://blog.csdn.net/konglongaa/article/details/81208946

import org.apache.kafka.clients.producer.*;
import java.util.Properties;
import java.util.concurrent.Executors;

public class Producer1 {
    private Producer<String,String> producer;

    public static void main(String[] args){
        new Producer1().start();
    }
    public void init(){
        Properties kafkaProps = new Properties();    //创建一个生产者
        kafkaProps.put("bootstrap.servers","192.168.235.128:9092");

        //消息序列化,将键和值定义成字符串类，将value和key都序列化为字节数组
        kafkaProps.put("key.serializer","org.apache.kafka.common.serialization.StringSerializer");
        kafkaProps.put("value.serializer","org.apache.kafka.common.serialization.StringSerializer");
       //acks=all是最安全的模式
        kafkaProps.put("acks","all");

        //该配置负责producer发送消息后是否等待broker的ack,默认是0不等待
        //kafkaProps.put("request.required.acks","2");

        producer = new KafkaProducer<String,String>(kafkaProps);
    }
    public void produceMsg(){
        long timestamp = System.currentTimeMillis();
        String msg = "Msseage" + timestamp;
        String topic = "kafka_test";
        System.out.println("发送消息" + msg);
        String key = "Msseage-Key" + timestamp;

        ProducerRecord<String,String> data = new ProducerRecord<>(topic,key,msg);
        try{
            producer.send(data).get();
        }catch (Exception e){
            e.printStackTrace();
        }
    }

    public void start(){
        System.out.println("开始发送新消息：");
        Executors.newSingleThreadExecutor().execute(new Runnable() {
            @Override
            public void run() {
                init();
                while(true){
                    try{
                        produceMsg();
                        Thread.sleep(3000);
                    }catch (Throwable e){
                        if(producer != null){
                            try{
                                producer.close();
                            }catch (Throwable e1){
                                System.out.println("Turn off kafka producer error!" );
                            }
                        }
                    }
                }
            }
        });
    }
}

上段代码执行后，每次发送消息后，都会报如下错误(待解决)，

发送消息Msseage1567755952639
java.util.concurrent.ExecutionException: org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for kafka_test-0:120000 ms has passed since batch creation
	at org.apache.kafka.clients.producer.internals.FutureRecordMetadata.valueOrError(FutureRecordMetadata.java:98)
	at org.apache.kafka.clients.producer.internals.FutureRecordMetadata.get(FutureRecordMetadata.java:67)
	at org.apache.kafka.clients.producer.internals.FutureRecordMetadata.get(FutureRecordMetadata.java:30)
	at Producer1.produceMsg(Producer1.java:45)
	at Producer1$1.run(Producer1.java:59)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.kafka.common.errors.TimeoutException: Expiring 1 record(s) for kafka_test-0:120000 ms has passed since batch creation