Java implementation example of Kafka's producers and consumers

RabbitMQ similar structure and Kafka, Kafka message producer sends a message to the server, after receiving a message Kafka, and then delivered to the consumer.
Consumer producers will be sent to the Topic, Topic preserved in all types of data, each piece of data using a key, value preservation.
Each of the Topic contains one or more physical partitions (the Partition), the contents and index partition maintenance messages, they may be stored in different servers.

Create a Maven project, pom.xml join rely on:

        <dependency>
            <groupId>org.apache.kafka</groupId>
            <artifactId>kafka-clients</artifactId>
            <version>2.3.0</version>
        </dependency>

1, write producer

Kafka delivered the message to the server name is "topic1" in Topic

package com.example.kafkatest;

import org.apache.kafka.clients.producer.KafkaProducer;
import org.apache.kafka.clients.producer.ProducerRecord;
import org.apache.kafka.common.serialization.StringSerializer;

import java.util.Properties;

public  class Producer {
     public  static  void main (String [] args) {
         // configuration information for 
        the Properties The props = new new the Properties ();
         // Kafka server address 
        props.put ( "bootstrap.servers", "localhost: 9092" );
         / / set key value and data processing sequences based 
        props.put ( "key.serializer", StringSerializer. class );
        props.put ( "value.serializer", StringSerializer. class );
         // Create instance of the producer 
        KafkaProducer <String, String> Producer = new new KafkaProducer <> (The props);
        ProducerRecord record = new ProducerRecord<String, String>("topic1", "userName", "lc");
        //发送记录
        producer.send(record);
        producer.close();
    }
}

After the run, you can open a command-line tool, enter Kafka directory, execute the command to query the server Topic:

bin\windows\kafka-topics.bat --list --zookeeper localhost:2181

The results are as follows:

2, write consumer

In this example, consumers and producers in the same project, but using a different startup class.
Consumers will specify the identity of a group of consumers for their own, each to Topic posted record will be delivered to a consumer example consumer groups.
If multiple instances consumers have the same group of consumers, these records will be assigned to each consumer instance, in order to achieve load balancing directory.
If all consumers have different consumer groups, each record will be broadcast to all consumers for processing.

package com.example.rabbittest;

import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.apache.kafka.clients.consumer.ConsumerRecords;
import org.apache.kafka.clients.consumer.KafkaConsumer;
import org.apache.kafka.common.serialization.StringDeserializer;

import java.time.Duration;
import java.util.Arrays;
import java.util.Properties;

public  class Consumer {
     public  static  void main (String [] args) {
         // configuration information for 
        the Properties The props = new new the Properties ();
         // Kafka server address 
        props.put ( "bootstrap.servers", "localhost: 9092" );
         / / must be specified consumer group 
        props.put ( "the group.id", "Test" );
         // set the key and value data processing sequence based 
        props.put ( "key.deserializer", the StringDeserializer. class );
        props.put ( "value.deserializer", the StringDeserializer. class );
         // create message instance by 
        KafkaConsumer <String, String> Consumer = new new KafkaConsumer <> (The props);
         // subscribe message topic1 
        consumer.subscribe (Arrays.asList ( "Topic1" ));
         // the server reads records 
        the while ( to true ) {
            ConsumerRecords<String,String> records = consumer.poll(Duration.ofMillis(100));
            for(ConsumerRecord<String,String> record : records){
                System.out.println("key:" + record.key() + "" + ",value:" + record.value());
            }
        }
    }
}

After running, IDEA wherein the console output is as follows: