Kafka Introduction Series—3. Consumer Group

1. Topic from-beginningparameters

• Start zookeeper and kafka

  bin/zkServer.sh start-foreground
  
  bin/kafka-server-start.sh config/server.properties
  

• Create topic

  bin/kafka-topics.sh --create --zookeeper localhost:2181 --topic testtopic
  

• Start the producer and two consumers

  Start producer

  bin/kafka-console-producer.sh --broker-list localhost:9092 --topic testtopic
  

  Open two command line windows to start two consumers

   bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic testtopic --from-beginning
  

• Enter some characters in the producer, and you can see that these characters have been obtained in the consumer.

producer

Display the received information in two consumers

• Open the third command line window and create a new consumer, but this time the command is different from the previous one :

  bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic testtopic
  

As you can see, there are no parameters when creating the consumer this time --from-beginning. At the same time, no information is output in the window.

• At this time, enter the new information in the producer to view the status of the three consumers:

New type in the producer will go to London

Complete display of all information in the first two consumers

The third consumer only shows the latest will go to London

The above example can confirm the previous concept:

1. Multiple consumers can subscribe to the same topic.
2. --from-beginningThe function of the parameter is to make the consumer start to consume from the earliest message of Kafka .

2. Consumer Group

Before that, close the three consumer windows created earlier. And use the following commands to create two new consumers. The commands this time are slightly different from the previous ones.

1. A consumer can only belong to one consumer group
2. The topic subscribed by the consumer group can only be consumed by one of the consumers
3. Consumers in different consumer groups can consume the same topic

• Instance

Open two new command line windows and create two new consumers:

 

bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic testtopic --group testgroup

Enter information in the producer window, and you will find that only one consumer window has output.

Close the two consumers you just created, and use the following two commands to create two new consumers:

 

bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic testtopic --group testgroup1

bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic testtopic --group testgroup2

testtopicThe two subscribed consumers belong to different consumer groups testgroup1and testgroup2, so the messages generated by the producer will be displayed in both windows.