"Chapter 4 Decoupling of Distributed Systems" "Section 5 Introduction to Common Message Middleware", "Section 6 Message Queue Persistence and Reliable Messages and Transaction Messages" in "Special Topic Four Service Transformation"

"3.5.1 Introduction to the concept of message middleware and RabbitMQ"

• 4 30 Two modes of message queue: point-to-point and publish and subscribe:
  

• RabbitMQ:
  

• 26 points AMQP model:
  

• 47 30 Message:
  

• 64 40 RabbitMQ architecture:
  

• 66 points RabbitMQ message operation process:
  

"3.5.2 Introduction to Kafka Technical Architecture and Configuration"

• The overall structure of Kafka is shown in the figure below. One broker can hold multiple topics

• 6 points Only one Consumer in each Consumer Group consumes one partition

• Broker:
  

• Consumer:
  
  

• 24分 kafka data flow:
  

• Production Process:
  

• 29 10 Kafka will try to allocate the leader to different brokers; the leader is responsible for reading and writing, while the replica is only responsible for reading; when a broker fails, it will re-elect the leader among them; the executor for partition allocation and leader election is called Controller, zookeeper is used by default

• • 31 10+ leader election
• • 37 40 In the earlier version, the offset is saved in zookeeper

• Partition is allocated during the consumption process:

• • 40 27 The choice of coordinator during consumption: generally the broker where the leader who saves the offset is located
• • 45 50 When does rebalance happen: The number of partitions or consumers has changed

• 47 45 Message delivery semantics:
  
  At least once: After the consumer has processed the business, update the offset
  At most once: After receiving the message, update the offset first, and then process the business

• 54 40 Production idempotence:
  

"3.5.3 Introduction to RocketMQ"

• 22 20 RocketMQ's physical architecture:
  
• • 26 40 Producer only sends messages to broker master, and consumer can read messages from broker master or broker slave
• 29 27 Unlike Kafka, all instances in the Consumer group will receive the same message

"3.6.1 Comparative Use Experience of Message Middleware"

• 18 50
  
  
  

• Selection suggestions:
  

• Realize that the message must reach:
  

• • 33 30 Core process:
    

• 37 points + realize message idempotence: the
  first half uses id, id has two characteristics: 1) Globally unique, 2) Generated by MQ, business side does not need to care about
  the biz id in the second half? Generated by the producer, in the corresponding consumption scenario It is the only one in the whole world, which is judged by consumers and guaranteed to be idempotent, and transparent to MQ. For example, put this id in the message body

• 45 25 Traffic peak reduction:
  the mode for consumers to fetch messages, use pull instead of push

• 46 50 Realizing delayed messages:
  cron timing tasks
  More efficient design: circular queue