problems in the project

1. Search and product service issues

Commodity service modifies the status of the product on the shelf, and the product can be searched. Using message notification, the product service modifies the status of the product on the shelf, sends a message to the search service, searches the service consumption message, and saves the product data ES. The same is true for the off-shelf.

2. The order service cancels the order problem

delay queue

After the order is saved, the timing starts, and when the time is up, cancel the unpaid order.

The delayed message of rabbitMQ.

3. Distributed transaction issues

One day before, I talked about distributed transactions, which are all concepts. Among them is the final data consistency of a message .

It is recommended that everyone go back and read the courseware or courses of distributed transactions.

Scenes:

Payment ---------- order ---------- inventory is a distributed transaction scenario.

4. When the spike

Use rabbitMQ for message notification and user queuing.

What problem does message queue solve

What problems do message queues solve?

1. Asynchronous

2. Parallel

3. Decoupling

4. Queuing, peak cutting, cutting off the peak value of traffic

Message queue tool RabbitMQ

1. Common MQ products

- ActiveMQ: Based on JMS (java protocol), there are fewer message types, 2 types: one-to-one and one-to-many

- RabbitMQ: based on the AMQP protocol, developed in erlang language, with good stability and multiple message types

- RocketMQ: Based on JMS, Alibaba products, currently handed over to the Apache Foundation, and used a lot.

- Kafka: distributed message system, high throughput, poor message accuracy

2. Basic concepts of RabbitMQ

Broker: Simply put, it is the message queue server entity

Exchange: message switch, which specifies which queue the message is routed to according to the rules

Queue: message queue carrier, each message will be put into one or more queues

Binding: Binding, its function is to bind exchange and queue according to routing rules

Routing Key: Routing keyword, exchange delivers messages based on this keyword

vhost: virtual host, multiple vhosts can be set up in one broker, which is used to separate permissions for different users

producer: The message producer is the program that delivers the message

consumer: The message consumer is the program that receives the message

channel: message channel, in each connection of the client, multiple channels can be established, and each channel represents a session task

3. Message model

RabbitMQ provides 6 message models, but the sixth is actually RPC, not MQ, so it will not be learned. Then there are 5 left.

But in fact, 3, 4, and 5 all belong to the subscription model, but the routing methods are different.

Basic message model: Producer –> Queue –> Consumer

Work message model: producer –> queue –> multiple consumers consume together

Subscription model-Fanout: Broadcast mode, hand over the message to all queues bound to the exchange, and each consumer will receive the same message

Subscription model-Direct: directional, deliver the message to the queue that matches the specified rotingKey

Subscription model-Topic topic mode: wildcard, send the message to the queue that conforms to the routing pattern (routing pattern)

message is not lost

The accuracy of the message is guaranteed! ! ! !

From the perspective of MQ, there are generally three ways to ensure that messages are not lost :

Producers don't lose data

The producer produces a message and accurately delivers it to the exchange and queue.

Solution: rabbitMQ provides message sending confirmation, switch reply and queue reply.

The message is produced and delivered to the interactive machine. If the delivery is successful, a true receipt will be given, and if it fails, a false receipt will be given.

When the switch routes messages to the queue, if the message does not arrive in the queue, the queue response will be triggered, and the corresponding response method will be executed.

rabbitmq:

    host: 192.168.200.128

    port: 5672

    username: guest

    password: guest

    publisher-confirm-type: correlated #开启交换机应答

    publisher-returns: true   #队列应答

    listener:

      simple:

        acknowledge-mode: manual #默认情况下消息消费者是自动确认消息的，如果要手动确认消息则需要修改确认模式为manual

        prefetch: 1 # 消费者每次从队列获取的消息数量。此属性当不设置时为：轮询分发，设置为1为：公平分发

prefetch : 1 The number of messages consumers consume from the queue at a time.

Polling distribution: In the case of multiple consumers, one person once, no matter whether the last message of the consumer has been consumed or not, it will be given to whoever takes the turn.

Fair distribution: In the case of multiple consumers, one person once, if it is the consumer's turn, but the previous message has not been consumed, the message will be given to others. Only when the previous message is consumed can the next message be consumed.

2. The MQ server does not lose data

The message queue data is stored in the memory. If MQ hangs up and restarts, the memory is easy to release, and the message is gone.

Persistence is provided, and exchanges, queues, and messages can be persisted.

3. Consumers do not lose data

Consumers may have problems in the process of consuming messages, and the messages must still exist.

Manually sign and receive the message. Once there is a problem in the message consumption process, you can refuse to sign for it, forward the message to another queue or record the abnormal consumption message, and wait until the problem is solved before re-delivering.

There are two ways to ensure that messages are not lost:

Enable transaction mode
message acknowledgment mode

Explanation: Opening a transaction will greatly reduce the efficiency of message sending and receiving, and it is relatively seldom used.

Turn on transaction support when delivering a message. If the delivery of the message fails, the transaction will be rolled back. However, few people do this, because this is a synchronous operation. After a message is sent, the sender will be blocked to wait for the RabbitMQ-Server response . After that, the next message can be sent, and the throughput and performance of the producer's production message will be greatly reduced.

Therefore, our production environment generally adopts the message confirmation mode.

1. Message persistence

If you want messages not to be lost after RabbitMQ restarts, you need to configure persistence for the following three entities

Exchange

Set persistence (durable = true) and not automatically delete (autoDelete = false) when declaring exchange

Queue

Set persistence (durable = true) and not automatically delete (autoDelete = false) when declaring the queue

message

Persist messages by setting deliveryMode=2 when sending messages

illustrate:

@Queue: Whether to automatically delete the queue when all consumer client connections are disconnected

true: delete false: do not delete

@Exchange: Whether to automatically delete the exchange when all bound queues are not in use

true: delete false: do not delete

2. Send confirmation

Sometimes, the business process is successful and the message is sent, but we don't know whether the message has successfully reached rabbitmq. If the business succeeds but the message fails to be sent due to reasons such as the network, then the sender will have an inconsistency problem. At this time, rabbitmq can be used The sending confirmation function requires rabbitmq to explicitly inform us whether the message has been successfully sent.

3. Manual consumption confirmation

Sometimes, the message is correctly delivered to the consumer, but the consumer fails to process it, then there will be an inconsistency problem on the consumer. For example, the message that the order has been created is sent to the user points subsystem to increase user points, but the point consumer fails to process it, and the user will ask: Why didn’t the points increase after I purchased something?

To solve this problem, consumer confirmation needs to be introduced, that is, rabbitmq is notified to ack only after the message is successfully processed, otherwise rabbitmq is notified to nack

Product search on and off shelves

1. Service-product sends a message

I send messages when items are available and when items are added

Product on shelves

实现类

@Override

@Transactional

  public void onSale(Long skuId) {

    // 更改销售状态

    SkuInfo skuInfoUp = new SkuInfo();

    skuInfoUp.setId(skuId);

    skuInfoUp.setIsSale(1);

    skuInfoMapper.updateById(skuInfoUp);

    //商品上架

    rabbitService.sendMessage(MqConst.EXCHANGE_DIRECT_GOODS, MqConst.ROUTING_GOODS_UPPER, skuId);

}

Product off the shelves

实现类

@Override

@Transactional

  public void cancelSale(Long skuId) {

    // 更改销售状态

    SkuInfo skuInfoUp = new SkuInfo();

    skuInfoUp.setId(skuId);

    skuInfoUp.setIsSale(0);

    skuInfoMapper.updateById(skuInfoUp);

    //商品下架

    rabbitService.sendMessage(MqConst.EXCHANGE_DIRECT_GOODS, MqConst.ROUTING_GOODS_LOWER, skuId);

}

2. service-list consumption message

package com.atguigu.gmall.list.receiver;

  @Component

  public class ListReceiver {

    @Autowired

    private SearchService searchService;

    /**

     * 商品上架

     * @param skuId

     * @throws IOException

     */

    @RabbitListener(bindings = @QueueBinding(

            value = @Queue(value = MqConst.QUEUE_GOODS_UPPER, durable = "true"),

            exchange = @Exchange(value = MqConst.EXCHANGE_DIRECT_GOODS, type = ExchangeTypes.DIRECT, durable = "true"),

            key = {MqConst.ROUTING_GOODS_UPPER}

    ))

    public void upperGoods(Long skuId, Message message, Channel channel) throws IOException {

        if (null != skuId) {

            searchService.upperGoods(skuId);

        }

        channel.basicAck(message.getMessageProperties().getDeliveryTag(), false);

    }

    /**

     * 商品下架

        * @param skuId

     */

    @RabbitListener(bindings = @QueueBinding(

            value = @Queue(value = MqConst.QUEUE_GOODS_LOWER, durable = "true"),

            exchange = @Exchange(value = MqConst.EXCHANGE_DIRECT_GOODS, type = ExchangeTypes.DIRECT, durable = "true"),

            key = {MqConst.ROUTING_GOODS_LOWER}

    ))

    public void lowerGoods(Long skuId, Message message, Channel channel) throws IOException {

        if (null != skuId) {

            searchService.lowerGoods(skuId);

        }

        channel.basicAck(message.getMessageProperties().getDeliveryTag(), false);

    }

}

Delay Queue Closing Expired Orders

There are two implementations for delayed messages:

1. Based on dead letter queue

Use the survival time of the message to set parameters for the queue. When the time is up, it will be forwarded to another queue. What the consumer actually consumes is another queue.

2. Integrated delay plug-in

There is a small library in the delay plug-in. The delay switch stores the message in the small library first, and then forwards it to the queue when the time is up.

the difference:

Delayed message of dead letter: The message time is required to be consistent, if not, subsequent messages will not come out.

Delay plugins: no such requirement. The timing of the messages can be inconsistent.

1. Delay messages based on dead letters

To use RabbitMQ to implement delayed messages, you must first understand two concepts of RabbitMQ: message TTL and dead letter Exchange, and the combination of the two to implement delay queues

1.1. Message TTL (Time To Live)

The TTL of the message is the lifetime of the message. RabbitMQ can set TTL for queues and messages separately. The queue setting is the retention time that the queue has no consumer connection, and it can also be set separately for each individual message. After this time, we consider the news to be dead and call it a dead letter.

我们创建一个队列queue.temp，在Arguments 中添加x-message-ttl 为5000 （单位是毫秒），那所在压在这个队列的消息在5秒后会消失。

1.2、死信交换器 Dead Letter Exchanges

一个消息在满足如下条件下，会进死信路由，记住这里是路由而不是队列，一个路由可以对应很多队列。

（1）一个消息被Consumer拒收了，并且reject方法的参数里requeue是false。也就是说不会被再次放在队列里，被其他消费者使用。

（2）上面的消息的TTL到了，消息过期了。

（3）队列的长度限制满了。排在前面的消息会被丢弃或者扔到死信路由上。

Dead Letter Exchange其实就是一种普通的exchange，和创建其他exchange没有两样。只是在某一个设置了Dead Letter Exchange的队列中有消息过期了，会自动触发消息的转发，发送到Dead Letter Exchange中去。

2 、基于延迟插件实现延迟消息

2.1、插件安装

1. 首先我们将刚下载下来的rabbitmq_delayed_message_exchange-3.8.0.ez文件上传到RabbitMQ所在服务器，下载地址：https://www.rabbitmq.com/community-plugins.html

2. 切换到插件所在目录，执行 docker cp rabbitmq_delayed_message_exchange-3.8.0.ez rabbitmq:/plugins 命令，将刚插件拷贝到容器内plugins目录下

3. 执行 docker exec -it rabbitmq /bin/bash 命令进入到容器内部，并 cd plugins 进入plugins目录

4. 执行 ls -l|grep delay 命令查看插件是否copy成功

5. 在容器内plugins目录下，执行 rabbitmq-plugins enable rabbitmq_delayed_message_exchange 命令启用插件

6. exit命令退出RabbitMQ容器内部，然后执行 docker restart rabbitmq 命令重启RabbitMQ容器

2.2、代码实现

配置队列

package com.atguigu.gmall.mq.config;

  

  

  @Configuration

  public class DelayedMqConfig {

  

    public static final String exchange_delay = "exchange.delay";

    public static final String routing_delay = "routing.delay";

    public static final String queue_delay_1 = "queue.delay.1";

  

    /**

     * 队列不要在RabbitListener上面做绑定，否则不会成功，如队列2，必须在此绑定

     *

     * @return

     */

  

    @Bean

    public Queue delayQeue1() {

        // 第一个参数是创建的queue的名字，第二个参数是是否支持持久化

        return new Queue(queue_delay_1, true);

    }

  

    @Bean

    public CustomExchange delayExchange() {

        Map<String, Object> args = new HashMap<String, Object>();

        args.put("x-delayed-type", "direct");

        return new CustomExchange(exchange_delay, "x-delayed-message", true, false, args);

    }

  

    @Bean

    public Binding delayBbinding1() {

        return BindingBuilder.bind(delayQeue1()).to(delayExchange()).with(routing_delay).noargs();

    }

  

}

发送消息

@GetMapping("sendDelay")

  public Result sendDelay() {

   SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");

   this.rabbitTemplate.convertAndSend(DelayedMqConfig.exchange_delay, DelayedMqConfig.routing_delay, sdf.format(new Date()), new MessagePostProcessor() {

      @Override

      public Message postProcessMessage(Message message) throws AmqpException {

         message.getMessageProperties().setDelay(10 * 1000);

         System.out.println(sdf.format(new Date()) + " Delay sent.");

         return message;

      }

   });

   return Result.ok();

}

接收消息

package com.atguigu.gmall.mq.receiver;

  

  

  @Component

@Configuration

  public class DelayReceiver {

  

    @RabbitListener(queues = DelayedMqConfig.queue_delay_1)

    public void get(String msg) {

        SimpleDateFormat sdf = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");

        System.out.println("Receive queue_delay_1: " + sdf.format(new Date()) + " Delay rece." + msg);

    }

  

}

3 、基于延迟插件实现取消订单

rabbit-util模块延迟接口封装

RabbitService
/**

 * 发送延迟消息

 * @param exchange 交换机

 * @param routingKey 路由键

 * @param message 消息

 * @param delayTime 单位：秒

 */

  public boolean sendDelayMessage(String exchange, String routingKey, Object message, int delayTime) {

    GmallCorrelationData correlationData = new GmallCorrelationData();

    String correlationId = UUID.randomUUID().toString();

    correlationData.setId(correlationId);

    correlationData.setMessage(message);

    correlationData.setExchange(exchange);

    correlationData.setRoutingKey(routingKey);

    correlationData.setDelay(true);

    correlationData.setDelayTime(delayTime);

  

    redisTemplate.opsForValue().set(correlationId, JSON.toJSONString(correlationData), OBJECT_TIMEOUT, TimeUnit.MINUTES);

    this.rabbitTemplate.convertAndSend(exchange, routingKey, message, new MessagePostProcessor() {

        @Override

        public Message postProcessMessage(Message message) throws AmqpException {

            message.getMessageProperties().setDelay(delayTime*1000);

            return message;

        }

    },correlationData);

    return true;

}

3.1、发送消息

创建订单时，发送延迟消息

修改保存订单方法

@Override

@Transactional

  public Long saveOrderInfo(OrderInfo orderInfo) {

    // orderInfo

    // 总金额，订单状态，用户Id，第三方交易编号，创建时间，过期时间，进程状态

    orderInfo.sumTotalAmount();

    orderInfo.setOrderStatus(OrderStatus.UNPAID.name());

    String outTradeNo = "ATGUIGU" + System.currentTimeMillis() + "" + new Random().nextInt(1000);

    orderInfo.setOutTradeNo(outTradeNo);

    orderInfo.setCreateTime(new Date());

    // 定义为1天

    Calendar calendar = Calendar.getInstance();

    calendar.add(Calendar.DATE, 1);

    orderInfo.setExpireTime(calendar.getTime());

  

    orderInfo.setProcessStatus(ProcessStatus.UNPAID.name());

    orderInfoMapper.insert(orderInfo);

  

    StringBuffer tradeBody = new StringBuffer();

    // 保存订单明细

    List<OrderDetail> orderDetailList = orderInfo.getOrderDetailList();

    for (OrderDetail orderDetail : orderDetailList) {

        orderDetail.setId(null);

        orderDetail.setOrderId(orderInfo.getId());

        orderDetailMapper.insert(orderDetail);

  

        tradeBody.append(orderDetail.getSkuName()).append(" ");

    }

  

    //更新支付描述

    orderInfo.setTradeBody(tradeBody.toString());

    orderInfoMapper.updateById(orderInfo);

    //发送延迟队列，如果到过期时间了未支付，取消订单

 rabbitService.sendDelayMessage(MqConst.EXCHANGE_DIRECT_ORDER_CANCEL, MqConst.ROUTING_ORDER_CANCEL, orderInfo.getId(), MqConst.DELAY_TIME);

    // 返回

    return orderInfo.getId();

}

3.2、接收消息

package com.atguigu.gmall.order.receiver;

  @Component

  public class OrderReceiver {

    @Autowired

    private OrderService orderService;

    /**

     * 取消订单消费者

       * 延迟队列，不能再这里做交换机与队列绑定
* 需要在配置类里去绑定死信交换机
     * @param orderId
     * @throws IOException
     */
    @RabbitListener(queues = MqConst.QUEUE_ORDER_CANCEL)
    public void orderCancel(Long orderId, Message message, Channel channel) throws IOException {
        if (null != orderId) {
            //防止重复消费
            OrderInfo orderInfo = orderService.getById(orderId);
            if (null != orderInfo && orderInfo.getOrderStatus().equals(ProcessStatus.UNPAID.getOrderStatus().name())) {
                orderService.execExpiredOrder(orderId);
            }
        }
        channel.basicAck(message.getMessageProperties().getDeliveryTag(), false);
    }

}

取消订单业务，取消订单要关闭支付交易

@Override

  public void execExpiredOrder(Long orderId) {

    // orderInfo

    updateOrderStatus(orderId, ProcessStatus.CLOSED);

    // paymentInfo

    //paymentFeignClient.closePayment(orderId);

    //发送取消交易的消息

    rabbitService.sendMessage(MqConst.EXCHANGE_DIRECT_PAYMENT_CLOSE, MqConst.ROUTING_PAYMENT_CLOSE, orderId);

}

关闭交易消息消费者

package com.atguigu.gmall.payment.receiver;

@Component

public class PaymentReceiver {

    @Autowired

    private PaymentService paymentService;

    /**

     * 取消交易

     * @param orderId

     * @throws IOException

     */

    @RabbitListener(bindings = @QueueBinding(

            value = @Queue(value = MqConst.QUEUE_PAYMENT_CLOSE, durable = "true"),

            exchange = @Exchange(value = MqConst.EXCHANGE_DIRECT_PAYMENT_CLOSE),

            key = {MqConst.ROUTING_PAYMENT_CLOSE}

    ))

    public void closePayment(Long orderId) throws IOException {

        if (null != orderId) {

            paymentService.closePayment(orderId);

        }

    }

}

更改支付日志表，状态为关闭交易

@Override

public void closePayment(Long orderId) {

    QueryWrapper<PaymentInfo> queryWrapper = new QueryWrapper<>();

    queryWrapper.eq("order_id", orderId);

    PaymentInfo paymentInfoUp = new PaymentInfo();

    paymentInfoUp.setPaymentStatus(PaymentStatus.ClOSED.name());

    paymentInfoMapper.update(paymentInfoUp, queryWrapper);

    //关闭交易

     alipayService.closePay(orderId);
}
支付宝支付AlipayServiceImpl实现类
/***

 * 关闭交易

 * @param orderId

 * @return

 */

@Override

public Boolean closePay(Long orderId) {

    OrderInfo orderInfo = orderFeignClient.getOrderInfo(orderId);



    //AlipayClient alipayClient = new DefaultAlipayClient("https://openapi.alipay.com/gateway.do","app_id","your private_key","json","GBK","alipay_public_key","RSA2");

    AlipayTradeCloseRequest request = new AlipayTradeCloseRequest();

    HashMap<String, Object> map = new HashMap<>();

    map.put("trade_no", "");

    map.put("out_trade_no", orderInfo.getOutTradeNo());

    map.put("operator_id", "YX01");



    request.setBizContent(JSON.toJSONString(map));

    AlipayTradeCloseResponse response = null;

    try {

        response = alipayClient.execute(request);

    } catch (AlipayApiException e) {

        e.printStackTrace();

    }

    if(response.isSuccess()){

        log.info("调用成功");

        return true;

    }

    return false;

}

项目中分布式事务的业务场景

RabbitMQ常见问题

1、使用RabbitMQ有什么好处？

1）.解耦：系统A在代码中直接调用系统B和系统C的代码，如果将来D系统接入，系统A还需要修改代码，过于麻烦！

2）.异步：将消息写入消息队列，非必要的业务逻辑以异步的方式运行，加快响应速度

3）.削峰：并发量大的时候，所有的请求直接怼到数据库，造成数据库连接异常

2、消息顺序问题

场景：比如支付操作，支付成功之后，会发送修改订单状态和扣减库存的消息，如果这两个消息同时发送，就不能保证完全按照顺序消费，有可能是先减库存了，后更改订单状态。

解决方案：同步执行，当一个消息执行完之后，再发布下一个消息。

3、如何保证RabbitMQ消息的可靠传输？

消息不可靠的原因是因为消息丢失

生产者丢失消息:

RabbitMQ提供transaction事务和confirm模式来确保生产者不丢消息；

Transaction事务机制就是说：发送消息前，开启事务（channel.txSelect()）,然后发送消息，如果发送过程中出现什么异常，事务就会回滚（channel.txRollback()）,如果发送成功则提交事务（channel.txCommit()），然而，这种方式有个缺点：吞吐量下降。

confirm模式用的居多：一旦channel进入confirm模式，所有在该信道上发布的消息都将会被指派一个唯一的ID（从1开始），一旦消息被投递到所有匹配的队列之后；

rabbitMQ就会发送一个ACK给生产者（包含消息的唯一ID），这就使得生产者知道消息已经正确到达目的队列了；

如果rabbitMQ没能处理该消息，则会发送一个Nack消息给你，可以进行重试操作。

消息列表丢失消息:

可以消息持久化, 即使rabbitMQ挂了，重启后也能恢复数据

消费者丢失消息:

消费者丢数据一般是因为采用了自动确认消息模式，消费者在收到消息之后，处理消息之前，会自动回复RabbitMQ已收到消息；如果这时处理消息失败，就会丢失该消息；改为手动确认消息即可！

4、消息重复消费问题

为什么会重复消费：

正常情况下，消费者在消费消息的时候，消费完毕后，会发送一个确认消息给消息队列，消息队列就知道该消息被消费了，就会将该消息从消息队列中删除；

但是因为网络传输等等故障，确认信息没有传送到消息队列，导致消息队列不知道已经消费过该消息了，再次将消息发送。

解决方案：保证消息的唯一性，就算是多次传输，不要让消息的多次消费带来影响，保证消息消费的幂等性；

5、幂等性操作

幂等性就是一个数据或者一个请求，给你重复来了多次，你得确保对应的数据是不会改变的，不能出错。

要保证消息的幂等性，这个要结合业务的类型来进行处理。

1）、可在内存中维护一个map集合，只要从消息队列里面消费一个消息，先查询这个消息在不在map里面，如果在表示已消费过，直接丢弃；如果不在，则在消费后将其加入map当中。
2）、如果要写入数据库，可以拿唯一键先去数据库查询一下，如果不存在在写，如果存在直接更新或者丢弃消息。

3）、消息执行完会更改某个数据状态，判断数据状态是否更新，如果更新，则不进行重复消费。
4）、如果是写redis那没有问题，每次都是set，天然的幂等性。

Shangpin Summary 9: Application of RabbitMQ in the project (for interview)