Reids ideas and implement message mode queues

table of Contents

Foreword

Features 1 queue

2 Use List simple queue

2.1 ideas

2.2 simple queue related commands

2.3 RedisTemplate Operation List message queue implemented

3 SortedSet implemented using delay queue

Latency Queuing 3.1 application scenarios

3.2 ideas

3.4 RedisTemplate operation SortedSet achieve Latency Queuing

4 Spring Boot environment in connection Redis implements the publish / subscribe

4.1 two producers

4.2 message listener configuration

4.3 Producer

4.4 Test Results

Foreword

Redis is now the most popular key-value database, there are many scenarios, including a cache, a distributed lock, counter, etc., described herein are implemented using Redis message queue. A dedicated message queue middleware, including RabbitMQ, RocketMQ, kafka other, they all belong to the heavyweight. In the project, if the demand is not necessary to use heavyweight messaging middleware, message queues can be implemented using Redis.

Features 1 queue

Queue data structure is a linear, there are two basic operations, enqueue and dequeue - enqueuing is writing data to the tail, a dequeue data is taken out of the queue head, is often said that a FIFO (First Input First Out), FIFO .

2 Use List simple queue

2.1 ideas

List of Redis type element from the list may be inserted into the header (leftmost) or a footer (far right), of course, the same can also remove elements from the header or footer. Based on this characteristic, assuming List of the leftmost element is the head of the team , the rightmost element is the tail , then to the right of the List element is inserted into the team , to the left is the most List of removing elements from the team .

2.2 simple queue related commands

Just, Redis's List type to support such operations, the command will be used include Rpush, Lpop, Blpop. Wherein, RPUSH command values for one or more inserted at the tail of the list (far right), may be used herein enqueued. Lpop BLPOP and commands are available for dequeuing, wherein Lpop command is used to remove and return the first element of the list, when the specified key does not exist, returns nil; BLPOP also removes and returns the first element of the list, and Lpop command is different, it will block until a timeout or wait until the discovery of elements may pop up when Blpop no elements in the list.

2.3 RedisTemplate Operation List message queue implemented

Here the message queue using simple SpringBoot + RedisTemplate, the code consists of two parts, the producer (enqueued) and consumer (dequeue).

The first is the team (producer) code here is equivalent to using Rpush "simpleQueue" List of three values written in "Java", "C ++" and "Python" is the key.

import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.test.context.junit4.SpringRunner;


@RunWith(SpringRunner.class)
@SpringBootTest
public class MessageSender {
    @Autowired
    private RedisTemplate<String, String> redisTemplate;

    @Test
    public void sendMessage() {
        // 往队列的最右边添加元素
        redisTemplate.opsForList().rightPushAll("simpleQueue", "Java", "C++", "Python");
    }
}

Then the team (consumer) Code, redisTemplate.opsForList (). LeftPop have more overloaded methods, the wording here is the equivalent of calling Blpop command, and set the time-out time of 60 seconds, a way to block pop-up element.

import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.dao.QueryTimeoutException;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.test.context.junit4.SpringRunner;

import java.util.concurrent.TimeUnit;

@RunWith(SpringRunner.class)
@SpringBootTest
public class MessageListener {
    @Autowired
    private RedisTemplate<String, String> redisTemplate;

    @Test
    public void listenMessage() {
        while(true) {
            try {
                String message = redisTemplate.opsForList().leftPop("simpleQueue", 60, TimeUnit.SECONDS);
                System.out.println("已消费队列" + message);
            } catch (QueryTimeoutException e) {
                System.out.println("60秒内没有新的消息，继续。。。");
            }
        }
    }
}

3 SortedSet implemented using delay queue

Latency Queuing 3.1 application scenarios

All orders created if not operated within 15 minutes, automatically canceled;
Send SMS, 11 send the whole message, performed again for 2 minutes;
Users under the takeout orders, businesses are not orders within 5 minutes, we need to automatically cancel the order.

3.2 ideas

As used herein, the Redis SortedSet type implementations delay queue. First of all, it is the same as String Set SortedSet and collection types of elements, and does not allow duplicate elements, and Set The difference is that a double score associated with each type of element will SortedSet, redis by a fraction of this sort as a member of SortedSet, Also this can be repeated fractional value.

Now there is such a demand, will create more than 15 minutes of an order to close, based on the characteristics SortedSet can do so.

In our producers here a SortedSet collection for the delay queue, Key is a constant, defined here as orderId. When you create an order, the order number is written in the key of orderId SortedSet, that is, the value of the order number of stored value, pushed back fifteen minutes when the score deposit timestamp order creation.

In the consumer side of the key to get all elements in the collection based on traversing these elements will score less than the current value of the timestamp value delete, and consume these messages.

3.3 Related command queue delay achieved

Latency Queuing will be used to achieve SortedSet related command are:

Zadd : for one or more elements and added to the ordered set point value;
Zrangebyscore: Returns the specified fraction section member list of elements, the results are arranged in ascending by score;
Zrem : removing one or more elements of the collection.

3.4 RedisTemplate operation SortedSet achieve Latency Queuing

Still is SpringBoot + RedisTemplate, includes producers and consumers of the code, of course RedisTemplate command made package, API's name redis command name itself is different.

Producer:

import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.test.context.junit4.SpringRunner;

import java.util.Calendar;
import java.util.Random;

@RunWith(SpringRunner.class)
@SpringBootTest
public class MessageSender {
    @Autowired
    private RedisTemplate<String, String> redisTemplate;

    @Test
    public void sendMessage() {
        // 时间戳取当前时间往后推15分钟，存入sorted-set的score值
        Calendar calendar = Calendar.getInstance();
        calendar.add(Calendar.MINUTE, 15);
        double millisecond = calendar.getTimeInMillis();
        // 以简单的方式模拟订单号
        Random random = new Random();
        int orderId = Math.abs(random.nextInt());
        redisTemplate.opsForZSet().add("orderId", String.valueOf(orderId), millisecond );
        System.out.println("发送订单任务，订单ID为===============" + orderId);
    }
}

consumer:

import org.junit.Test;
import org.junit.runner.RunWith;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.data.redis.core.ZSetOperations;
import org.springframework.test.context.junit4.SpringRunner;

import java.util.Iterator;
import java.util.Set;

@RunWith(SpringRunner.class)
@SpringBootTest
public class MessageListener {
    @Autowired
    private RedisTemplate<String, String> redisTemplate;

    @Test
    public void listenMessage() {
        while(true){
            Set<ZSetOperations.TypedTuple<String>> orderIdSet = redisTemplate.opsForZSet().rangeWithScores("orderId", 0, -1);
            if(orderIdSet == null || orderIdSet.isEmpty()){
                try {
                    Thread.sleep(500);
                } catch (InterruptedException e) {
                    e.printStackTrace();
                    break;
                }
                continue;
            }
            Iterator<ZSetOperations.TypedTuple<String>> iterator = orderIdSet.iterator();
            ZSetOperations.TypedTuple<String> next = iterator.next();
            Double score = next.getScore();
            if(score == null) {
                continue;
            }
            double nowTime = System.currentTimeMillis();
            if(nowTime >= score) {
                String value = next.getValue();
                redisTemplate.opsForZSet().remove("orderId", value);
                System.out.println("已成功处理一条订单，订单id为" + value);
            }
        }
    }
}

4 Spring Boot environment in connection Redis implements the publish / subscribe

Publish / Subscribe namely Publish / the Subscribe , it is a messaging mode of communication. In this mode you can have multiple consumers subscribe to any number of channels, the producer sends a message to the channel, subscribed to the channel of all consumers will receive the message.

Redis itself supports publish / subscribe, subscribe to channels of command SUBSCRIBE, command post messages for PUBLISH, following on the two consumers subscribe to the same channel, otherwise a producer announced that , for example, achieved with Spring Boot + RedisTemplate functional and tested.

4.1 two producers

Definition of two producers, the producers in the two methods have a message consumer.

/**
 * 消费者一
 */
public class SubscribeOne {
    public void receive(String message) {
        System.out.println("这里是一号订阅客户端，接收到信息：" + message);
    }
}

/**
 * 消费者二
 */
public class SubscribeTwo {
    public void receive(String message) {
        System.out.println("这里是二号订阅客户端，接收到信息：" + message);
    }
}

4.2 message listener configuration

In this configuration, the container defines a message listener listening message with two adapters.

import com.bigsea.Controller.ChatController;
import com.bigsea.subscribe.SubscribeOne;
import com.bigsea.subscribe.SubscribeTwo;
import org.springframework.context.annotation.Bean;
import org.springframework.data.redis.connection.RedisConnectionFactory;
import org.springframework.data.redis.listener.PatternTopic;
import org.springframework.data.redis.listener.RedisMessageListenerContainer;
import org.springframework.data.redis.listener.adapter.MessageListenerAdapter;
import org.springframework.stereotype.Component;

@Component
public class subscribeConfig {
    private static final String RECEIVE_NAME = "receive";
    /**
     * 消息监听适配器一
     * @return MessageListenerAdapter
     */
    @Bean
    public MessageListenerAdapter listenerAdapterOne() {
        return new MessageListenerAdapter(new SubscribeOne(), RECEIVE_NAME);
    }

    /**
     * 消息监听适配器二
     * @return MessageListenerAdapter
     */
    @Bean
    public MessageListenerAdapter listenerAdapterTwo() {
        return new MessageListenerAdapter(new SubscribeTwo(), RECEIVE_NAME);
    }

    /**
     * 定义消息监听者容器
     * @param connectionFactory 连接工厂
     * @param listenerAdapterOne MessageListenerAdapter
     * @param listenerAdapterTwo MessageListenerAdapter
     * @return RedisMessageListenerContainer
     */
    @Bean
    public RedisMessageListenerContainer container(RedisConnectionFactory connectionFactory,
                                                   MessageListenerAdapter listenerAdapterOne,
                                                   MessageListenerAdapter listenerAdapterTwo) {
        RedisMessageListenerContainer listenerContainer = new RedisMessageListenerContainer();
        listenerContainer.setConnectionFactory(connectionFactory);
        listenerContainer.addMessageListener(listenerAdapterOne, new PatternTopic(ChatController.CHAT_NAME));
        listenerContainer.addMessageListener(listenerAdapterTwo, new PatternTopic(ChatController.CHAT_NAME));
        return listenerContainer;
    }
}

4.3 Producer

Producers have announced to "myMessage" channel, two consumers is also monitor this channel.

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.data.redis.core.StringRedisTemplate;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
@RequestMapping("/chat")
public class ChatController {
    @Autowired
    private StringRedisTemplate stringRedisTemplate;

    public static final String CHAT_NAME = "myMessage";

    private static final int MESSAGE_COUNT = 10;

    @GetMapping("/pub")
    public void publish() {
        for(int i = 1; i <= MESSAGE_COUNT; i++) {
            stringRedisTemplate.convertAndSend(CHAT_NAME, "发布的第" + i + "条消息");
        }
    }
}

4.4 Test Results

This started a Spring Boot application, directly in the browser through the URL to access the console print results are as follows.

这里是一号订阅客户端，接收到信息：发布的第1条消息
这里是一号订阅客户端，接收到信息：发布的第2条消息
这里是二号订阅客户端，接收到信息：发布的第2条消息
这里是二号订阅客户端，接收到信息：发布的第1条消息
这里是一号订阅客户端，接收到信息：发布的第3条消息
这里是二号订阅客户端，接收到信息：发布的第3条消息
这里是二号订阅客户端，接收到信息：发布的第4条消息
这里是一号订阅客户端，接收到信息：发布的第4条消息
这里是二号订阅客户端，接收到信息：发布的第5条消息
这里是一号订阅客户端，接收到信息：发布的第5条消息
这里是一号订阅客户端，接收到信息：发布的第6条消息
这里是二号订阅客户端，接收到信息：发布的第6条消息
这里是一号订阅客户端，接收到信息：发布的第7条消息
这里是二号订阅客户端，接收到信息：发布的第7条消息
这里是一号订阅客户端，接收到信息：发布的第8条消息
这里是二号订阅客户端，接收到信息：发布的第8条消息
这里是一号订阅客户端，接收到信息：发布的第9条消息
这里是二号订阅客户端，接收到信息：发布的第9条消息
这里是二号订阅客户端，接收到信息：发布的第10条消息
这里是一号订阅客户端，接收到信息：发布的第10条消息

Sea _sea

Published 48 original articles · won praise 66 · views 20000 +

Private letter concerns