The use of redis as a message queue

=========================== 2017/3/31 Update =================== =====

Redisson implements a distributed and scalable java data structure. The supported data structures are: List, Set, Map, Queue, SortedSet, ConcureentMap, Lock, AtomicLong, CountDownLatch. And it is thread-safe, and the bottom layer uses Netty 4 to implement network communication. Compared with jedis, the function is relatively simple, and it does not support redis features such as sorting, transactions, pipelines, and partitioning. It can be considered as a supplement to jedis and cannot be replaced by jedis.

resolution.getQueue supports queues

==============================================================

 

Among the data structures supported by redis, one is the collection list. Common operations on the list include lpush lrange and so on. For this common operation, we use sets as 'sets' in the typical sense. Often overlooked is the use of List as a "queue".

Decompile the redis jar package and you will find:

public String rpop(String key)
  {
    checkIsInMulti();
    this.client.rpop(key);
    return this.client.getBulkReply();
  }

public String rpoplpush(String srckey, String dstkey)
  {
    checkIsInMulti();
    this.client.rpoplpush(srckey, dstkey);
    return this.client.getBulkReply();
  }

public String lpop(String key)
  {
    checkIsInMulti();
    this.client.lpop(key);
    return this.client.getBulkReply();
  }

 pop means "bounce" and is the element taken out of the queue. rpop means "right pop", which means to take elements from the right side of the queue, and lpop means "left pop" of course, which means to take elements from the left side of the queue. Corresponding to taking elements, we also push elements in the queue in two directions:

public Long lpush(String key, String[] strings)
  {
    checkIsInMulti();
    this.client.lpush(key, strings);
    return this.client.getIntegerReply();
  }

public Long rpush(String key, String[] strings)
  {
    checkIsInMulti();
    this.client.rpush(key, strings);
    return this.client.getIntegerReply();
  }

 Well, let's officially start to test the use of the List data structure in redis as a queue:

Suppose I need to push the map to the queue, how should I do it? If you push the String value, the price comparison is simple, so I won't study it here...

 

There are two ways to push map to the queue:

The first:

Push a map to the queue:

class PushThread implements Runnable {
	
	@Override
	public void run() {
		Jedis jedis = null;
		try{
			jedis = JedisPoolUtils.getJedis();
			
			for(int i=0; i<10; i++) {
				Map map = new HashMap();
				map.put("test", "test" + i);
				map.put("ok", "ok" + i);
				jedis.lpush("test".getBytes(), ObjectUtils.objectToBytes(map));//The method of converting Object objects into byte arrays, omitted here
			}	
			
			
		} catch(Exception e) {
			e.printStackTrace ();
		} finally{
			JedisPoolUtils.returnRes(jedis);
		}
	}
	
}

 Remove the map from the queue:

class PopThread implements Runnable {

	@Override
	public void run() {
		Jedis jedis = null;
		try{
			jedis = JedisPoolUtils.getJedis();
			
			for(int i=0; i<10; i++) {
				byte[] key = jedis.rpop("test".getBytes());
				System.out.println("Pop up: " + ObjectUtils.bytesToObject(key));//The method of converting byte array to Object object, omitted here

			}	
			
			
		} catch(Exception e) {
			e.printStackTrace ();
		} finally{
			JedisPoolUtils.returnRes(jedis);
		}
	}

 Start the Push thread and the pop thread in turn, and you can see the print result on the pop thread side:

Popup: {ok=ok0, test=test0}

Popup: {ok=ok1, test=test1}

Popup: {ok=ok2, test=test2}

Popup: {ok=ok3, test=test3}

Popup: {ok=ok4, test=test4}

Popup: {ok=ok5, test=test5}

Popup: {ok=ok6, test=test6}

Popup: {ok=ok7, test=test7}

Popup: {ok=ok8, test=test8}

Popup: {ok=ok9, test=test9}

 

The second way:

因为第一种方式毕竟需要在byte数组和Object对象之间来回转换，效率上可能存在一定影响。所以我们加以改进。不妨只把map的key放在一个队列中，取队列元素时，只取key,然后根据key取map即可。

push:

class PushThread implements Runnable {
	
	@Override
	public void run() {
		Jedis jedis = null;
		try{
			jedis = JedisPoolUtils.getJedis();
			
			for(int i=0; i<10; i++) {
				Map map = new HashMap();
				map.put("test", "test" + i);
				map.put("ok", "ok" + i);
				String key = "map" + i;
				jedis.lpush("test", key);//把所有map的key放入一个队列test中
				jedis.hmset(key, map);
			}	
			
			
		} catch(Exception e) {
			e.printStackTrace();
		} finally{
			JedisPoolUtils.returnRes(jedis);
		}
}

 pop元素：

class PopThread implements Runnable {

	@Override
	public void run() {
		Jedis jedis = null;
		try{
			jedis = JedisPoolUtils.getJedis();
			
			for(int i=0; i<10; i++) {
				String key = jedis.rpop("test");
				System.out.println("弹出：" + jedis.hgetAll(key));
			}	
			
			
		} catch(Exception e) {
			e.printStackTrace();
		} finally{
			JedisPoolUtils.returnRes(jedis);
		}
	}
	
}

  依次启动Push线程和pop线程，可在pop线程端看到打印结果：

弹出：{test=test0, ok=ok0}

弹出：{test=test1, ok=ok1}

弹出：{test=test2, ok=ok2}

弹出：{test=test3, ok=ok3}

弹出：{test=test4, ok=ok4}

弹出：{test=test5, ok=ok5}

弹出：{test=test6, ok=ok6}

弹出：{test=test7, ok=ok7}

弹出：{test=test8, ok=ok8}

弹出：{test=test9, ok=ok9}

 

两种方式效果一样，只不过第二种效率上更好一些。

注：当你依次从队列里取出元素后，队列在redis中就不存在了，所以当你pop完元素，再尝试pop的话，会报异常：

redis.clients.jedis.exceptions.JedisDataException: value sent to redis cannot be null

同时，redis中这个队列也不复存在了！

 

利用redis作为队列的性质我们可以用它来达到类似ZMQ的作用。。