redis note 2

Distributed Lock implementation

Lock is used to solve any problem;

1. Multiple threads in a process, multiple threads concurrently access the same resources when and how to solve thread safety issues.
2. A distributed architecture system both modules to access a file to read and write files
3. Multiple applications to do the same when the data changes, how to ensure the security of data

In a single process, we can use synchronized, lock like synchronous operation to solve, but for the case of multiple processes in a distributed architecture, how to do cross-process lock. They need the help of some third-party tools to complete

A distributed design problem to be solved

Distributed Lock Solutions

1. How to acquire a lock

Database, the only constraint by

lock(

  id  int(11)

  methodName  varchar(100),

  memo varchar(1000)

  modifyTime timestamp

 unique key mn (method) - the only constraint

)

Get lock pseudocode

try{

exec  insert into lock(methodName,memo) values(‘method’,’desc’);    method

return true;

}Catch(DuplicateException e){

return false;

}

Release the lock

delete from lock where methodName=’’;

Problems need to think about

1. Lock no expiration time, once the unlock operation fails, it will lead to a lock has been recorded in the database, other threads can not get re-lock
2. It is non-blocking lock, insert data operation, once the error will directly insert fails. Thread does not get the lock and will not enter the queue queue, to get going again trigger lock to obtain a lock operation again
3. Non-reentrant lock, the same thread can not get the lock again before the lock is not released

zookeeper achieve Distributed Lock

Using a unique node zookeeper ordered temporary characteristic or characteristic obtained node as the lowest node lock. Zookeeper relatively simple to achieve, by curator client, for the lock operation has been packaged, it works as follows

 

 

zookeeper advantage

1. High reliability, simple

2. zookeeper because the characteristics of the temporary node, if for other clients as abnormal and zookeeper connection is interrupted, then the node will be deleted, meaning that the lock will be released automatically

3. zookeeper itself provides a good cluster solution, stable

4. Release the lock operation, will watch notification mechanism, that is, the server will take the initiative to send a message to the client that the lock has been released

Based on a distributed cache lock

redis setNx has a command, which only set the key value in the case where the key does not exist. So you can use this feature to implement distributed lock operation

Specific implementation code

1. Add dependencies

 

1. Write redis connection code

 

Releases the lock code

 

1. Realization of distributed lock

 

1. How to release the lock

 

redis multiplexing mechanism

linux kernel will all external devices to operate as a file, read and write operations on the system command calls the kernel to provide a file and returns a file descriptor (file descriptor). For a socket descriptor will have read and write response, called socketfd (socket descriptor). The IO multiplexing means to inform the kernel once the process one or more file descriptors IO process conditions specified ready to discover later

IO multiplexing, also known as event-driven operating system provides a function, when a socket readable or writable time, it will give a notification. When used with a non-blocking socket used only when the system inform me which descriptor is readable, and I was to perform read operations, you can ensure that each read can read valid data. Operating system function call by select / pool / epoll / kqueue such system functions used, these functions can monitor multiple descriptors read readiness simultaneously, so that a plurality of descriptors of I / O operations can be in a concurrent alternately thread of completion, which is called I / O multiplexer, multiplexing herein refers to a thread with the same

Multiplexing the advantage that the user can simultaneously process a plurality of socket io request in a thread. Achieve the same thread handle multiple IO requests. In synchronous blocking model, in order to achieve their goals must be multi-threaded mode

 

redis use lua script

lua script

Lua is an efficient lightweight scripting language, written in standard C language source code form and is open, which is designed for embedded applications, thereby providing a flexible expansion and customization of applications

The benefits of using a script

1. Reduce network overhead, in Lua script can put multiple commands in the same script runs
2. Atomic operation, the entire script Redis will perform as a whole, can not be inserted into the middle of other commands. In other words, the process of writing the script race conditions there will be no need to worry
3. Reusability, the script will always be stored in redis sent by the client, which means that other clients can reuse this script to accomplish the same logic

Lua installed in the linux

Tar.gz to the official website to download the source package lua

Contact-5.3.0.tar.gz tar -zxvf

Into the extracted directory:

cd-moon 5.2.0

make linux (linux environment compiler)

make install

If errors are reported, that can not find readline / readline.h, can be installed via yum command

yum -y install readline-devel ncurses-devel

Then make linux / make install After the installation,

Finally, you can enter commands directly into lua lua console

lua syntax

 

Redis and Lua

Redis commands to call in Lua script, you can use redis.call function call. For example, we call the string type of command

redis.call(‘set’,’hello’,’world’)

redis.call function's return value is the result of the implementation of redis command. Earlier we introduced the type of value 5 in the type of data returned redis are not the same. redis.call This function will return the value of five types of conversion data corresponding to the type of Lua

The return value obtained from the Lua script

In many cases we need a script can return a value, you can use the return statement in the script to return a value to the redis client is performed by the return statement, if there is no return execution, default return is nil.

How to execute lua scripts in redis

Redis provides EVAL command can make calls to other developers like Redis built-in command as the calling script.

[EVAL] [script content] [number of key parameters] [key ...] [arg ...]

Data can be passed to the script by key and arg these two parameters, their values, respectively, can be used in scripts KEYS and ARGV these two types of access a global variable. For example, we achieved a set script command, by redis client calls, then the statement is executed:

Content lua script is: return redis.call ( 'set', KEYS [1], ARGV [1]) // KEYS and ARGV must be capitalized.

eval "return redis.call('set',KEYS[1],ARGV[1])" 1 hello world

EVAL command key parameter is the number of - in the above example is 1 to all the parameters are stored in a script later KEYS ARGV and two types of global variable table. When the script without any parameters can not omit this parameter. If there is no argument or 0

eval "return redis.call(‘get’,’hello’)" 0

EVALSHA command

Considering the situation we execute lua script eval, under a long script, each call script the entire script need to pass redis, compare bandwidth. To solve this problem, redis provides EVALSHA command allows developers to SHA1 digest the contents of a script to execute the script. Usage of the command and EVAL, in that it is the content of the script replace the contents of the script SHA1 digest

 

1. Redis will be calculated at the time of script execution EVAL command SHA1 digest and recorded in the script cache
2. Redis will look for when performing EVALSHA command from the script according to a summary provided by the cache contents corresponding script, the script is executed if found, otherwise "NOSCRIPT No matching script, Please use EVAL"

 

EVALSHA order to demonstrate the effect of the following cases

script load "return redis.call ( 'get', 'hello')" will be added to the cache and generates a script command sha1

evalsha "a5a402e90df3eaeca2ff03d56d99982e05cf6574" 0

Before we call the eval command, perform evalsha command prompt if the script does not exist, then call eval command

lua script combat

Achieve a frequency for access to a phone number of times, the following is a lua script, save it as phone_limit.lua

local num=redis.call('incr',KEYS[1])

if tonumber(num)==1 then

   redis.call('expire',KEYS[1],ARGV[1])

   return 1

elseif tonumber(num)>tonumber(ARGV[2]) then

   return 0

else

   return 1

end

Call the following command

./redis-cli --eval phone_limit.lua rate.limiting:13700000000 , 10 3

 

The syntax is ./redis-cli -eval [lua script] [key ...] space, space [args ...]

Atomic script

Redis script execution is atomic, that Redis during script execution will not execute other commands. All commands must wait for the script After executing to perform. In order to prevent a script execution time of the process leading to Redis can not provide services. Redis provides a lua-time-limit parameter limits the maximum script running time. The default is 5 seconds.

When the script runs longer than this limit, Redis will begin to accept other commands but does not execute (atomicity to ensure that the script), but returns an error BUSY

Hands-on

Open two client window

Lua script execution in the first window in an infinite loop

eval “while true do end” 0

 

Get hello to run in a second window

 

Final results of the second run of the window is Busy, you can terminate the script is executed by the script kill command. If lua script currently executing on data redis were modified, such as (set) operation, then the script kill command can not terminate the operation of the script, because to ensure atomicity lua scripts. If the execution part of the termination, on the contrary to this principle

In this case, only through the shutdown nosave forced termination command

 

java code

RedisManager.java

import redis.clients.jedis.Jedis;
import redis.clients.jedis.JedisPool;
import redis.clients.jedis.JedisPoolConfig;

public class RedisManager {

	private static JedisPool jedisPool;

	static {
		JedisPoolConfig jedisPoolConfig = new JedisPoolConfig();
		jedisPoolConfig.setMaxTotal(20);
		jedisPoolConfig.setMaxIdle(10);
		jedisPool = new JedisPool(jedisPoolConfig, "120.79.174.118", 6379);
	}

	public static Jedis getJedis() throws Exception {
		if (null != jedisPool) {
			return jedisPool.getResource();
		}

		throw new Exception("Jedispool was not init");
	}
}

　　

RedisLock.java simple implementation of distributed lock

java.util.List Import; 
Import java.util.UUID; 
Import redis.clients.jedis.Jedis; 
Import redis.clients.jedis.Transaction; 

public class RedisLock { 

	public String a GETLOCK (String Key, int timeout) { 
		the try { 
			Jedis = RedisManager.getJedis jedis (); 
			String value = UUID.randomUUID () toString ();. 
			Long End = System.currentTimeMillis () + timeout; 
			the while (System.currentTimeMillis () <End) { 
				IF (jedis.setnx (Key , value) ==. 1) { 
					// set successful lock, the operation is successful Redis 
					jedis.expire (Key, timeout); 
					return value; 
				} 
				IF (jedis.ttl (Key) == -1) { 
					// the detection time expires, if not already set  
					jedis.expire (key, timeout);
				} 
				the Thread.sleep (1000);
			} 
		} The catch (Exception E) { 
			e.printStackTrace (); 
		} 
		return null; 
	} 

	public Boolean RELEASELOCK (String Key, String value) { 
		the try { 
			Jedis jedis RedisManager.getJedis = (); 
			the while (to true) { 
				jedis.watch ( Key); // Watch 
				IF (value.equals (jedis.get (Key))) {// get the lock thread is determined and stored in redis current is the same lock 
					the Transaction Transaction jedis.multi = (); 
					transaction.del (Key); 
					List <Object> = transaction.exec List (); 
					IF (List == null) { 
						Continue; 
					} 
					return to true; 
				} 
				jedis.unwatch (); 
				BREAK; 
			}
		The catch} (Exception E) { 
		IF (RET) {
			e.printStackTrace (); 
		} 
		return to false; 
	} 

	public static void main (String [] args) { 
		String Key = "AAA"; 
		RedisLock redisLock new new RedisLock = (); 
		String LOCKID = redisLock.getLock (Key, 10000); 
		IF (! LOCKID = null) { 
			System.out.println ( "success obtained lock"); 
		} the else { 
			System.out.println ( "failure to obtain a lock"); 
		} 
		String lockId2 = redisLock.getLock (Key, 10000); 
		IF (! lockId2 = null) { 
			System.out.println ( "success obtained lock"); 
		} the else { 
			System.out.println ( "failure to obtain a lock"); 
		} 

		Boolean RET = redisLock.releaseLock (Key, LOCKID);
		} else {
			System.out.println ( "successful release lock");
			System.out.println ( "lock release failed"); 
		} 

		String lockId3 = redisLock.getLock (Key, 10000); 
		IF (! LockId3 = null) { 
			System.out.println ( "success obtained lock"); 
		} the else { 
			System.out.println ( "failure to obtain a lock"); 
		} 

		Boolean RET2 = redisLock.releaseLock (Key, lockId3); 
		IF (RET2) { 
			System.out.println ( "successful release lock"); 
		} {the else 
			the System.out .println ( "lock release failure"); 
		} 
	} 
}

　　

LuaDemo.java lua script execution

import java.util.ArrayList;
import java.util.List;

import redis.clients.jedis.Jedis;

public class LuaDemo {
	public static void main(String[] args) throws Exception {
		Jedis jedis = RedisManager.getJedis();
		
		String lua="local num=redis.call('incr',KEYS[1])\n"+
					"if tonumber(num)==1 then\n"+
					"  redis.call('expire',KEYS[1],ARGV[1])\n"+
					"  return 1\n"+
					"elseif tonumber(num)>tonumber(ARGV[2]) then\n"+
					"  return 0\n"+
					"else\n"+
					"  return 1\n"+
					"end";
		
		List<String> keys=new ArrayList<>();
		keys.add("ip:limit:127.0.0.1");
		List<String> arggs=new ArrayList<>();
		arggs.add("6000");
		arggs.add("10");
		Object obj=jedis.eval(lua,keys,arggs);
		System.out.println(obj);
		
	}
}

　　

LuaDemo2.java by sha summary cache lua script

import java.util.ArrayList;
import java.util.List;
import redis.clients.jedis.Jedis;

public class LuaDemo2 {
	public static void main(String[] args) throws Exception {
		Jedis jedis = RedisManager.getJedis();

		String lua="local num=redis.call('incr',KEYS[1])\n"+
				"if tonumber(num)==1 then\n"+
				"  redis.call('expire',KEYS[1],ARGV[1])\n"+
				"  return 1\n"+
				"elseif tonumber(num)>tonumber(ARGV[2]) then\n"+
				"  return 0\n"+
				"else\n"+
				"  return 1\n"+
				"end";

		List<String> keys = new ArrayList<>();
		keys.add("ip:limit:127.0.0.1");

		List<String> arggs = new ArrayList<>();
		arggs.add ( "6000");
		arggs.add("10");
		// cache by sha summary lua script, reduce network traffic and improve performance. (redis sha summary to restart the cache will be lost) 
		String SHA = jedis.scriptLoad (Lua); 
		System.out.println (SHA); 
		Object obj = jedis.evalsha (SHA, Keys, arggs); 
		System.out.println (obj ); 

	} 
}

　　

Configuring maven

      <dependency>
          <groupId>redis.clients</groupId>
          <artifactId>jedis</artifactId>
          <version>2.9.0</version>
      </dependency>
      <dependency>
          <groupId>org.apache.commons</groupId>
          <artifactId>commons-pool2</artifactId>
          <version>2.4.3</version>
      </dependency>