Redis Journal - Distributed Lock

The original text is transferred from http://mzorro.me/2017/10/25/redis-distributed-lock/

At present, there are mainly three ways to realize distributed locks: database, Redis and Zookeeper. This article mainly describes the use of Redis related commands to realize distributed locks.

Related Redis Commands

SETNX

Set it to and return 1 if there is no value currently in it, otherwise return 0.

EXPIRE

Set to expire automatically after seconds.

GETSET

Sets the value of , and returns its old old value. Returns nil if there was no old value.

EVAL与EVALSHA

The function supported after Redis 2.6 can send a lua script to the Redis server to run.

Starting - A Preliminary Exploration of Distributed Locks

Using the atomicity of the SETNX command, we can simply implement a preliminary distributed lock (the principle will not be detailed here, just go to the pseudo code):

boolean tryLock(String key, int lockSeconds) {
  if (SETNX key "1" == 1) {
    EXPIRE key lockSeconds
    return true
  } else {
    return false
  }
}
boolean unlock(String key) {
  DEL key
}

tryLockis a non-blocking distributed lock method that returns immediately after failing to acquire the lock. If you need a blocking lock method, you can tryLockwrap the method as polling (it is important to poll at certain intervals, otherwise Redis will be overwhelmed!).

There seems to be no problem with this method, but there is actually a loophole: in the process of locking, the client sends the SETNX and EXPIRE commands to the Redis server in sequence, then assuming that after the SETNX command is executed, the EXPIRE command is issued. Before going out, the client crashes (or the network connection between the client and the Redis server is suddenly disconnected), resulting in the EXPIRE command not being executed, and other clients will have permanent deadlock!

Inheritance - Improvement of Distributed Locks

Update 2017-11-01:

There are loopholes in this method of unlocking, see the appendix at the end of this chapter for details.

In order to solve the problem raised above, you can store the lock expiration time (current client timestamp + lock time) in the key when locking, and then when the lock fails to acquire, take out the value and compare it with the current client time, if If it is determined that the lock has expired, you can confirm that the error condition described above has occurred. At this time, you can use DEL to clear the key, and then try to obtain the lock again. May I? Of course not! If there is no way to guarantee the atomicity between the DEL operation and the next SETNX operation, a race condition will still occur, such as this:

C1 DEL key
C1 SETNX key <expireTime>
C2 DEL key
C2 SETNX key <expireTime>

When the Redis server receives such an instruction sequence, the SETNX of C1 and C2 both return 1 at the same time. At this time, both C1 and C2 think that they have obtained the lock, which is obviously not in line with expectations.

To solve this problem, Redis' GETSET command comes in handy. The client can use the GETSET command to set its own expiration time, and then compare the returned value with the return value from the previous GET. If it is different, it means that the expired lock has been preempted by other clients (at this time, the GETSET command has actually been Effective, that is to say, the expiration time in the key has been modified, but this error is small and can be ignored).

According to the above analysis ideas, an improved distributed lock can be obtained. Here is the implementation code of Java:

public class RedisLock {
    private static final Logger logger = LoggerFactory.getLogger(RedisLock.class);
    private final StringRedisTemplate stringRedisTemplate;
    private final byte[] lockKey;
    public RedisLock(StringRedisTemplate stringRedisTemplate, String lockKey) {
        this.stringRedisTemplate = stringRedisTemplate;
        this.lockKey = lockKey.getBytes();
    }
    private boolean tryLock(RedisConnection conn, int lockSeconds) throws Exception {
        long nowTime = System.currentTimeMillis();
        long expireTime = nowTime + lockSeconds * 1000 + 1000; // 容忍不同服务器时间有1秒内的误差
        if (conn.setNX(lockKey, longToBytes(expireTime))) {
            conn.expire(lockKey, lockSeconds);
            return true;
        } else {
            byte[] oldValue = conn.get(lockKey);
            if (oldValue != null && bytesToLong(oldValue) < nowTime) {
                // 这个锁已经过期了，可以获得它
                // PS: 如果setNX和expire之间客户端发生崩溃，可能会出现这样的情况
                byte[] oldValue2 = conn.getSet(lockKey, longToBytes(expireTime));
                if (Arrays.equals(oldValue, oldValue2)) {
                    // 获得了锁
                    conn.expire(lockKey, lockSeconds);
                    return true;
                } else {
                    // 被别人抢占了锁(此时已经修改了lockKey中的值，不过误差很小可以忽略)
                    return false;
                }
            }
        }
        return false;
    }
    /**
     * 尝试获得锁，成功返回true，如果失败或异常立即返回false
     *
     * @param lockSeconds 加锁的时间(秒)，超过这个时间后锁会自动释放
     */
    public boolean tryLock(final int lockSeconds) {
        return stringRedisTemplate.execute(new RedisCallback<Boolean>() {
            @Override
            public Boolean doInRedis(RedisConnection conn) throws DataAccessException {
                try {
                    return tryLock(conn, lockSeconds);
                } catch (Exception e) {
                    logger.error("tryLock Error", e);
                    return false;
                }
            }
        });
    }
    /**
     * 轮询的方式去获得锁，成功返回true，超过轮询次数或异常返回false
     *
     * @param lockSeconds       加锁的时间(秒)，超过这个时间后锁会自动释放
     * @param tryIntervalMillis 轮询的时间间隔(毫秒)
     * @param maxTryCount       最大的轮询次数
     */
    public boolean tryLock(final int lockSeconds, final long tryIntervalMillis, final int maxTryCount) {
        return stringRedisTemplate.execute(new RedisCallback<Boolean>() {
            @Override
            public Boolean doInRedis(RedisConnection conn) throws DataAccessException {
                int tryCount = 0;
                while (true) {
                    if (++tryCount >= maxTryCount) {
                        // 获取锁超时
                        return false;
                    }
                    try {
                        if (tryLock(conn, lockSeconds)) {
                            return true;
                        }
                    } catch (Exception e) {
                        logger.error("tryLock Error", e);
                        return false;
                    }
                    try {
                        Thread.sleep(tryIntervalMillis);
                    } catch (InterruptedException e) {
                        logger.error("tryLock interrupted", e);
                        return false;
                    }
                }
            }
        });
    }
    /**
     * 如果加锁后的操作比较耗时，调用方其实可以在unlock前根据时间判断下锁是否已经过期
     * 如果已经过期可以不用调用，减少一次请求
     */
    public void unlock() {
        stringRedisTemplate.delete(new String(lockKey));
    }
    public byte[] longToBytes(long value) {
        ByteBuffer buffer = ByteBuffer.allocate(Long.SIZE / Byte.SIZE);
        buffer.putLong(value);
        return buffer.array();
    }
    public long bytesToLong(byte[] bytes) {
        if (bytes.length != Long.SIZE / Byte.SIZE) {
            throw new IllegalArgumentException("wrong length of bytes!");
        }
        return ByteBuffer.wrap(bytes).getLong();
    }
}

Turn - Optimization of Distributed Locks

Update 2017-11-01:

There are loopholes in this method of unlocking, see the appendix at the end of this chapter for details.

The above distributed lock implementation logic is relatively complex, involving more Redis commands, and each time the process of trying to lock will have at least 2 Redis commands executed, which means at least twice with the Redis server. Telecommunication. The reason for adding the complex logic behind is only because the atomicity of the execution of the two commands SETNX and EXPIRE cannot be guaranteed. (Some students will mention the pipeline feature of Redis, which is obviously not applicable here, because the pipeline cannot be realized since the execution of the second instruction and the result of the first execution)

In addition, the above distributed lock also has a problem, that is, the problem of time synchronization between servers. In a distributed scenario, it is very difficult to synchronize the time between multiple servers, so I added 1 second time fault tolerance in the code, but it may be unreliable to rely on the synchronization of server time.

Starting from Redis 2.6, the client can submit Lua scripts directly to the Redis server, which means that some more complex logic can be executed directly on the Redis server, and the submission of this script is relatively atomic for the client. This just happened to solve our problem!

We can use a lua script like this to describe the locking logic (see here for the script submission command and Redis related rules ):

if (redis.call('setnx', KEYS[1], ARGV[1]) == 1) then
    redis.call('expire', KEYS[1], tonumber(ARGV[2]))
    return true
else
    return false
end

Note: The execution of the commands in this script is not strictly atomic. If the execution of the second instruction EXPIRE fails, the execution of the entire script will return an error, but the first instruction SETNX is still in effect! However, in this case, it can basically be considered that the Redis server has crashed (unless it is a problem such as a parameter error that can be excluded in the development stage), then the security of the lock is no longer a concern here. Here it is considered that relatively atomic to the client is sufficient.

This simple script is executed on the Redis server and returns whether the lock is obtained. Because there is only one Redis command for script submission and execution, the client exception problem mentioned above is avoided.

The logic and performance of the lock are optimized using scripts. Here is the final Java implementation code:

public class RedisLock {
    private static final Logger logger = LoggerFactory.getLogger(RedisLock.class);
    private final StringRedisTemplate stringRedisTemplate;
    private final String lockKey;
    private final List<String> keys;
    /**
     * 使用脚本在redis服务器执行这个逻辑可以在一定程度上保证此操作的原子性
     * （即不会发生客户端在执行setNX和expire命令之间，发生崩溃或失去与服务器的连接导致expire没有得到执行，发生永久死锁）
     * <p>
     * 除非脚本在redis服务器执行时redis服务器发生崩溃，不过此种情况锁也会失效
     */
    private static final RedisScript<Boolean> SETNX_AND_EXPIRE_SCRIPT;
    static {
        StringBuilder sb = new StringBuilder();
        sb.append("if (redis.call('setnx', KEYS[1], ARGV[1]) == 1) then\n");
        sb.append("\tredis.call('expire', KEYS[1], tonumber(ARGV[2]))\n");
        sb.append("\treturn true\n");
        sb.append("else\n");
        sb.append("\treturn false\n");
        sb.append("end");
        SETNX_AND_EXPIRE_SCRIPT = new RedisScriptImpl<Boolean>(sb.toString(), Boolean.class);
    }
    public RedisLock(StringRedisTemplate stringRedisTemplate, String lockKey) {
        this.stringRedisTemplate = stringRedisTemplate;
        this.lockKey = lockKey;
        this.keys = Collections.singletonList(lockKey);
    }
    private boolean doTryLock(int lockSeconds) throws Exception {
        return stringRedisTemplate.execute(SETNX_AND_EXPIRE_SCRIPT, keys, "1", String.valueOf(lockSeconds));
    }
    /**
     * 尝试获得锁，成功返回true，如果失败立即返回false
     *
     * @param lockSeconds 加锁的时间(秒)，超过这个时间后锁会自动释放
     */
    public boolean tryLock(int lockSeconds) {
        try {
            return doTryLock(lockSeconds);
        } catch (Exception e) {
            logger.error("tryLock Error", e);
            return false;
        }
    }
    /**
     * 轮询的方式去获得锁，成功返回true，超过轮询次数或异常返回false
     *
     * @param lockSeconds       加锁的时间(秒)，超过这个时间后锁会自动释放
     * @param tryIntervalMillis 轮询的时间间隔(毫秒)
     * @param maxTryCount       最大的轮询次数
     */
    public boolean tryLock(final int lockSeconds, final long tryIntervalMillis, final int maxTryCount) {
        int tryCount = 0;
        while (true) {
            if (++tryCount >= maxTryCount) {
                // 获取锁超时
                return false;
            }
            try {
                if (doTryLock(lockSeconds)) {
                    return true;
                }
            } catch (Exception e) {
                logger.error("tryLock Error", e);
                return false;
            }
            try {
                Thread.sleep(tryIntervalMillis);
            } catch (InterruptedException e) {
                logger.error("tryLock interrupted", e);
                return false;
            }
        }
    }
    /**
     * 如果加锁后的操作比较耗时，调用方其实可以在unlock前根据时间判断下锁是否已经过期
     * 如果已经过期可以不用调用，减少一次请求
     */
    public void unlock() {
        stringRedisTemplate.delete(lockKey);
    }
    private static class RedisScriptImpl<T> implements RedisScript<T> {
        private final String script;
        private final String sha1;
        private final Class<T> resultType;
        public RedisScriptImpl(String script, Class<T> resultType) {
            this.script = script;
            this.sha1 = DigestUtils.sha1DigestAsHex(script);
            this.resultType = resultType;
        }
        @Override
        public String getSha1() {
            return sha1;
        }
        @Override
        public Class<T> getResultType() {
            return resultType;
        }
        @Override
        public String getScriptAsString() {
            return script;
        }
    }
}

Combined - subsection

In the end, the content of this article is only the result of the author's own learning and tossing. If there are any bugs that the author has not considered, please feel free to point it out. Let's learn and make progress together~

Chase - Unlock Vulnerabilities (Updated 2017-11-01)

After careful consideration, it is found that the distributed lock implemented above has a relatively serious unlocking vulnerability: because the unlocking operation is only simple DEL KEY, if a client executes business after obtaining the lock for more than the expiration time of the lock, the last Unlock operations can misinterpret other clients' operations.

To solve this problem, we RedisLockuse the native timestamp and UUID to create an absolutely unique object when creating an object lockValue, then store this value when locking, and GETcompare it with the fetched value before unlocking, and only do it if there is a match DEL. It is still necessary to use the LUA script to ensure the atomicity of the entire unlocking process.

Here is the code after fixing this vulnerability and making some small optimizations:

import java.util.Collections;
import java.util.UUID;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.data.redis.core.StringRedisTemplate;
import org.springframework.data.redis.core.script.DigestUtils;
import org.springframework.data.redis.core.script.RedisScript;
/**
 * Created On 10/24 2017
 * Redis实现的分布式锁(不可重入)
 * 此对象非线程安全，使用时务必注意
 */
public class RedisLock {
    private static final Logger logger = LoggerFactory.getLogger(RedisLock.class);
    private final StringRedisTemplate stringRedisTemplate;
    private final String lockKey;
    private final String lockValue;
    private boolean locked = false;
    /**
     * 使用脚本在redis服务器执行这个逻辑可以在一定程度上保证此操作的原子性
     * （即不会发生客户端在执行setNX和expire命令之间，发生崩溃或失去与服务器的连接导致expire没有得到执行，发生永久死锁）
     * <p>
     * 除非脚本在redis服务器执行时redis服务器发生崩溃，不过此种情况锁也会失效
     */
    private static final RedisScript<Boolean> SETNX_AND_EXPIRE_SCRIPT;
    static {
        StringBuilder sb = new StringBuilder();
        sb.append("if (redis.call('setnx', KEYS[1], ARGV[1]) == 1) then\n");
        sb.append("\tredis.call('expire', KEYS[1], tonumber(ARGV[2]))\n");
        sb.append("\treturn true\n");
        sb.append("else\n");
        sb.append("\treturn false\n");
        sb.append("end");
        SETNX_AND_EXPIRE_SCRIPT = new RedisScriptImpl<Boolean>(sb.toString(), Boolean.class);
    }
    private static final RedisScript<Boolean> DEL_IF_GET_EQUALS;
    static {
        StringBuilder sb = new StringBuilder();
        sb.append("if (redis.call('get', KEYS[1]) == ARGV[1]) then\n");
        sb.append("\tredis.call('del', KEYS[1])\n");
        sb.append("\treturn true\n");
        sb.append("else\n");
        sb.append("\treturn false\n");
        sb.append("end");
        DEL_IF_GET_EQUALS = new RedisScriptImpl<Boolean>(sb.toString(), Boolean.class);
    }
    public RedisLock(StringRedisTemplate stringRedisTemplate, String lockKey) {
        this.stringRedisTemplate = stringRedisTemplate;
        this.lockKey = lockKey;
        this.lockValue = UUID.randomUUID().toString() + "." + System.currentTimeMillis();
    }
    private boolean doTryLock(int lockSeconds) throws Exception {
        if (locked) {
            throw new IllegalStateException("already locked!");
        }
        locked = stringRedisTemplate.execute(SETNX_AND_EXPIRE_SCRIPT, Collections.singletonList(lockKey), lockValue,
                                             String.valueOf(lockSeconds));
        return locked;
    }
    /**
     * 尝试获得锁，成功返回true，如果失败立即返回false
     *
     * @param lockSeconds 加锁的时间(秒)，超过这个时间后锁会自动释放
     */
    public boolean tryLock(int lockSeconds) {
        try {
            return doTryLock(lockSeconds);
        } catch (Exception e) {
            logger.error("tryLock Error", e);
            return false;
        }
    }
    /**
     * 轮询的方式去获得锁，成功返回true，超过轮询次数或异常返回false
     *
     * @param lockSeconds       加锁的时间(秒)，超过这个时间后锁会自动释放
     * @param tryIntervalMillis 轮询的时间间隔(毫秒)
     * @param maxTryCount       最大的轮询次数
     */
    public boolean tryLock(final int lockSeconds, final long tryIntervalMillis, final int maxTryCount) {
        int tryCount = 0;
        while (true) {
            if (++tryCount >= maxTryCount) {
                // 获取锁超时
                return false;
            }
            try {
                if (doTryLock(lockSeconds)) {
                    return true;
                }
            } catch (Exception e) {
                logger.error("tryLock Error", e);
                return false;
            }
            try {
                Thread.sleep(tryIntervalMillis);
            } catch (InterruptedException e) {
                logger.error("tryLock interrupted", e);
                return false;
            }
        }
    }
    /**
     * 解锁操作
     */
    public void unlock() {
        if (!locked) {
            throw new IllegalStateException("not locked yet!");
        }
        locked = false;
        // 忽略结果
        stringRedisTemplate.execute(DEL_IF_GET_EQUALS, Collections.singletonList(lockKey), lockValue);
    }
    private static class RedisScriptImpl<T> implements RedisScript<T> {
        private final String script;
        private final String sha1;
        private final Class<T> resultType;
        public RedisScriptImpl(String script, Class<T> resultType) {
            this.script = script;
            this.sha1 = DigestUtils.sha1DigestAsHex(script);
            this.resultType = resultType;
        }
        @Override
        public String getSha1() {
            return sha1;
        }
        @Override
        public Class<T> getResultType() {
            return resultType;
        }
        @Override
        public String getScriptAsString() {
            return script;
        }
    }
}