Hello everyone, today I will share a problem encountered in the process of using redis lua script. The problem is not difficult, but it is easy to step on the pit.
How to use lua script
java
copy code
// 定义脚本资源 DefaultRedisScript redisScript = new DefaultRedisScript<>(); redisScript.setScriptSource(new ResourceScriptSource(new ClassPathResource(SCRIPT_PATH + scriptName))); redisScript.setResultType(List.class); // 预加载脚本（可选） redisTemplate.getConnectionFactory().getClusterConnection().scriptLoad(redisScript.getScriptAsString().getBytes()); // 执行脚本 stringRedisTemplate.execute(redisScript, keys, args);
Redis script related command description
- script load: load the lua script into the script cache of redis, return the sha1 checksum of the script, and then use the checksum to call the script through the evalsha command.
- evalsha: Execute the loaded lua script according to sha1.
- eval: Execute a lua script code, and the script will be cached in the redis script cache after execution.
- script exists: Check whether the script already exists in the script cache according to sha1.
- script flush: Clear the redis script cache and delete all loaded lua scripts.
- script kill: kill the lua script being executed.
execute method execution process
The reidsTemplate holds a ScriptExecutor, and the final execution is delegated to the ScriptExecutor. The ScriptExecutor will execute the script on the redis server through the evalsha command.
If the lua script has been preloaded through the script load command before, evalsha will execute normally; if the script is not loaded in advance and the script is executed for the first time, evalsha will return "NOSCRIPT No matching script. Please use EVAL." Abnormal, the The exception will be encapsulated into RedisSystemException thrown.
After catching the exception, judge if the exception type is NonTransientDataAccessException, and the exception information contains the "NOSCRIPT" keyword, then pass the complete script through the eval command to execute once, and the script will be cached after execution, and each subsequent call only needs to pass evalsha The command can be executed by passing the sha1.
Problems encountered in the project
In the project I am in charge of, there is a lua script used to limit the frequency of SMS sending. After the service is deployed to a new environment, it is found that the request error "NOSCRIPT No matching script. Please use EVAL." According to the above introduction, the error means The redis server cannot find the corresponding script through the passed sha1.
The current practice of this service is that the script is not preloaded through script load in advance, but the loading operation is done by the first request through lazy loading. Because the redis cluster in this new environment is also newly built, this script must not be cached, but according to the above analysis, eval will be executed after the first evalsha request fails.
So it can be inferred that the exception type is not NonTransientDataAccessException, or the "NOSCRIPT" keyword is not included in the exception information, causing the exception to be thrown directly.
After investigation, it was found that the problem was caused by accessing the sentinel-redis flow control function in the first two weeks.
xml
copy code
<dependency> <groupId>com.xxx</groupId> <artifactId>xxx-sentinel-spring-boot-starter-redis</artifactId> <version>${xxx-sentinel.version}</version> </dependency>
This module will intercept every redis command, and then wrap it through sentinel flow control api.
The actual command is executed reflectively via method.invoke(). If there is an exception inside the execution, an InvocationTargetException will be thrown.
In summary, the "NOSCRIPT" RedisSystemException thrown by the first execution of the evalsha command is wrapped into an InvocationTargetException exception, so the judgment here returns false directly, causing the exception to be thrown directly, and the subsequent eval command is not executed.
How to intercept redis commands
We know that redis commands are executed through the RedisConnection object, and RedisConnection is obtained from RedisConnectionFactory.
RedisConnectionFactory generally has two implementations: jedis and lettuce.
The specific implementation of RedisConnectionFactory is loaded into the Spring container through SpringBoot automatic assembly.
So we can get RedisConnection by intercepting the getConnection method of RedisConnectionFactory, and then proxy RedisConnection once, so that all redis commands can go to our own interceptor.
Solution
Back to the topic, how do we solve this problem?
-
It is best to preload the lua script through script load after the service starts.
-
After intercepting redis commands (not limited to), it is best to return the original exception.
Summarize
-
This problem is quite pitiful, and it is not easy to reproduce. Before migrating to the new environment, this problem has never occurred. The main reason is that the sentine-redis package was introduced recently. Regardless of the dev, test, and prod environments, lua scripts are actually early. It has already been cached to the redis server, and it cannot go to the logic of the first loading.
-
Before intercepting and extending the execution process of various components, you need to carefully look at the original execution process to see if there is any special handling for exceptions, and it is best to return the original exception.
Personal open source project
DynamicTp is a lightweight dynamic thread pool management tool based on the configuration center. Its main functions can be summarized into several categories, such as dynamic parameter adjustment, notification and alarm, operation monitoring, and tripartite package thread pool management.
At present, the total number of stars is 4.3k. Welcome everyone to try it out. Thank you for your star. Welcome to pr. Apart from business, we will contribute to open source together.