Lua script
Perform a complicated operation on the server side
Additional features learned previously
Pipeline: Packing send multiple commands, and receiving a reply to which all the results of the executed command.
Services: first execution of multiple commands, the command is executed or to be executed all, or to a not executed. And not be interrupted by other tasks during the transaction.
Optimistic locking: monitor specific keys to prevent the emergence of competitive conditions Affairs.
Although these additional features are very useful, but they also have some drawbacks.
Pipeline defects
Although the use of the pipeline to send multiple commands at once, but for a complex operation consisting of a plurality of command, in order to perform this operation continue to repeatedly send the same command, this is not the most efficient approach would result in network resources waste.
If we have a way to avoid repeatedly send the same command, then the client can reduce the time spent in network transmission, the operation can be performed faster.
Affairs and optimistic locking defects
Although the use of a transaction can execute multiple commands, and can prevent the transaction by optimistic locking in competition conditions, but in practice, to correctly use the optimistic locking and transactions is not an easy thing.
-
For a complex transaction, it usually requires careful consideration in order to know which key lock: Lock should not lock the key will increase the chance of failed transactions, and may even cause the program error; forgetting to lock should key lock, the program will generate competition.
-
Sometimes in order to prevent race condition occurs, even if the operator does not need to use their own affairs, but in order to make optimistic locking into effect, we will use the transaction will command wrapped up, which increases the complexity of implementation, and brings additional performance loss .
Examples of misuse
To achieve a "transaction" ZDECRBY command of the introduction of transaction here merely to make use of the entry into force WATCH:
def ZDECRBY(key, decrment, member):
# 监视输入的有序集合
WATCH key
# 取得元素当前的分值
old_score = ZSCORE key member
# 使用当前分值减去指定的减量，得出新的分 值
new_score = old_score - decrment
# 使用事务包裹 ZADD 命令
# 确保 ZADD 命令只会在有序集合没有被修改的情况下 执行
MULTI
ZADD key new_score member # 为元素设置新分值，覆盖现有的分值
EXEC
Ways to avoid misuse of the transaction
If there is a way that allows us to execute in a transactional way multiple commands, and this method does not introduce any race conditions, then we can use this method instead Affairs and optimistic locking.
Extended functionality trouble Redis
Redis data structures are provided for each corresponding operation command, the database also provides the operational command itself, but if we need to do some data structures Redis command does not support the operation, you will need to use the client to retrieve data, and then by the client data is processed, the final data processing and then stored back to the Redis server.
As a simple example, because there is no Redis which provides a list of all of the even numbers delete command, so in order to do this, clients need to remove all of the items in the list which, then filtered at the client inside, and finally the filtered items push back inside to the list:
lst = LRANGE lst 0 -1 # 取出列表包含的所有元素
DEL lst # 删除现有的列表
for item in lst: # 遍历整个列表
if item % 2 != 0: # 将非偶数元素推入到列表里面
RPUSH lst item
And in order to ensure the safety of this operation, but also used in the transaction and optimistic locking, very troublesome.
Lua script
In order to solve the problems mentioned above, the Redis from version 2.6 server embedded inside a Lua interpreter, so that the user can execute on the server side script Lua.
This feature has the following benefits:
-
Using a script may be executed directly on the server side Redis command, general data processing operations may be used directly or library Lua, Lua interpreter provides to complete, without having to return to the client for processing.
-
All scripts are in the form of a transaction to be executed, the script will not be interrupted by other tasks in the implementation process, it will not cause any competition conditions, can use Lua script instead Affairs and optimistic locking.
-
When all the scripts are reusable, it is to say, repeat the same operation, as long as the call is stored in the cache inside the server script on it, do not re-send the entire script, thereby saving network resources as much as possible.
Lua script execution
Key script numkeys EVAL [Key ...] Arg [Arg ...]
Lua script script parameter is to be executed.
numkeys key is the number of database scripts to be processed after the key [key ...] key script parameter specifies the database to be processed, the key can be passed to obtain by accessing an array KEYS in the script, such as KEYS [1] on retrieves the first key input, kEYS [2] remove the second key input, and so forth.
arg [arg ...] parameter specifies the script to use the parameters, these parameters can be obtained by visiting the ARGV array in the script.
Explicitly specify the key script which used the check is in line with Redis cluster of keys, if you do not do so, use the script in a cluster which may be wrong.
In addition, by explicitly specify a script to use the database and related key parameters rather than key database parameters and hard to write in the script, users can more easily reuse the same script.
EVAL command example
redis> EVAL "return 'hello world'" 0
"hello world"
redis> EVAL "return 1+1" 0
(integer) 2
redis> EVAL "return {KEYS[1], KEYS[2], ARGV[1], ARGV[2]}" 2 "msg" "age" 123 "hello world"
1) "msg" # KEYS[1]
2) "age" # KEYS[2]
3) "123" # ARGV[1]
4) "hello world" # ARGV[2]
Redis commands execute scripts in Lua
By calling redis.call () function or redis.pcall () function, we can execute commands directly in Redis Lua script.
redis> EVAL "return redis.call('PING')" 0 # 在 Lua 脚本里面执行 PING 命令
PONG
redis> EVAL "return redis.call('DBSIZE')" 0 # 在 Lua 脚本里面执行 DBSIZE 命令
(integer) 4
# 在 Lua 脚本里面执行 GET 命令，取出键 msg 的值，并对值进行字符串拼接操作
redis> SET msg "hello world"
OK
redis> EVAL "return 'The message is: ' .. redis.call('GET', KEYS[1]) '" 1 msg
"The message is: hello world"
Difference redis.call () and redis.pcall () of
redis.call () and redis.pcall () can be used to perform Redis commands, their difference is that, when the script is executed error,
error redis.call () returns the name of the wrong script and the EVAL command information, redis.pcall () only returns the error message EVAL command.
redis> EVAL "return redis.call('NotExistsCommand')" 0
(error) ERR Error running script (call to f_ddabd662fa0a8e105765181ee7606562c1e6f1ce):
@user_script:1: @user_script: 1: Unknown Redis command called from Lua script
redis> EVAL "return redis.pcall('NotExistsCommand')" 0
(error) @user_script: 1: Unknown Redis command called from Lua script
In other words, when the script is executed in error, redis.call () can provide more detailed error information to facilitate troubleshooting.
Example: re-implement the ZDECRBY command Lua script
Zdecrby.lua create a file that contains the following:
local old_score = redis.call('ZSCORE', KEYS[1], ARGV[2])
local new_score = old_score - ARGV[1]
return redis.call('ZADD', KEYS[1], new_score, ARGV[2])
然后通过以下命令来执行脚本：
$ redis-cli --eval zdecrby.lua salary , 300 peter
(integer) 0
This implementation of EVAL "local ..." in redis-cli inside 1 salary 300 peter same effect, but first save the contents to a file inside the script, and then execute the script file approach than directly into a word which is easier on the client some.
In addition, the script realized ZDECRBY also much simpler than using ZDECRBY Affairs and optimistic locking implementation.
Use EVALSHA to reduce network resource consumption
Any Lua script, as long as the EVAL command is executed once, the script will be stored in the server's cache inside, as long as the user through EVALSHA command, specify the SHA1 value is cached script, it will be sent in without a script, execute the script again :
EVALSHA SHA1 numkeys Key [Key ...] Arg [Arg ...]
by SHA1 script that returns value reuse 'hello world' information:
redis> EVAL "return 'hello world'" 0
"hello world"
redis> EVALSHA 5332031c6b470dc5a0dd9b4bf2030dea6d65de91 0
"hello world"
通过 SHA1 值来重用之前实现的 ZDECRBY 命令，这样就不用每次都发送整个脚本了：
redis> EVALSHA 918130cae39ff0759b8256948742f77300a91cb2 1 salary 500 peter
(integer) 0
Scripting Commands
SCRIPT EXISTS sha1 [sha1 ...]
whether the script checks sha1 value represents the cache has been added to the script which is then returns 1, not 0, then return.
SCRIPT LOAD script
to script stored in the script cache inside, waiting for the future EVALSHA use.
SCRIPT FLUSH
clear all scripts Script cache storage.
SCRIPT KILL
kill script timeouts. If the script has been written to perform the operation, we also need to use SHUTDOWN NOSAVE command to force the server does not save the data, in order to avoid erroneous data is saved to the database inside.
Library
Redis Lua environment in which loaded some common function library, we can use these libraries, data processing directly in the script, they are standard libraries: • base library: Contains Lua core (core) functions, such as assert, tostring, error, type and so on.
• string library: Contains functions for working with strings, such as find, format, len, reverse and so on.
• table Library: contains functions for processing forms, such as concat, insert, remove, sort and so on.
• math library: Contains commonly used mathematical functions such as abs, sqrt, log, etc.
• debug library: Contains functions required for commissioning program, such as sethook, gethook and so on.
As well as external libraries • struct Library: convert between Lua language structure and value of the C language. • cjson library: Lua value will be converted to a JSON object, or convert a JSON object to Lua value. • cmsgpack Library: The Lua value MessagePack encoded format, or decoded from the inside MessagePack Lua value format.
There is also a function for external redis.sha1hex sha1 calculated values.
review
Review (1/2)
Lua scripts can be the same as the transaction is not disturbed execute multiple commands once, and because the script does not cause race conditions, so use a script to implement complex operations will be more convenient than using transactions and optimistic locking.
When required by EVALSHA command, we can, without sending the entire script, the script is performed directly by the SHA1 script, a complex operation of a plurality of commands, the use can be effectively reduced EVALSHA client and server communications amount of data transmitted.
By using Lua, and each library Lua built environment, the script can be executed directly on the server a number of data processing operations, without having to process to the client, so as to reduce unnecessary communication.
Review (2/2)
Execute script commands:
• EVAL Script numkeys Key [Key ...] Arg [Arg ...]
• EVALSHA sha1 numkeys Key [Key ...] Arg [Arg ...]
management script commands:
• SCRIPT EXISTS
• SCRIPT the LOAD
• SCRIPT FLUSH
• SCRIPT KILL
Reference material
Programming in Lua, Third Edition
http://www.lua.org/pil/
Lua 5.1 refernece Manual
www.lua.org/manual/5.1/
