Redis token bucket restrictor

First, the scene description

In the development interface of the server process, in order to prevent abuse of the client interface, server resource protection, in general, we will limit the number of calls for a variety of interfaces on the server. For example, for the number of times a user, who in a period of time (interval), such as one minute, call the server interface can not be greater than an upper limit (limit), for example, 100 times. If the number of users exceeds the upper limit of the call interface, then it refused to direct the user's request, returns an error message.
Service interface flow control strategy: diversion, demotion, current limiting. This article discusses the lower limit traffic policy, although reducing the frequency and amount of concurrent access to service interface, but in exchange for high-availability service interfaces and business application systems.

Second, the common limiting algorithm

1, the token bucket algorithm
leaky bucket (Leaky Bucket) algorithm is very simple idea, water (request) to the drain into the first bucket, bucket water at a constant speed (rate responsive to the interface), the inflow rate when the water overflows through direct Assembly ( Interface access frequency exceeds the response rate), and then rejects the request, the transmission rate can be seen that the leaky bucket algorithm can impose restrictions on the data in the following schematic:

 

　　　

 

 

There are two variables can be seen, it is the size of a barrel, how much water (burst) can be saved when the sudden increase in traffic support, and the other is the size of a bucket vulnerability (rate).
Because the leakage rate of leaky bucket parameters are fixed, so that, even if the network resource conflict does not exist (no congestion occurs), the token bucket algorithm does not make stream burst (Burst) to the port speed. Thus, the presence of the token bucket algorithm for projection flow characteristics of the hair is inefficient.

2, the token bucket algorithm
token bucket algorithm (Token Bucket), and the same effect Leaky Bucket algorithm, but in the opposite direction, more readily understood. Over time, the system will be a constant 1 / QPS interval (if QPS = 100, then interval is 10ms) in the bucket added Token (imagine leaks and loopholes contrary, there is a constant increase in the tap water), if the bucket is full is not coupled with a new request comes, we will each take a Token, if not Token can be picked them blocked or denial of service.

 

 

 

Another advantage is the token bucket can easily change the speed. Once the need to increase the rate, the demand to increase the rate of tokens into the bucket. Generally timing (for example 100 milliseconds) increased the tub to a certain number of tokens , the number of variants of some of the real-time calculation algorithm should increase the token.

Third, based on PHP + token bucket algorithm implemented Redis

<?php
namespace Api\Lib; /** * 限流控制 */ class RateLimit { private $minNum = 60; //单个用户每分访问数 private $dayNum = 10000; //单个用户每天总的访问量 public function minLimit($uid) { $minNumKey = $uid . '_minNum'; $dayNumKey = $uid . '_dayNum'; $resMin = $this->getRedis($minNumKey, $this->minNum, 60); $resDay = $this->getRedis($minNumKey, $this->minNum, 86400); if (!$resMin['status'] || !$resDay['status']) { exit($resMin['msg'] . $resDay['msg']); } } public function getRedis($key, $initNum, $expire) { $nowtime = time(); $result = ['status' => true, 'msg' => '']; $redisObj = $this->di->get('redis'); $redis->watch($key); $limitVal = $redis->get($key); if ($limitVal) { $limitVal = json_decode($limitVal, true); $newNum = min($initNum, ($limitVal['num'] - 1) + (($initNum / $expire) * ($nowtime - $limitVal['time']))); if ($newNum > 0) { $redisVal = json_encode(['num' => $newNum, 'time' => time()]); } else { return ['status' => false, 'msg' => '当前时刻令牌消耗完！']; } } else { $redisVal = json_encode(['num' => $initNum, 'time' => time()]); } $redis->multi(); $redis->set($key, $redisVal); $rob_result = $redis->exec(); if (!$rob_result) { $result = ['status' => false, 'msg' => '访问频次过多！']; } return $result; } } 

Code points:
1: First, the rules define
a single user visits per minute ($ minNum), a single user of the total number of visits per day ($ dayNum), different rules interfaces the total number of visits and the like.
2: calculating the rate
of the second sample code to the smallest unit of time, rate = number of accesses / times (initNum $ / $ The expire)
. 3: after each access token calculated complement number
acquired i.e., the last access time time deposit token time to calculate the current time and last access time difference is multiplied by the number of tokens in the rate needs to be supplemented, pay attention to add the token after token can not be greater than the total number of the number of token initialization to supplement the minimum number and the number of initialization prevail.
4: program flow
when you first access token initialization number ($ minNum), while the deposit Redis current timestamp token in order to calculate the number of stores next time you need to add. Acquiring second visit number of remaining tokens, the number and adds this token should be added, the number of tokens added after how> 0, then the current access is valid be accessible or inaccessible token after use. First supplement and then determine whether the token token> 0 of the reason is because there is this concept that is, if the rate of the last remaining token to zero but this should be added token> 1 then this can still be accessed.
5: For concurrent processing
optimistic locking mechanism using the Redis

Four, Redis optimistic locking Introduction

redis support for transactions is relatively simple. redis can only guarantee a client initiated transaction commands can be executed, among other matters will not be inserted. Because redis is single-threaded, so it is very easy to do the above. After receiving the client's general redis command is executed immediately, but if the client initiates multi command, redis does not happen immediately, but let the current connection into the transaction context, the command into the queue, the exec command is received, will redis command queue sequentially. And implementation of the results returned to the client packaged together, and then it ended the transaction context.
A simple transaction control

 

 

 

 

This example can be seen: two set command is issued but not executed immediately into the queue, redis received exec command began execution.
If two threads simultaneously modify the value of a variable, how to control the transaction is rolled back? Let's look at how optimistic locking control?
Second, the optimistic lock control transactions
1. What is the optimistic locking?
Mostly based version of the data recording mechanism. What is the version of the data? After the data is to add a version identifier for the database table to add a version field, when reading data, the database version read together, as a modified version of the database will be +1, submitted along with the changes. If the version number of the submitted data> database current version number, submitted successfully. Figure:

 

 

 

Example 2. The optimistic locking
assumed account information database table has a version field, the current value is 1, the account balance is $ 500

 

 

 

This avoids the possibility of the operator B by modifying old data recorded in the table.
3. redis reflected in how?
redis monitor watch with key, if the key is modified before submission, submission is unsuccessful. as follows:

 

 

 

当session1还没来得及对age进行修改，session2已经将age的值设为30，session1再执行的时候失败，因为session1对age加了乐观锁的缘故。
watch命令会监视key，当exec时如果监视的key从调用watch后发生过变化，则整个事务会失败。也可以调用watch多次监视多个key。
三、redis事务存在的问题
redis保证事务中的命令连续执行，但是如果其中一条命令执行失败，事务并不回滚。

 

 

 

 

为age +1的命令成功，因为anme是string类型的，所以不能做加操作，命令有一个失败也不会回滚，age的值已经被修改了