introduce
In the previous article, we introduced the application-level current limiting method, but we cannot perform full current limiting. In this article, we will introduce distributed current limiting and access layer current limiting that can perform global current limiting .
Access layer current limiting
Access layer current limiting usually refers to the entry of request traffic. The main purposes are: load balancing, illegal request filtering, request aggregation, caching, current limiting, A/B testing, and service quality monitoring. Usually Nginx (OpenResty) access layer current limiting is used .
Two modules that come with nginx can be used: the connection number current limiting module ngx_http_limit_conn_module and the request current limiting module ngx_http_limit_req_module implemented by the leaky bucket algorithm .
ngx_http_limit_conn_module
limit_conn is to limit the total number of network connections corresponding to a key. The total number of connections can be limited according to IP or service domain name . Not every request connection will be counted by the counter, only the request connection processed by nginx and the entire request header has been read will be counted by the counter . The configuration example is as follows:
http {
# 配置限流key、存放key对应信息的共享内存区域大小。此处key为$binary_remote_addr表示ip地址，也可以使用$server_name作为key
limit_conn_zone $binary_remote_addr zone=addr:10m;
# 配置记录被限流后的日志级别，默认error级别
limit_conn_log_level error;
# 配置被限流后返回的状态码，默认返回503
limit_conn_status 503;
...
server {
location /limit {
# 要配置存放key和计数器的共享内存区域和指定key的最大连接数
# 此处指定的最大连接数为1(nginx最多同时并发处理1个连接)
limit_conn addr 1;
}
...
}
复制代码The main execution process is as follows:
limit_conn can limit the total concurrent number/request number of a key, and the key can be changed as needed.
ngx_http_limit_req_module
limit_req is the implementation of the leaky bucket algorithm, which is used to specify the request corresponding to the key to limit the current.
Let's take a look at an example configuration:
http {
# 配置限流key、存放key对应信息的共享内存区域大小，固定请求速率。此处key为$binary_remote_addr表示ip地.固定请求速率使用rate参数配置，支持10r/s和60r/m
limit_req_zone $binary_remote_addr zone=one:10m rate=1r/s;
# 配置记录被限流后的日志级别，默认error级别
limit_conn_log_level error;
# 配置被限流后返回的状态码，默认返回503
limit_conn_status 503;
...
server {
location /limit {
# 配置限流区域、桶容量(突发容量，默认为0)、是否延迟模式（默认延迟）
limit_req zone=one burst=5 nodelay;
}
...
}
复制代码The execution process is as follows:
1. Request entry First judge whether the last request time relative to the current time (the first time is 0) needs to limit the current , if it needs to limit the current, go to step 2, otherwise go to step 3
2. If the bucket capacity is not configured (burst=0), requests are processed at a fixed rate. If the request is limited, the corresponding error code (503) will be returned directly.
如果配置了桶容量(burst>0)及延迟模式（没有配置nodelay）。如果桶满了，则新进入的请求被限流。如果没有满，则以固定平均速率处理(需要延迟处理请求，延迟使用休眠实现)
如果配置了桶容量(burst>0)及非延迟模式(配置了nodelay)，则不会按照固定速率处理请求，允许突发处理请求。如果桶满了，则请求被限流，直接返回状态503
3.如果没有被限流，正常处理请求
4.Nginx会在相应时机选择一些(3个节点)限流key进行过期处理，进行内存回收
OpenResty中的lua-resty-limit-traffic
上面介绍的两种使用比较简单，指定key、指定限流速率等就可以了。如果根据实际情况变化key、速率、桶大小等这种动态特性那么使用标准模块就很难实现。因此需要一种可编程方式解决此问题。OpenResty提供了Lua限流模块lua-resty-limit-traffic，可以按照更复杂的业务逻辑进行动态限流处理。提供了limit.conn和limit.req实现，算法和nginx limit_conn和limit_req是一样的。
OpenResty 1.11.2.2+默认已经支持该限流库。低版本的话在使用之前需要下载lua-resty-limit-traffic模块并添加到OpenResty的lualib中。我们当前使用的OpenResty版本为1.19.9.1,所以可以直接使用。
我们来看一个limit.req的使用示例：
limit_traffic.lua
local limit_req = require "resty.limit.req"
local rate = 2 -- 固定平均速率 2r/s
local burst = 3 -- 桶容量
local error_status = 503
local nodelay = false -- 是否需要不延迟处理
local lim, err = limit_req.new("limit_req_store", rate, burst)
if not lim then -- 没定义共享字典
ngx.log(ngx.ERR, "没定义共享字典", delay)
ngx.exit(error_status)
end
local key = ngx.var.binary_remote_addr -- IP维度限流,如果想根据参数可以查询所需参数进行限流
-- 流入请求，如果请求需要被延迟，则delay > 0
local delay, err = lim:incoming(key, true)
if not delay and err == "rejected" then -- 超出桶大小了
ngx.log(ngx.ERR, "超出桶大小：", err)
ngx.exit(error_status)
end
if delay > 0 then -- 根据需要决定是延迟还是不延迟处理
if nodelay then
-- 直接突发处理
ngx.log(ngx.ERR, "突发处理,正常需要延迟时间", delay)
else
ngx.sleep(delay) -- 延迟处理
ngx.log(ngx.ERR, "延迟处理,延迟时间:", delay)
end
end
复制代码上面我们配置了固定平均速率 2r/s，桶容量设置为3. 其中nginx.conf我们简单配置下：
worker_processes 1;
error_log logs/error.log;
events {
worker_connections 1024;
}
http {
lua_package_path "$prefix/lua/?.lua;$prefix/libs/?.lua;;";
lua_shared_dict limit_req_store 10m;
server {
server_name localhost;
listen 8080;
charset utf-8;
set $LESSON_ROOT lua/;
error_log logs/error.log;
access_log logs/access.log;
location /limit {
default_type text/html;
content_by_lua_file $LESSON_ROOT/limit_traffic.lua;
}
}
}
复制代码启动openresty后，我们使用ab命令来并发请求下。
wukongdeMacBook-Pro:limit wukong$ ab -c 3 -n 3 http://127.0.0.1:8080/limit
复制代码我们会发现有一个请求正常处理，另外两个进行了延迟处理。延迟处理的等待时间分别为500ms和1s,和我们设置的2r/s和桶容量3个是可以对应上的。如果想要突发处理的话可以在limit_traffic.lua中是否允许突发的变量nodelay设置为true.
2022/04/09 17:09:09 [error] 67100#12461017: *1008 [lua] limit_traffic.lua:24: 延迟处理,延迟时间:0.499, client: 127.0.0.1, server: localhost, request: "GET /limit HTTP/1.0", host: "127.0.0.1:8080"
2022/04/09 17:09:09 [error] 67100#12461017: *1009 [lua] limit_traffic.lua:24: 延迟处理,延迟时间:0.999, client: 127.0.0.1, server: localhost, request: "GET /limit HTTP/1.0", host: "127.0.0.1:8080"
复制代码如果同一时间并发请求远超过3个那后面的请求就会被拒绝，我们来简单测试看下，我们就以同时10个并发为例：
wukongdeMacBook-Pro:limit wukong$ ab -c 10 -n 10 http://127.0.0.1:8080/limit
复制代码我们再来看下日志：
2022/04/09 17:16:53 [error] 67100#12461017: *1014 [lua] limit_traffic.lua:15: 超出桶大小rejected, client: 127.0.0.1, server: localhost, request: "GET /limit HTTP/1.0", host: "127.0.0.1:8080"
2022/04/09 17:16:53 [error] 67100#12461017: *1015 [lua] limit_traffic.lua:15: 超出桶大小rejected, client: 127.0.0.1, server: localhost, request: "GET /limit HTTP/1.0", host: "127.0.0.1:8080"
2022/04/09 17:16:53 [error] 67100#12461017: *1016 [lua] limit_traffic.lua:15: 超出桶大小rejected, client: 127.0.0.1, server: localhost, request: "GET /limit HTTP/1.0", host: "127.0.0.1:8080"
2022/04/09 17:16:53 [error] 67100#12461017: *1017 [lua] limit_traffic.lua:15: 超出桶大小rejected, client: 127.0.0.1, server: localhost, request: "GET /limit HTTP/1.0", host: "127.0.0.1:8080"
2022/04/09 17:16:53 [error] 67100#12461017: *1018 [lua] limit_traffic.lua:15: 超出桶大小rejected, client: 127.0.0.1, server: localhost, request: "GET /limit HTTP/1.0", host: "127.0.0.1:8080"
2022/04/09 17:16:53 [error] 67100#12461017: *1019 [lua] limit_traffic.lua:15: 超出桶大小rejected, client: 127.0.0.1, server: localhost, request: "GET /limit HTTP/1.0", host: "127.0.0.1:8080"
2022/04/09 17:16:53 [error] 67100#12461017: *1011 [lua] limit_traffic.lua:24: 延迟处理,延迟时间:0.499, client: 127.0.0.1, server: localhost, request: "GET /limit HTTP/1.0", host: "127.0.0.1:8080"
2022/04/09 17:16:54 [error] 67100#12461017: *1012 [lua] limit_traffic.lua:24: 延迟处理,延迟时间:0.999, client: 127.0.0.1, server: localhost, request: "GET /limit HTTP/1.0", host: "127.0.0.1:8080"
2022/04/09 17:16:54 [error] 67100#12461017: *1013 [lua] limit_traffic.lua:24: 延迟处理,延迟时间:1.498, client: 127.0.0.1, server: localhost, request: "GET /limit HTTP/1.0", host: "127.0.0.1:8080"
复制代码我们会发现有6个请求被拒绝，1个请求已正常处理，3个请求延迟处理。
上面根据Nginx+lua实现简单的limit.req限流，示例代码已放到github，实际使用根据自身需要灵活运用。
另外Nginx也提供了limit_rate对流量限速，例如limit_rate 50K,表示限制下载速度为50K。
分布式限流
分布式限流最关键的是将限流服务做成原子化，解决方案：Redis+Lua或者Nginx+Lua,通过以上方案可实现高并发和高性能。
下面我们来看一个使用Redis+Lua实现时间窗口内某个接口的请求数限流，后期可改造为限流总并发/请求总资源数。Lua本身就是一种变成语言，可实现复杂令牌桶或漏桶算法。
Redis+Lua实现
@Component
@Slf4j
public class RedisLimit {
@Autowired
private JedisPool jedisPool;
/**
* 限流lua
*/
private static final String LIMIT_LUA;
static {
LIMIT_LUA = new StringBuilder()
//限流key
.append("local key = KEYS[1] ")
//限流大小
.append("local limit = tonumber(ARGV[1]) ")
.append("local current = tonumber(redis.call('get', key) or '0') ")
//如果超出限流大小
.append("if current + 1 > limit then ")
.append(" return 0 ")
//请求数+1并设置2s过期
.append("else ")
.append("redis.call('INCRBY', key, '1') ")
.append("redis.call('expire', key, '2') ")
.append(" return 1")
.append(" end").toString();
}
/**
* 是否需要限流
* @return
*/
public boolean acquire(){
Jedis jedis = jedisPool.getResource();
try {
//ip
String key = IpUtils.getIp();
//限流大小(分布式配置)
String limit = "1000";
return (Long) jedis.eval(LIMIT_LUA, Lists.newArrayList(key), Lists.newArrayList(limit)) == 1;
} catch (Exception e) {
log.error("是否需要限流异常", e);
} finally {
if(null != jedis){
jedis.close();
}
}
return false;
}
}
复制代码调用acquire来知道是否需要限流。上面我们限流大小设置的为1000，可以使用分布式配置进行配置，可实时修改。
Nginx+Lua
limit_lock.lua
local locks = require "resty.lock"
local function acquire()
local lock = locks:new("locks")
local elapsed, err = lock:lock("limit_key") --互斥锁
local limit_counter = ngx.shared.limit_counter -- 计数器
local key = "ip" ..os.time()
local limit = 5 -- 限流大小
local current = limit_counter:get(key)
if current ~= nil and current + 1 > limit then -- 如果超出限流大小
lock:unlock()
return 0
end
if current == nil then
limit_counter:set(key, 1, 1) -- 第一次需要设置过期时间，设置key的值为1，过期时间为1s
else
limit_counter:incr(key, 1) -- 第二次开始加1即可
end
lock:unlock()
return 1
end
ngx.log(ngx.ERR, "限流标识(1:正常 0:限流):", acquire())
复制代码我们用ab命令简单测试下：
wukongdeMacBook-Pro:limit wukong$ ab -c 10 -n 10 http://127.0.0.1:8080/limit/lock
复制代码输出日志如下：
2022/04/09 21:56:13 [error] 30012#12823604: *26 [lua] limit_lock.lua:22: 限流标识(1:正常 0:限流):1, client: 127.0.0.1, server: localhost, request: "GET /limit/lock HTTP/1.0", host: "127.0.0.1:8080"
2022/04/09 21:56:13 [error] 30012#12823604: *27 [lua] limit_lock.lua:22: 限流标识(1:正常 0:限流):1, client: 127.0.0.1, server: localhost, request: "GET /limit/lock HTTP/1.0", host: "127.0.0.1:8080"
2022/04/09 21:56:13 [error] 30012#12823604: *28 [lua] limit_lock.lua:22: 限流标识(1:正常 0:限流):1, client: 127.0.0.1, server: localhost, request: "GET /limit/lock HTTP/1.0", host: "127.0.0.1:8080"
2022/04/09 21:56:13 [error] 30012#12823604: *29 [lua] limit_lock.lua:22: 限流标识(1:正常 0:限流):1, client: 127.0.0.1, server: localhost, request: "GET /limit/lock HTTP/1.0", host: "127.0.0.1:8080"
2022/04/09 21:56:13 [error] 30012#12823604: *30 [lua] limit_lock.lua:22: 限流标识(1:正常 0:限流):1, client: 127.0.0.1, server: localhost, request: "GET /limit/lock HTTP/1.0", host: "127.0.0.1:8080"
2022/04/09 21:56:13 [error] 30012#12823604: *31 [lua] limit_lock.lua:22: 限流标识(1:正常 0:限流):0, client: 127.0.0.1, server: localhost, request: "GET /limit/lock HTTP/1.0", host: "127.0.0.1:8080"
2022/04/09 21:56:13 [error] 30012#12823604: *32 [lua] limit_lock.lua:22: 限流标识(1:正常 0:限流):0, client: 127.0.0.1, server: localhost, request: "GET /limit/lock HTTP/1.0", host: "127.0.0.1:8080"
2022/04/09 21:56:13 [error] 30012#12823604: *33 [lua] limit_lock.lua:22: 限流标识(1:正常 0:限流):0, client: 127.0.0.1, server: localhost, request: "GET /limit/lock HTTP/1.0", host: "127.0.0.1:8080"
2022/04/09 21:56:13 [error] 30012#12823604: *34 [lua] limit_lock.lua:22: 限流标识(1:正常 0:限流):0, client: 127.0.0.1, server: localhost, request: "GET /limit/lock HTTP/1.0", host: "127.0.0.1:8080"
2022/04/09 21:56:13 [error] 30012#12823604: *35 [lua] limit_lock.lua:22: 限流标识(1:正常 0:限流):0, client: 127.0.0.1, server: localhost, request: "GET /limit/lock HTTP/1.0", host: "127.0.0.1:8080"
复制代码可以看到有五个正常执行，和预期一致。
在上面nginx+lua的代码实现中我们使用了lua-resty-lock互斥锁来解决原子性问题，在实际使用过程中我们需要考虑获取锁的超时问题。
聊到现在你可能会问如果并发量特别大，redis或者nginx是否能扛的住，可以从以下方面考虑：
1.综合考虑下流量是不是真的特别大，当前限流是否已经满足 2.通过一致性哈希将分布式限流进行分片 3.可以当并发量太大时降级为应用级限流 ...
总结
本文主要讨论了分布式限流和接入层限流，实际使用中结合自身需要进行选择。应用级限流可参考之前一篇文章应用级限流
参考书籍：《亿级流量网站架构核心技术》