Detailed explanation of Spring Cloud current limit (including source code)

Original: http://www.itmuch.com/spring-cloud-sum/spring-cloud-ratelimit/

In high-concurrency applications, current limiting is often an unavoidable topic. This article discusses in detail how to implement throttling in Spring Cloud.

It Zuulis a good choice to implement current limiting on . You only need to write a filter. The key lies in how to implement the current limiting algorithm. Common current limiting algorithms include leaky bucket algorithm and token bucket algorithm. You can refer to https://www.cnblogs.com/LBSer/p/4083131.html for this. It is written in an easy-to-understand manner. You deserve it, so I will not drag the text.

Google GuavaProvides us with a current limiting tool class RateLimiter, so we can code.

<!-- more -->

code example

@Component
public class RateLimitZuulFilter extends ZuulFilter {

    private final RateLimiter rateLimiter = RateLimiter.create(1000.0);

    @Override
    public String filterType() {
        return FilterConstants.PRE_TYPE;
    }

    @Override
    public int filterOrder() {
        return Ordered.HIGHEST_PRECEDENCE;
    }

    @Override
    public boolean shouldFilter() {
        // 这里可以考虑弄个限流开启的开关，开启限流返回true，关闭限流返回false，你懂的。
        return true;
    }

    @Override
    public Object run() {
        try {
            RequestContext currentContext = RequestContext.getCurrentContext();
            HttpServletResponse response = currentContext.getResponse();
            if (!rateLimiter.tryAcquire()) {
                HttpStatus httpStatus = HttpStatus.TOO_MANY_REQUESTS;

                response.setContentType(MediaType.TEXT_PLAIN_VALUE);
                response.setStatus(httpStatus.value());
                response.getWriter().append(httpStatus.getReasonPhrase());

                currentContext.setSendZuulResponse(false);

                throw new ZuulException(
                        httpStatus.getReasonPhrase(),
                        httpStatus.value(),
                        httpStatus.getReasonPhrase()
                );
            }
        } catch (Exception e) {
            ReflectionUtils.rethrowRuntimeException(e);
        }
        return null;
    }
}

As above, we wrote a prefilter of type . For questions about Zuul filters, please refer to my blog:

• Detailed explanation of the built-in Zuul filter in Spring Cloud: http://www.itmuch.com/spring-cloud/zuul/zuul-filter-in-spring-cloud

• Detailed explanation of Spring Cloud Zuul filter: http://www.itmuch.com/spring-cloud/zuul/spring-cloud-zuul-filter

In the filter, we use Guava RateLimiterto limit the flow and throw an exception if the maximum flow has been reached.

Current Limiting in Distributed Scenarios

The above current limit under single-node Zuul, but in production, we often have multiple Zuul instances. How to limit the current in this scenario? We can use Redis to implement current limiting.

Implemented using redis, two keys are stored, one for timing and one for counting. Each time the request is called, the counter is incremented by 1. If the counter does not exceed the threshold within the timer time, the task can be processed

if(!cacheDao.hasKey(TIME_KEY)) {
    cacheDao.putToValue(TIME_KEY, 0, 1, TimeUnit.SECONDS);
}       
if(cacheDao.hasKey(TIME_KEY) && cacheDao.incrBy(COUNTER_KEY, 1) > 400) {
    // 抛个异常什么的
}

Implement microservice-level current throttling

In some scenarios, we may also need to implement microservice granularity current limiting. There are two options at this point:

Method 1: Implement current limiting in the microservice itself.

Similar to implementing current limiting on Zuul, you only need to write a filter or interceptor, which is relatively simple and will not be described in detail. Personally, I don't like this method very much, because each microservice has to be coded, which feels very expensive.

With so much overtime, as programmers, we should learn to be lazy, so that we may have time to be filial to our parents, hug our wives, tease our sons, walk dogs and birds, chat and spank, and pursue our beliefs in life. Well, let’s not talk nonsense, let’s look at method two.

Method 2: Implement microservice granularity current limiting on Zuul.

Before explaining, we might as well simulate two routing rules, which respectively represent the two routing methods of Zuul.

zuul:
  routes:
    microservice-provider-user: /user/**
    user2:
      url: http://localhost:8000/
      path: /user2/**

As shown in the configuration, here, we define two routing rules, microservice-provider-userand this routing rule uses Ribbon + Hystrix, which is yes user2; and this routing does not use Ribbon or Hystrix, and it is yes . If you can't figure this out, see my blog:microservice-provider-userRibbonRoutingFilteruser2SipleRoutingFilter

• Detailed explanation of the built-in Zuul filter in Spring Cloud: http://www.itmuch.com/spring-cloud/zuul/zuul-filter-in-spring-cloud
• Detailed explanation of Spring Cloud Zuul filter: http://www.itmuch.com/spring-cloud/zuul/spring-cloud-zuul-filter

Once we figure this out, we can run the code:

@Component
public class RateLimitZuulFilter extends ZuulFilter {

    private Map<String, RateLimiter> map = Maps.newConcurrentMap();

    @Override
    public String filterType() {
        return FilterConstants.PRE_TYPE;
    }

    @Override
    public int filterOrder() {
        // 这边的order一定要大于org.springframework.cloud.netflix.zuul.filters.pre.PreDecorationFilter的order
        // 也就是要大于5
        // 否则，RequestContext.getCurrentContext()里拿不到serviceId等数据。
        return Ordered.LOWEST_PRECEDENCE;
    }

    @Override
    public boolean shouldFilter() {
        // 这里可以考虑弄个限流开启的开关，开启限流返回true，关闭限流返回false，你懂的。
        return true;
    }

    @Override
    public Object run() {
        try {
            RequestContext context = RequestContext.getCurrentContext();
            HttpServletResponse response = context.getResponse();

            String key = null;
            // 对于service格式的路由，走RibbonRoutingFilter
            String serviceId = (String) context.get(SERVICE_ID_KEY);
            if (serviceId != null) {
                key = serviceId;
                map.putIfAbsent(serviceId, RateLimiter.create(1000.0));
            }
            // 如果压根不走RibbonRoutingFilter，则认为是URL格式的路由
            else {
                // 对于URL格式的路由，走SimpleHostRoutingFilter
                URL routeHost = context.getRouteHost();
                if (routeHost != null) {
                    String url = routeHost.toString();
                    key = url;
                    map.putIfAbsent(url, RateLimiter.create(2000.0));
                }
            }
            RateLimiter rateLimiter = map.get(key);
            if (!rateLimiter.tryAcquire()) {
                HttpStatus httpStatus = HttpStatus.TOO_MANY_REQUESTS;

                response.setContentType(MediaType.TEXT_PLAIN_VALUE);
                response.setStatus(httpStatus.value());
                response.getWriter().append(httpStatus.getReasonPhrase());

                context.setSendZuulResponse(false);

                throw new ZuulException(
                        httpStatus.getReasonPhrase(),
                        httpStatus.value(),
                        httpStatus.getReasonPhrase()
                );
            }
        } catch (Exception e) {
            ReflectionUtils.rethrowRuntimeException(e);
        }
        return null;
    }
}

Briefly explain this code:

For microservice-provider-userthis route, we can use context.get(SERVICE_ID_KEY);to get the serviceId, and get it microservice-provider-user:

For user2this route, we use context.get(SERVICE_ID_KEY);to get is null, but we can use to context.getRouteHost()get the address of the route, and get it out http://localhost:8000/. What happens next, you know.

Improve and enhance

In actual projects, in addition to the current limiting method implemented above, it may also:

1. On the basis of the above, add configuration items, control the current limiting indicators of each route, and realize dynamic refresh , so as to achieve more flexible management

2. Based on CPU, memory, database and other pressure current limiting (thanks to Ping An Chang Haozhi) proposed. .

Next, the author uses the capabilities provided by Spring Boot Actuator Metricsto implement current limiting based on memory pressure - when the available memory is lower than a certain threshold, the current limiting is turned on, otherwise the current limiting is not turned on.

@Component
public class RateLimitZuulFilter extends ZuulFilter {
    @Autowired
    private SystemPublicMetrics systemPublicMetrics;
    @Override
    public boolean shouldFilter() {
        // 这里可以考虑弄个限流开启的开关，开启限流返回true，关闭限流返回false，你懂的。
        Collection<Metric<?>> metrics = systemPublicMetrics.metrics();
        Optional<Metric<?>> freeMemoryMetric = metrics.stream()
                .filter(t -> "mem.free".equals(t.getName()))
                .findFirst();
        // 如果不存在这个指标，稳妥起见，返回true，开启限流
        if (!freeMemoryMetric.isPresent()) {
            return true;
        }
        long freeMemory = freeMemoryMetric.get()
                .getValue()
                .longValue();
        // 如果可用内存小于1000000KB，开启流控
        return freeMemory < 1000000L;
    }
    // 省略其他方法
}

**3. Realize current limiting in different dimensions, ** for example:

• Limit the target URL of the request (for example: how many times a URL is only allowed to be called per minute)
• Limit the current access IP of the client (for example: how many times a certain IP is only allowed to request per minute)
• Limit current for some specific users or user groups (for example, non-VIP users are limited to only 100 calls to an API per minute, etc.)
• Current limiting for multi-dimensional mixing. At this point, it is necessary to implement an orchestration mechanism for some current limiting rules. and, or, not, etc.

Reference documentation

• Implementation of current limiting scheme in distributed environment: http://blog.csdn.net/Justnow_/article/details/53055299