Table of contents
2. Overview of current limiting
2.1 dubbo service governance model
2.1.1 dubbo frame-level current limiting
2.1.3 Integrate third-party components
2.2 springcloud service governance model
2.3 Gateway layer current limiting
3. Common current limiting strategies
3.1 Algorithms commonly used for current limiting
4. General Current Limiting Implementation Scheme
4.1 Realization based on guava current limiting
4.1.1 Introducing guava dependencies
4.1.2 Custom current limit annotation
4.1.3 Current Limiting AOP Class
4.2 Realization based on sentinel current limiting
4.2.1 Introducing the sentinel core dependency package
4.2.2 Custom current limit annotation
4.2.3 Custom AOP class to implement current limiting
4.3 Implementation of current limiting based on redis+lua
4.3.1 Introduce redis dependency
4.3.3 Custom redis configuration class
4.3.4 Custom current limiting AOP class
5. Custom starter current limiting implementation
5.2 Code Integration Completion Steps
5.2.1 Import basic dependencies
5.2.3 AOP implementation of current limiting
5.2.4 Configuring automatic assembly AOP implementation
5.2.5 Make the project into a jar for installation
5.2.6 Introduce the above SDK into other projects
Sixth, write at the end of the text
1. Background
Current limiting is very important for a microservice architecture system, otherwise one of the microservices will become a hidden avalanche factor for the entire system, why do you say that? For example, a certain SAAS platform has more than 100 microservice applications, but one or several of the underlying applications will be frequently invoked by all upper-level applications. During peak business hours, if the underlying applications are not restricted For stream processing, the application is bound to face tremendous pressure, especially for those interfaces that are called frequently, the most direct performance is that subsequent new incoming requests are blocked, queued, and the response times out... Finally, until the service Where JVM resources are exhausted.
2. Overview of current limiting
At the beginning of the design of most microservice architectures, such as in the technology selection stage, architects will plan the combination of technology stacks from a global perspective. For example, considering the status quo of current products, should we use dubbo? Or spring cloud? As the underlying framework of microservice governance. Even in order to meet the rapid launch, iteration and delivery, the development is directly based on springboot, and a new technology stack is introduced later...
Therefore, when talking about specific technical solutions for a certain business scenario, we should not generalize them. Instead, we need to comprehensively evaluate the status quo of products and services. In terms of current limiting, it may not be the same when choosing under the following different technical architectures. Same.
2.1 dubbo service governance model
Choosing the dubbo framework as the basic service governance is good for applications that are biased towards internal platforms. Dubbo uses netty at the bottom layer. Compared with the http protocol, it still has advantages in certain scenarios. If you choose dubbo, you must choose current limiting The following references can be made on the plan.
2.1.1 dubbo frame-level current limiting
Dubbo officially provides comprehensive service management, which can meet the needs of most development scenarios. For the current limiting scenario, it specifically includes the following methods. For specific configuration, you can refer to the official manual;
- Client current limiting
Semaphore current limiting (by means of statistics)
Connection number current limiting (socket->tcp) - Server-side current limiting
thread pool current limiting (isolation means)
semaphore current limiting (non-isolation means)
receiving number current limiting (socket->tcp)
2.1.2 Thread Pool Settings
Multi-threaded concurrent operations must be inseparable from the thread pool. Dubbo itself provides support for four types of thread pools. The key parameters of the thread pool can be configured in the producer <dubbo:protocol>tab, such as the thread pool type, the size of the blocking queue, and the number of core threads. By configuring the number of thread pools on the production side, the current limiting effect can be achieved to a certain extent.
2.1.3 Integrate third-party components
If it is a springboot framework project, you can consider directly introducing local components or SDKs, such as hystrix, guava, sentinel native SDK, etc. If the technical strength is strong enough, you can even consider building your own wheels.
2.2 springcloud service governance model
If you use springcloud or springcloud-alibaba as your service governance framework, the framework’s own ecology already contains corresponding current-limiting components, which can be used out of the box. Here are some commonly used current-limiting components based on the springcloud framework. .
2.2.1 hystrix
Hystrix is a fault-tolerant framework open sourced by Netflix. When springcloud was launched to the market in the early stage, it was used as a component in the springcloud ecosystem for current limiting, fusing, and downgrading. Hystrix provides the current limiting function. In the springcloud architecture system, Hystrix can be enabled on the gateway for current limiting processing, and each microservice can also enable Hystrix for current limiting.
Hystrix uses the thread isolation mode by default, and can limit the current through the number of threads + queue size. For specific parameter configuration, please refer to the relevant information on the official website.
2.2.2 sentinel
Sentinel, known as the traffic guard of the distributed system, is an important component in the springcloud-alibaba ecology. It is a traffic control component for the distributed service architecture. , hotspot protection and other dimensions to help developers ensure the stability of microservices.
2.3 Gateway layer current limiting
As the scale of microservices increases, when many microservices in the entire system need to implement current limiting, you can consider limiting current at the gateway layer. Generally speaking, the current limiting at the gateway layer is for general business, such as those malicious requests, crawlers, attacks, etc. In simple terms, traffic limiting at the gateway level provides a layer of protection for the system as a whole.
3. Common current limiting strategies
3.1 Algorithms commonly used for current limiting
No matter what kind of current-limiting components, the underlying current-limiting implementation algorithms are similar. Here are some commonly used current-limiting algorithms for understanding.
3.1.1 Token Bucket Algorithm
The token bucket algorithm is currently the most widely used current limiting algorithm. As the name implies, it has the following two key roles:
- Token : Requests that get tokens will be processed, and other Requests will either be queued or discarded directly;
- Bucket : The place used to hold tokens, all Requests get tokens from this bucket
The token bucket mainly involves two processes, namely token generation and token acquisition
3.1.2 Leaky Bucket Algorithm
The first half of the leaky bucket algorithm is similar to the token bucket, but the objects of the operation are different, which can be understood in conjunction with the following figure.
The token bucket is to put the token into the bucket, and the leaky bucket is to put the data packet of the access request into the bucket. Similarly, if the bucket is full, new incoming packets will be discarded.
3.1.3 Sliding Time Window
According to the figure below, briefly describe the process of sliding the time window:
- The big black frame is the time window, and the time unit of the window can be set to 5 seconds, and it will slide backwards as time goes by. We divide the time in the window into five small grids, each grid represents 1 second, and this grid also contains a counter to calculate the number of requests accessed within the current time. Then the total number of visits in this time window is the accumulated value of all grid counters;
- For example, if we have 5 user visits every second, and 10 user visits within the 5th second, then the number of visits in the time window from 0 to 5 seconds is 15. If our interface sets the upper limit of visits in the time window to 20, then when the time reaches the sixth second, the sum of the counts in this time window becomes 10, because the grid of 1 second has exited the time window, so in The number of visits that can be received within the sixth second is 20-10=10;
Sliding window is actually a calculator algorithm. It has a remarkable feature. The longer the time window span, the smoother the current limiting effect. For example, if the current time window is only two seconds, and all access requests are concentrated in the first second, when the time slides back by one second, the number of counts in the current window will change greatly. Extending the time window can reduce the chance of this happening
4. General Current Limiting Implementation Scheme
Leaving aside the current limiting at the gateway layer, in microservice applications, considering factors such as the combination of technology stacks, the development level of team members, and ease of maintenance, a more common approach is to use AOP technology + custom The annotation implements current limiting for a specific method or interface. Based on this idea, the following introduces the implementation of several commonly used current limiting schemes.
4.1 Realization based on guava current limiting
Guava is a relatively practical component open sourced by Google. Using this component can help developers complete conventional current limiting operations. Next, we will see the specific implementation steps.
4.1.1 Introducing guava dependencies
The version can choose a higher or other version
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>23.0</version>
</dependency>4.1.2 Custom current limit annotation
Customize an annotation for current limiting, and then only need to add this annotation on the method or interface that needs current limiting;
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
@Target(value = ElementType.METHOD)
@Retention(value = RetentionPolicy.RUNTIME)
public @interface RateConfigAnno {
String limitType();
double limitCount() default 5d;
}4.1.3 Current Limiting AOP Class
Intercept the method of adding the above-mentioned custom current-limiting annotation through AOP pre-notification, parse the attribute value in the annotation, and use the attribute value as the current-limiting parameter provided by guava. This class is the core of the entire implementation.
import com.alibaba.fastjson2.JSONObject;
import com.google.common.util.concurrent.RateLimiter;
import org.aspectj.lang.JoinPoint;
import org.aspectj.lang.annotation.Aspect;
import org.aspectj.lang.annotation.Before;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.stereotype.Component;
import org.springframework.web.context.request.RequestContextHolder;
import org.springframework.web.context.request.ServletRequestAttributes;
import javax.servlet.ServletOutputStream;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;
import java.lang.reflect.Method;
import java.util.Objects;
@Aspect
@Component
public class GuavaLimitAop {
private static Logger logger = LoggerFactory.getLogger(GuavaLimitAop.class);
@Before("execution(@RateConfigAnno * *(..))")
public void limit(JoinPoint joinPoint) {
//1、获取当前的调用方法
Method currentMethod = getCurrentMethod(joinPoint);
if (Objects.isNull(currentMethod)) {
return;
}
//2、从方法注解定义上获取限流的类型
String limitType = currentMethod.getAnnotation(RateConfigAnno.class).limitType();
double limitCount = currentMethod.getAnnotation(RateConfigAnno.class).limitCount();
//使用guava的令牌桶算法获取一个令牌，获取不到先等待
RateLimiter rateLimiter = RateLimitHelper.getRateLimiter(limitType, limitCount);
boolean b = rateLimiter.tryAcquire();
if (b) {
System.out.println("获取到令牌");
}else {
HttpServletResponse resp = ((ServletRequestAttributes) RequestContextHolder.getRequestAttributes()).getResponse();
JSONObject jsonObject=new JSONObject();
jsonObject.put("success",false);
jsonObject.put("msg","限流中");
try {
output(resp, jsonObject.toJSONString());
}catch (Exception e){
logger.error("error,e:{}",e);
}
}
}
private Method getCurrentMethod(JoinPoint joinPoint) {
Method[] methods = joinPoint.getTarget().getClass().getMethods();
Method target = null;
for (Method method : methods) {
if (method.getName().equals(joinPoint.getSignature().getName())) {
target = method;
break;
}
}
return target;
}
public void output(HttpServletResponse response, String msg) throws IOException {
response.setContentType("application/json;charset=UTF-8");
ServletOutputStream outputStream = null;
try {
outputStream = response.getOutputStream();
outputStream.write(msg.getBytes("UTF-8"));
} catch (IOException e) {
e.printStackTrace();
} finally {
outputStream.flush();
outputStream.close();
}
}
}
Among them, the core API of current limiting is the object of RateLimiter, and the related RateLimitHelper class is as follows
import com.google.common.util.concurrent.RateLimiter;
import java.util.HashMap;
import java.util.Map;
public class RateLimitHelper {
private RateLimitHelper(){}
private static Map<String,RateLimiter> rateMap = new HashMap<>();
public static RateLimiter getRateLimiter(String limitType,double limitCount ){
RateLimiter rateLimiter = rateMap.get(limitType);
if(rateLimiter == null){
rateLimiter = RateLimiter.create(limitCount);
rateMap.put(limitType,rateLimiter);
}
return rateLimiter;
}
}4.1.4 Test interface
Add a test interface below to test whether the above code works
@RestController
public class OrderController {
//localhost:8081/save
@GetMapping("/save")
@RateConfigAnno(limitType = "saveOrder",limitCount = 1)
public String save(){
return "success";
}
}In order to simulate the effect in the interface, we set the parameters very small, that is, QPS is 1. It can be expected that when the request per second exceeds 1, there will be a prompt of current limitation. Start the project and verify the interface, once per second Request, the result can be obtained normally, the effect is as follows:
Quickly brush the interface, you will see the following effect
4.2 Realization based on sentinel current limiting
In the consciousness of many students, sentinel usually needs to be combined with the springcloud-alibaba framework to be practical, and after integrating with the framework, it can be used with the console to achieve better results. In fact, sentinel officially provides a relatively native The SDK is available, and the integration is done this way.
4.2.1 Introducing the sentinel core dependency package
<dependency>
<groupId>com.alibaba.csp</groupId>
<artifactId>sentinel-core</artifactId>
<version>1.8.0</version>
</dependency>
4.2.2 Custom current limit annotation
You can add more attributes as needed
import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;
@Target(value = ElementType.METHOD)
@Retention(value = RetentionPolicy.RUNTIME)
public @interface SentinelLimitAnnotation {
String resourceName();
int limitCount() default 5;
}4.2.3 Custom AOP class to implement current limiting
The implementation idea of this class is similar to the above-mentioned use of guava, the difference is that the original Sentinel current-limiting related API is used here, and if the property is not enough, you can refer to the official document for learning, so I won’t expand it here.
import com.alibaba.csp.sentinel.Entry;
import com.alibaba.csp.sentinel.SphU;
import com.alibaba.csp.sentinel.Tracer;
import com.alibaba.csp.sentinel.slots.block.BlockException;
import com.alibaba.csp.sentinel.slots.block.RuleConstant;
import com.alibaba.csp.sentinel.slots.block.flow.FlowRule;
import com.alibaba.csp.sentinel.slots.block.flow.FlowRuleManager;
import org.apache.commons.lang3.StringUtils;
import org.aspectj.lang.JoinPoint;
import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.annotation.Aspect;
import org.aspectj.lang.annotation.Pointcut;
import org.springframework.stereotype.Component;
import java.lang.reflect.Method;
import java.util.ArrayList;
import java.util.List;
import java.util.Objects;
@Aspect
@Component
public class SentinelMethodLimitAop {
private static void initFlowRule(String resourceName,int limitCount) {
List<FlowRule> rules = new ArrayList<>();
FlowRule rule = new FlowRule();
//设置受保护的资源
rule.setResource(resourceName);
//设置流控规则 QPS
rule.setGrade(RuleConstant.FLOW_GRADE_QPS);
//设置受保护的资源阈值
rule.setCount(limitCount);
rules.add(rule);
//加载配置好的规则
FlowRuleManager.loadRules(rules);
}
@Pointcut(value = "@annotation(com.congge.sentinel.SentinelLimitAnnotation)")
public void rateLimit() {
}
@Around("rateLimit()")
public Object around(ProceedingJoinPoint joinPoint) {
//1、获取当前的调用方法
Method currentMethod = getCurrentMethod(joinPoint);
if (Objects.isNull(currentMethod)) {
return null;
}
//2、从方法注解定义上获取限流的类型
String resourceName = currentMethod.getAnnotation(SentinelLimitAnnotation.class).resourceName();
if(StringUtils.isEmpty(resourceName)){
throw new RuntimeException("资源名称为空");
}
int limitCount = currentMethod.getAnnotation(SentinelLimitAnnotation.class).limitCount();
initFlowRule(resourceName,limitCount);
Entry entry = null;
Object result = null;
try {
entry = SphU.entry(resourceName);
try {
result = joinPoint.proceed();
} catch (Throwable throwable) {
throwable.printStackTrace();
}
} catch (BlockException ex) {
// 资源访问阻止，被限流或被降级
// 在此处进行相应的处理操作
System.out.println("blocked");
return "被限流了";
} catch (Exception e) {
Tracer.traceEntry(e, entry);
} finally {
if (entry != null) {
entry.exit();
}
}
return result;
}
private Method getCurrentMethod(JoinPoint joinPoint) {
Method[] methods = joinPoint.getTarget().getClass().getMethods();
Method target = null;
for (Method method : methods) {
if (method.getName().equals(joinPoint.getSignature().getName())) {
target = method;
break;
}
}
return target;
}
}
4.2.4 Custom test interface
In order to simulate the effect, the number of QPS is set to 1 here
//localhost:8081/limit
@GetMapping("/limit")
@SentinelLimitAnnotation(limitCount = 1,resourceName = "sentinelLimit")
public String sentinelLimit(){
return "sentinelLimit";
}After starting the project, the browser calls the interface to test, one request per second can pass normally
Quickly refresh the interface, when it exceeds 1 time per second, the effect is as follows
This is just to demonstrate the effect. It is recommended to encapsulate the returned result when used in a real project.
4.3 Implementation of current limiting based on redis+lua
Redis is thread-safe, naturally has thread-safe features, and supports atomic operations. The current-limiting service not only needs to undertake ultra-high QPS , but also ensures that the execution level of the current-limiting logic has thread-safe features. Use these features of Redis for current limiting , which can ensure both thread safety and performance. The complete flow of redis-based current limiting implementation is as follows:
Combined with the above flowchart, here is an overall implementation idea:
- Write a lua script to specify the current limiting rules for input parameters. For example, when limiting the current of a specific interface, you can make a judgment based on one or several parameters, call the request of the interface, and monitor the number of requests within a certain time window;
- Since it is current limiting, it is best to be universal, and the current limiting rules can be applied to any interface, so the most suitable way is to cut in through custom annotations;
- Provide a configuration class, which is managed by the spring container, and the DefaultRedisScript bean is provided in redisTemplate;
- Provide a class that can dynamically analyze interface parameters, and trigger current limiting after matching rules according to interface parameters;
4.3.1 Introduce redis dependency
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-data-redis</artifactId>
</dependency>4.3.2 Custom annotations
@Target({ElementType.METHOD, ElementType.TYPE})
@Retention(RetentionPolicy.RUNTIME)
@Inherited
@Documented
public @interface RedisLimitAnnotation {
/**
* key
*/
String key() default "";
/**
* Key的前缀
*/
String prefix() default "";
/**
* 一定时间内最多访问次数
*/
int count();
/**
* 给定的时间范围 单位(秒)
*/
int period();
/**
* 限流的类型(用户自定义key或者请求ip)
*/
LimitType limitType() default LimitType.CUSTOMER;
}4.3.3 Custom redis configuration class
import org.springframework.context.annotation.Bean;
import org.springframework.core.io.ClassPathResource;
import org.springframework.data.redis.connection.RedisConnectionFactory;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.data.redis.core.script.DefaultRedisScript;
import org.springframework.data.redis.serializer.Jackson2JsonRedisSerializer;
import org.springframework.data.redis.serializer.StringRedisSerializer;
import org.springframework.scripting.support.ResourceScriptSource;
import org.springframework.stereotype.Component;
import java.io.Serializable;
@Component
public class RedisConfiguration {
@Bean
public DefaultRedisScript<Number> redisluaScript() {
DefaultRedisScript<Number> redisScript = new DefaultRedisScript<>();
redisScript.setScriptSource(new ResourceScriptSource(new ClassPathResource("limit.lua")));
redisScript.setResultType(Number.class);
return redisScript;
}
@Bean("redisTemplate")
public RedisTemplate<String, Object> redisTemplate(RedisConnectionFactory redisConnectionFactory) {
RedisTemplate<String, Object> redisTemplate = new RedisTemplate<>();
redisTemplate.setConnectionFactory(redisConnectionFactory);
Jackson2JsonRedisSerializer jackson2JsonRedisSerializer = new Jackson2JsonRedisSerializer(Object.class);
//设置value的序列化方式为JSOn
redisTemplate.setValueSerializer(jackson2JsonRedisSerializer);
//设置key的序列化方式为String
redisTemplate.setKeySerializer(new StringRedisSerializer());
redisTemplate.setHashKeySerializer(new StringRedisSerializer());
redisTemplate.setHashValueSerializer(jackson2JsonRedisSerializer);
redisTemplate.afterPropertiesSet();
return redisTemplate;
}
}4.3.4 Custom current limiting AOP class
import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.annotation.Aspect;
import org.aspectj.lang.annotation.Pointcut;
import org.aspectj.lang.reflect.MethodSignature;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Configuration;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.data.redis.core.script.DefaultRedisScript;
import org.springframework.web.context.request.RequestContextHolder;
import org.springframework.web.context.request.ServletRequestAttributes;
import javax.servlet.http.HttpServletRequest;
import java.io.Serializable;
import java.lang.reflect.Method;
import java.util.Collections;
import java.util.List;
@Aspect
@Configuration
public class LimitRestAspect {
private static final Logger logger = LoggerFactory.getLogger(LimitRestAspect.class);
@Autowired
private RedisTemplate<String, Object> redisTemplate;
@Autowired
private DefaultRedisScript<Number> redisluaScript;
@Pointcut(value = "@annotation(com.congge.config.limit.RedisLimitAnnotation)")
public void rateLimit() {
}
@Around("rateLimit()")
public Object interceptor(ProceedingJoinPoint joinPoint) throws Throwable {
MethodSignature signature = (MethodSignature) joinPoint.getSignature();
Method method = signature.getMethod();
Class<?> targetClass = method.getDeclaringClass();
RedisLimitAnnotation rateLimit = method.getAnnotation(RedisLimitAnnotation.class);
if (rateLimit != null) {
HttpServletRequest request = ((ServletRequestAttributes) RequestContextHolder.getRequestAttributes()).getRequest();
String ipAddress = getIpAddr(request);
StringBuffer stringBuffer = new StringBuffer();
stringBuffer.append(ipAddress).append("-")
.append(targetClass.getName()).append("- ")
.append(method.getName()).append("-")
.append(rateLimit.key());
List<String> keys = Collections.singletonList(stringBuffer.toString());
//调用lua脚本，获取返回结果，这里即为请求的次数
Number number = redisTemplate.execute(
redisluaScript,
keys,
rateLimit.count(),
rateLimit.period()
);
if (number != null && number.intValue() != 0 && number.intValue() <= rateLimit.count()) {
logger.info("限流时间段内访问了第：{} 次", number.toString());
return joinPoint.proceed();
}
} else {
return joinPoint.proceed();
}
throw new RuntimeException("访问频率过快，被限流了");
}
/**
* 获取请求的IP方法
* @param request
* @return
*/
private static String getIpAddr(HttpServletRequest request) {
String ipAddress = null;
try {
ipAddress = request.getHeader("x-forwarded-for");
if (ipAddress == null || ipAddress.length() == 0 || "unknown".equalsIgnoreCase(ipAddress)) {
ipAddress = request.getHeader("Proxy-Client-IP");
}
if (ipAddress == null || ipAddress.length() == 0 || "unknown".equalsIgnoreCase(ipAddress)) {
ipAddress = request.getHeader("WL-Proxy-Client-IP");
}
if (ipAddress == null || ipAddress.length() == 0 || "unknown".equalsIgnoreCase(ipAddress)) {
ipAddress = request.getRemoteAddr();
}
// 对于通过多个代理的情况，第一个IP为客户端真实IP,多个IP按照','分割
if (ipAddress != null && ipAddress.length() > 15) {
if (ipAddress.indexOf(",") > 0) {
ipAddress = ipAddress.substring(0, ipAddress.indexOf(","));
}
}
} catch (Exception e) {
ipAddress = "";
}
return ipAddress;
}
}
What this class has to do is similar to the above two current limiting measures, but here the core current limiting is realized by reading the lua script and passing parameters to the lua script.
4.3.5 Customize lua script
In the resources directory of the project, add the following lua script
local key = "rate.limit:" .. KEYS[1]
local limit = tonumber(ARGV[1])
local current = tonumber(redis.call('get', key) or "0")
if current + 1 > limit then
return 0
else
-- 没有超阈值，将当前访问数量+1，并设置2秒过期（可根据自己的业务情况调整）
redis.call("INCRBY", key,"1")
redis.call("expire", key,"2")
return current + 1
end
4.3.6 Add test interface
@RestController
public class RedisController {
//localhost:8081/redis/limit
@GetMapping("/redis/limit")
@RedisLimitAnnotation(key = "queryFromRedis",period = 1, count = 1)
public String queryFromRedis(){
return "success";
}
}In order to simulate the effect, set QPS to 1 here. After starting the project (starting the redis service in advance), call the interface. The normal effect is as follows:
Quickly refresh the interface, and see the following effect when more than 1 request per second
5. Custom starter current limiting implementation
The above introduces several commonly used current limiting implementations through cases, but careful students can see that these current limiting implementations are embedded in specific engineering modules. In fact, in real microservice development, a project It may contain many micro-service modules. In order to reduce repeated wheel creation and avoid separate implementation in each micro-service module, you can consider encapsulating the current-limiting logic implementation into an SDK, that is, as a springboot starter to be used by other micro-services Modules can be referenced. This is also a relatively common practice in many production practices at present. Let's take a look at the specific implementation next.
5.1 Pre-preparation
Create an empty springboot project. The project directory structure is as shown in the figure below. The directory description:
- annotation: store custom annotations related to current limiting;
- aop: stores different current limiting implementations, such as guava-based aop, sentinel-based aop implementation, etc.;
- spring.factories: customize the aop implementation class to be assembled;
5.2 Code Integration Completion Steps
5.2.1 Import basic dependencies
This includes the following necessary dependencies, and other dependencies can be reasonably selected according to your own situation;
- spring-boot-starter；
- guava；
- spring-boot-autoconfigure；
- sentinel-core；
<parent>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-parent</artifactId>
<version>2.2.1.RELEASE</version>
<relativePath/>
</parent>
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
<java.version>1.8</java.version>
</properties>
<dependencies>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-aop</artifactId>
</dependency>
<dependency>
<groupId>log4j</groupId>
<artifactId>log4j</artifactId>
<version>1.2.17</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-starter-web</artifactId>
</dependency>
<dependency>
<groupId>org.projectlombok</groupId>
<artifactId>lombok</artifactId>
</dependency>
<!-- guava-->
<dependency>
<groupId>com.google.guava</groupId>
<artifactId>guava</artifactId>
<version>23.0</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-autoconfigure</artifactId>
<version>2.2.1.RELEASE</version>
</dependency>
<dependency>
<groupId>org.springframework.boot</groupId>
<artifactId>spring-boot-configuration-processor</artifactId>
<version>2.2.1.RELEASE</version>
</dependency>
<dependency>
<groupId>com.alibaba.csp</groupId>
<artifactId>sentinel-core</artifactId>
<version>1.8.0</version>
</dependency>
<dependency>
<groupId>org.apache.commons</groupId>
<artifactId>commons-lang3</artifactId>
<version>3.4</version>
</dependency>
<dependency>
<groupId>com.alibaba.fastjson2</groupId>
<artifactId>fastjson2</artifactId>
<version>2.0.22</version>
</dependency>
</dependencies>
<build>
<resources>
<resource>
<directory>src/main/resources</directory>
<includes>
<include>**/**</include>
</includes>
</resource>
</resources>
</build>5.2.2 Custom annotations
At present, the SDK supports three current limiting methods, that is, in other subsequent microservice projects, the current limiting can be realized by adding these three kinds of annotations, which are token bucket based on guava, current limiting based on sentinel, and based on Java's own Semaphore current limiting, three custom annotation classes are as follows:
token bucket
@Documented
@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
public @interface TokenBucketLimiter {
int value() default 50;
}Semaphore
@Documented
@Target(ElementType.METHOD)
@Retention(RetentionPolicy.RUNTIME)
public @interface ShLimiter {
int value() default 50;
}sentinel
@Target(value = ElementType.METHOD)
@Retention(value = RetentionPolicy.RUNTIME)
public @interface SentinelLimiter {
String resourceName();
int limitCount() default 50;
}5.2.3 AOP implementation of current limiting
The specific current limiting is implemented in AOP, and the idea is similar to the previous chapter, that is, through the way of surrounding notifications, first analyze the methods with added current limiting annotations, and then analyze the parameters inside to realize the current limiting business.
Guava-based aop implementation
import com.alibaba.fastjson2.JSONObject;
import com.congge.annotation.TokenBucketLimiter;
import com.google.common.util.concurrent.RateLimiter;
import lombok.extern.slf4j.Slf4j;
import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.annotation.Aspect;
import org.aspectj.lang.annotation.Pointcut;
import org.springframework.cglib.core.ReflectUtils;
import org.springframework.stereotype.Component;
import org.springframework.web.context.request.RequestContextHolder;
import org.springframework.web.context.request.ServletRequestAttributes;
import javax.servlet.ServletOutputStream;
import javax.servlet.http.HttpServletResponse;
import java.io.IOException;
import java.lang.reflect.Method;
import java.util.Arrays;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
@Aspect
@Component
@Slf4j
public class GuavaLimiterAop {
private final Map<String, RateLimiter> rateLimiters = new ConcurrentHashMap<String, RateLimiter>();
@Pointcut("@annotation(com.congge.annotation.TokenBucketLimiter)")
public void aspect() {
}
@Around(value = "aspect()")
public Object around(ProceedingJoinPoint point) throws Throwable {
log.debug("准备限流");
Object target = point.getTarget();
String targetName = target.getClass().getName();
String methodName = point.getSignature().getName();
Object[] arguments = point.getArgs();
Class<?> targetClass = Class.forName(targetName);
Class<?>[] argTypes = ReflectUtils.getClasses(arguments);
Method method = targetClass.getDeclaredMethod(methodName, argTypes);
// 获取目标method上的限流注解@Limiter
TokenBucketLimiter limiter = method.getAnnotation(TokenBucketLimiter.class);
RateLimiter rateLimiter = null;
Object result = null;
if (null != limiter) {
// 以 class + method + parameters为key，避免重载、重写带来的混乱
String key = targetName + "." + methodName + Arrays.toString(argTypes);
rateLimiter = rateLimiters.get(key);
if (null == rateLimiter) {
// 获取限定的流量
// 为了防止并发
rateLimiters.putIfAbsent(key, RateLimiter.create(limiter.value()));
rateLimiter = rateLimiters.get(key);
}
boolean b = rateLimiter.tryAcquire();
if(b){
log.debug("得到令牌，准备执行业务");
result = point.proceed();
}else {
HttpServletResponse resp = ((ServletRequestAttributes) RequestContextHolder.getRequestAttributes()).getResponse();
JSONObject jsonObject=new JSONObject();
jsonObject.put("success",false);
jsonObject.put("msg","限流中");
try {
output(resp, jsonObject.toJSONString());
}catch (Exception e){
log.error("error,e:{}",e);
}
}
} else {
result = point.proceed();
}
log.debug("退出限流");
return result;
}
public void output(HttpServletResponse response, String msg) throws IOException {
response.setContentType("application/json;charset=UTF-8");
ServletOutputStream outputStream = null;
try {
outputStream = response.getOutputStream();
outputStream.write(msg.getBytes("UTF-8"));
} catch (IOException e) {
e.printStackTrace();
} finally {
outputStream.flush();
outputStream.close();
}
}
}Aop implementation based on Semaphore
import com.congge.annotation.ShLimiter;
import lombok.extern.slf4j.Slf4j;
import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.annotation.Aspect;
import org.aspectj.lang.annotation.Pointcut;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.cglib.core.ReflectUtils;
import org.springframework.stereotype.Component;
import java.lang.reflect.Method;
import java.util.Arrays;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.Semaphore;
@Aspect
@Component
@Slf4j
public class SemaphoreLimiterAop {
private final Map<String, Semaphore> semaphores = new ConcurrentHashMap<String, Semaphore>();
private final static Logger LOG = LoggerFactory.getLogger(SemaphoreLimiterAop.class);
@Pointcut("@annotation(com.congge.annotation.ShLimiter)")
public void aspect() {
}
@Around(value = "aspect()")
public Object around(ProceedingJoinPoint point) throws Throwable {
log.debug("进入限流aop");
Object target = point.getTarget();
String targetName = target.getClass().getName();
String methodName = point.getSignature().getName();
Object[] arguments = point.getArgs();
Class<?> targetClass = Class.forName(targetName);
Class<?>[] argTypes = ReflectUtils.getClasses(arguments);
Method method = targetClass.getDeclaredMethod(methodName, argTypes);
// 获取目标method上的限流注解@Limiter
ShLimiter limiter = method.getAnnotation(ShLimiter.class);
Object result = null;
if (null != limiter) {
// 以 class + method + parameters为key，避免重载、重写带来的混乱
String key = targetName + "." + methodName + Arrays.toString(argTypes);
// 获取限定的流量
Semaphore semaphore = semaphores.get(key);
if (null == semaphore) {
semaphores.putIfAbsent(key, new Semaphore(limiter.value()));
semaphore = semaphores.get(key);
}
try {
semaphore.acquire();
result = point.proceed();
} finally {
if (null != semaphore) {
semaphore.release();
}
}
} else {
result = point.proceed();
}
log.debug("退出限流");
return result;
}
}
AOP implementation based on sentinel
import com.alibaba.csp.sentinel.Entry;
import com.alibaba.csp.sentinel.SphU;
import com.alibaba.csp.sentinel.Tracer;
import com.alibaba.csp.sentinel.slots.block.BlockException;
import com.alibaba.csp.sentinel.slots.block.RuleConstant;
import com.alibaba.csp.sentinel.slots.block.flow.FlowRule;
import com.alibaba.csp.sentinel.slots.block.flow.FlowRuleManager;
import com.congge.annotation.SentinelLimiter;
import org.apache.commons.lang3.StringUtils;
import org.aspectj.lang.JoinPoint;
import org.aspectj.lang.ProceedingJoinPoint;
import org.aspectj.lang.annotation.Around;
import org.aspectj.lang.annotation.Aspect;
import org.aspectj.lang.annotation.Pointcut;
import org.springframework.stereotype.Component;
import java.lang.reflect.Method;
import java.util.ArrayList;
import java.util.List;
import java.util.Objects;
@Aspect
@Component
public class SentinelLimiterAop {
private static void initFlowRule(String resourceName,int limitCount) {
List<FlowRule> rules = new ArrayList<>();
FlowRule rule = new FlowRule();
//设置受保护的资源
rule.setResource(resourceName);
//设置流控规则 QPS
rule.setGrade(RuleConstant.FLOW_GRADE_QPS);
//设置受保护的资源阈值
rule.setCount(limitCount);
rules.add(rule);
//加载配置好的规则
FlowRuleManager.loadRules(rules);
}
@Pointcut(value = "@annotation(com.congge.annotation.SentinelLimiter)")
public void rateLimit() {
}
@Around("rateLimit()")
public Object around(ProceedingJoinPoint joinPoint) {
//1、获取当前的调用方法
Method currentMethod = getCurrentMethod(joinPoint);
if (Objects.isNull(currentMethod)) {
return null;
}
//2、从方法注解定义上获取限流的类型
String resourceName = currentMethod.getAnnotation(SentinelLimiter.class).resourceName();
if(StringUtils.isEmpty(resourceName)){
throw new RuntimeException("资源名称为空");
}
int limitCount = currentMethod.getAnnotation(SentinelLimiter.class).limitCount();
initFlowRule(resourceName,limitCount);
Entry entry = null;
Object result = null;
try {
entry = SphU.entry(resourceName);
try {
result = joinPoint.proceed();
} catch (Throwable throwable) {
throwable.printStackTrace();
}
} catch (BlockException ex) {
// 资源访问阻止，被限流或被降级
// 在此处进行相应的处理操作
System.out.println("blocked");
return "被限流了";
} catch (Exception e) {
Tracer.traceEntry(e, entry);
} finally {
if (entry != null) {
entry.exit();
}
}
return result;
}
private Method getCurrentMethod(JoinPoint joinPoint) {
Method[] methods = joinPoint.getTarget().getClass().getMethods();
Method target = null;
for (Method method : methods) {
if (method.getName().equals(joinPoint.getSignature().getName())) {
target = method;
break;
}
}
return target;
}
}
5.2.4 Configuring automatic assembly AOP implementation
Create the above spring.factories file in the resources directory, the content is as follows, after configuration in this way, other application modules can be used out of the box after introducing the jar of the current SDK;
org.springframework.boot.autoconfigure.EnableAutoConfiguration=\
com.congge.aop.SemaphoreLimiterAop,\
com.congge.aop.GuavaLimiterAop,\
com.congge.aop.SemaphoreLimiterAop5.2.5 Make the project into a jar for installation
This step is easier to skip
5.2.6 Introduce the above SDK into other projects
<dependency>
<groupId>cm.congge</groupId>
<artifactId>biz-limit</artifactId>
<version>1.0-SNAPSHOT</version>
</dependency>5.2.7 Write test interface
In other projects, write a test interface and use the above annotations. Here we take guava's current-limiting annotations as an example to illustrate
import com.congge.annotation.TokenBucketLimiter;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;
@RestController
public class SdkController {
//localhost:8081/query
@GetMapping("/query")
@TokenBucketLimiter(1)
public String queryUser(){
return "queryUser";
}
}5.2.8 Function test
After starting the current project, call the interface normally, request once per second, and get the result normally
Quickly refresh the interface, after the QPS exceeds 1, the current limit will be triggered, and the following effect will be seen
Through the above method, the expected effect can also be obtained. Students who are interested in the other two current-limiting annotations can also continue to test and verify, and the reason for the space will not be repeated.
The above-mentioned starter method implements a more elegant current-limiting integration method, which is also a recommended method in production, but the current case is still relatively rough, and students who need to use it need to improve the logic in it according to their own situation. Further encapsulation in order to get better results.
Sixth, write at the end of the text
This paper elaborates some implementation schemes of current limiting in microservices in a large space combined with actual cases. Current limiting is of great significance to a stable operating system. It can be said to be an important aspect of service governance. Hope It was helpful to the students who saw it, thank you for watching.