flow control
Used to adjust the data sent by network packets. Requests coming at any time are often random and uncontrollable, and the processing capacity of the system is limited. We need to control the traffic according to the processing capacity of the system. As an adapter, Sentinel can adjust random requests into appropriate shapes as needed, as follows:
There are two types of statistics for flow control - counting the number of threads and counting QPS. You can view real-time statistics with the following command:
curl http://localhost:8719/cnode?id=resourceName
The format of the output is as follows:
idx id thread pass blocked success total Rt 1m-pass 1m-block 1m-all exeption
2 abc647 0 46 0 46 46 1 2763 0 2763 0
- thread: represents the number of threads currently processing the resource;
- pass: represents the request that arrives within one second;
- blocked: Represents the number of requests controlled by flow within one second;
- success: Represents requests that are successfully processed within one second;
- total: represents the sum of incoming requests and blocked requests within one second;
- RT: represents the average response time of the resource within one second;
- 1m-pass: Represents requests that arrive within one minute;
- 1m-block: Represents blocked requests within one minute;
- 1m-all: represents the sum of incoming requests and blocked requests within one minute;
- exception: Represents the sum of exceptions in the business itself within one second.
Definition of Traffic Rules
Corresponds to FlowRule in the actual code.
The same resource can have multiple traffic rules at the same time.
| Field | illustrate | Defaults |
|---|---|---|
| resource | Resource Name | |
| count | current limit threshold | |
| grade | Indicator for flow control (0 - number of threads, 1 - QPS) | 1 (i.e. QPS) |
| limitApp | The source of the call for flow control | default (does not distinguish the source of the call) |
| strategy | Resource call relationship (0 - direct, 1 - association, 2 - link) | 0 (i.e. direct) |
| controlBehavior | Effect of flow control (0 - direct rejection, 1 - cold start, 2 - uniform speed, 3 - cold start + uniform speed) | 0 (i.e. direct rejection) |
Indicators for Flow Control
Based on the number of concurrent threads
Simply count the number of threads in the current request context. If the threshold is exceeded, the new request will be rejected. Used to protect the number of business threads from being exhausted.
For example, the downstream application that the application depends on causes unstable service and increased response time for some reason. For the caller, it means that the throughput decreases and more threads are occupied. In extreme cases, the thread pool is exhausted. In order to deal with high thread occupation, there are isolation solutions in the industry, such as using different thread pools for different business logics to isolate resource contention between businesses (thread pool isolation), or using semaphores to control the number of simultaneous requests (semaphore isolation). Although the semaphore isolation strategy can control the number of threads, it cannot control the queuing time of requests. When there are too many requests, queuing is also useless, and direct rejection can quickly reduce system pressure. Sentinel thread limit is not responsible for creating and managing the thread pool, but simply counts the number of threads in the current request context. If it exceeds the threshold, the new request will be rejected immediately.
The effect of flow control is only direct denial.
[
{
"resource": "sayHello",
"limitApp": "default",
"grade": 0,
"count": 4,
"strategy": 0,
"controlBehavior": 0
}
]
Based on QPS
When the QPS exceeds the threshold, measures are taken for flow control.
The effect of flow control
directly refuse
correspond RuleConstant.CONTROL_BEHAVIOR_DEFAULT. This method is the default flow control method. When the QPS exceeds the threshold, new requests are directly rejected (the rejection method is to throw FlowException). This method is suitable for situations where the processing capacity of the system is known exactly, such as when the exact water level of the system is determined through pressure measurement.
[
{
"resource": "sayHello",
"limitApp": "default",
"grade": 1,
"count": 4,
"strategy": 0,
"controlBehavior": 0
}
]
Cold start
correspond RuleConstant.CONTROL_BEHAVIOR_WARM_UP. This method is used when the system is at a low water level for a long time. When the flow suddenly increases, the system is directly raised to a high water level. This situation may instantly crush the system. The cold start method allows the passing flow to increase slowly, and gradually increase to the upper threshold within a certain period of time ( warmUpPeriodSecthe parameter controls the warm-up time, in seconds), giving the cold system a warm-up time to avoid the cold system from being overwhelmed.
[
{
"resource": "sayHello",
"limitApp": "default",
"grade": 1,
"count": 10,
"strategy": 0,
"controlBehavior": 1,
"warmUpPeriodSec": 3
}
]
Uniform velocity device
correspond RuleConstant.CONTROL_BEHAVIOR_RATE_LIMITER. Strictly control the interval time for requests to pass through, that is, let requests pass through at a uniform speed, which corresponds to the leaky bucket algorithm. This method is mainly used to handle intermittent burst traffic, such as message queues. Imagine a scenario where a large number of requests arrive in a certain second and are idle for the next few seconds. We hope to process these requests gradually during the next idle period instead of directly rejecting them in the first second Redundant request.
[
{
"resource": "sayHello",
"limitApp": "default",
"grade": 1,
"count": 10,
"strategy": 0,
"controlBehavior": 2,
"maxQueueingTimeMs": 500
}
]
When the number of requests per second is greater than 5, excess requests are queued. If these requests are not processed within 500 milliseconds, the requests will be throttled and the access will fail.
Cold start + uniform speed
correspond RuleConstant.CONTROL_BEHAVIOR_WARM_UP_RATE_LIMITER. Combination of cold start and homogenizer.
[
{
"resource": "sayHello",
"limitApp": "default",
"grade": 1,
"count": 10,
"strategy": 0,
"controlBehavior": 3,
"maxQueueingTimeMs": 500,
"warmUpPeriodSec": 5
}
]
Resource call relationship
The call relationship includes the caller and the callee; a method may call other methods to form a hierarchical relationship of a call link.
Throttle based on caller
You can use the following command to view the call data of different callers for the same resource.
curl http://localhost:8719/origin?id=nodeA
An example call data is as follows:
id: nodeA
idx origin threadNum passedQps blockedQps totalQps aRt 1m-passed 1m-blocked 1m-total
1 caller1 0 0 0 0 0 0 0 0
2 caller2 0 0 0 0 0 0 0 0
The limitApp field in the rate limiting rule is used for flow control according to the caller. The value of this field has the following three options, corresponding to different scenarios:
- default: Indicates that the caller is not distinguished, and requests from any caller will be subject to current limiting statistics. If the sum of the calls of this resource name exceeds the threshold, the current limit will be triggered.
- {some_origin_name}: Indicates the target caller, and only requests from this caller will perform flow control. For example, if NodeA configures a rule for caller1, then flow control will be triggered if and only if the request from caller1 to NodeA is made.
- other: Indicates that flow control is performed on the traffic of other callers except {some_origin_name}. For example, resource NodeA is configured with a current limiting rule for the caller caller1, and at the same time configures a rule for the caller as other, then any call to NodeA from a non-caller1 cannot exceed the threshold defined by the other rule.
Multiple rules can be configured for the same resource name, and the order in which the rules take effect is: {some_origin_name} > other > default
[
{
"resource": "sayHello",
"limitApp": "spring-cloud-demo-consumer",
"grade": 1,
"count": 5,
"controlBehavior": 2,
"maxQueueingTimeMs": 500
},
{
"resource": "sayHello",
"limitApp": "other",
"grade": 1,
"count": 5,
"controlBehavior": 2,
"maxQueueingTimeMs": 500
}
]
An additional Bean of type RequestOriginParser needs to be registered.
@Component
public class MyRequestOriginParser implements RequestOriginParser {
@Override
public String parseOrigin(HttpServletRequest httpServletRequest) {
String origin = httpServletRequest.getHeader("origin");
if (StringUtils.isBlank(origin)) {
origin = "default";
}
return origin;
}
}
Then the request header carries the origin attribute and attribute value.
link current limiting
A typical call tree is shown in the following figure:
machine-root
/ \
/ \
Entrance1 Entrance2
/ \
/ \
DefaultNode(nodeA) DefaultNode(nodeA)
In the above figure Entrance1, Entrance2the requests from the portals and are called to resources NodeA, and Sentinel allows the resource flow to be limited only based on the statistical information of a certain portal. For example, we can set FlowRule.strategyto atRuleConstant.CHAIN the same time to indicate that only the calls from will be recorded in the current limiting statistics of the , and we are indifferent to the calls from .refResourceEntrance1Entrance1NodeAEntrance2
[
{
"resource": "sayHello",
"limitApp": "default",
"strategy": 2,
"refResource": "/hello/say",
"grade": 1,
"count": 3,
"controlBehavior": 2,
"maxQueueingTimeMs": 500
}
]
Associated current limiting
When there is resource contention or dependency between two resources, the two resources are associated.
For example, there is contention between read and write operations on the same field in the database. If the reading speed is too high, the writing speed will be affected, and if the writing speed is too high, the reading speed will be affected. If read and write operations are allowed to compete for resources, the overhead caused by the contention itself will reduce the overall throughput. Associated current limiting can be used to avoid excessive contention between resources with associated relationships. For example, the two resources read_db and write_db represent database read and write respectively. We can set current limiting rules for read_db to achieve the purpose of writing priority : Set FlowRule.strategy to RuleConstant.RELATE, and set refResource to write_db. In this way, when the operation of writing to the library is too frequent, the request for reading data will be limited.
[
{
"resource": "/hello/say",
"limitApp": "default",
"strategy": 1,
"refResource": "/hello/say2",
"grade": 1,
"count": 1
}
]
When the access to the /hello/say2 interface reaches the current limiting threshold (the current is not limited), the /hello/say interface is limited according to the configured current limiting means.
FlowSlot
Responsible for judging the current limiting rules.
@Override
public void entry(Context context, ResourceWrapper resourceWrapper, DefaultNode node, int count,
boolean prioritized, Object... args) throws Throwable {
// 校验限流规则
checkFlow(resourceWrapper, context, node, count, prioritized);
// 交给下一个ProcessorSlot处理
fireEntry(context, resourceWrapper, node, count, prioritized, args);
}
void checkFlow(ResourceWrapper resource, Context context, DefaultNode node, int count, boolean prioritized)
throws BlockException {
// 交给FlowRuleChecker继续处理
checker.checkFlow(ruleProvider, resource, context, node, count, prioritized);
}
checkFlow
public void checkFlow(Function<String, Collection<FlowRule>> ruleProvider, ResourceWrapper resource,
Context context, DefaultNode node, int count, boolean prioritized) throws BlockException {
if (ruleProvider == null || resource == null) {
return;
}
Collection<FlowRule> rules = ruleProvider.apply(resource.getName());
if (rules != null) {
// 只要有一个流量规则校验失败，则抛出FlowException异常
for (FlowRule rule : rules) {
if (!canPassCheck(rule, context, node, count, prioritized)) {
throw new FlowException(rule.getLimitApp(), rule);
}
}
}
}
canPassCheck
public boolean canPassCheck(/*@NonNull*/ FlowRule rule, Context context, DefaultNode node,
int acquireCount) {
return canPassCheck(rule, context, node, acquireCount, false);
}
public boolean canPassCheck(/*@NonNull*/ FlowRule rule, Context context, DefaultNode node, int acquireCount,
boolean prioritized) {
String limitApp = rule.getLimitApp();
if (limitApp == null) {
return true;
}
// 如果是集群模式
if (rule.isClusterMode()) {
// 进行集群模式下的流量规则校验（该部分在集群流控文章中进行分析）
return passClusterCheck(rule, context, node, acquireCount, prioritized);
}
// 进行本地模式下的流量规则校验
return passLocalCheck(rule, context, node, acquireCount, prioritized);
}
passLocalCheck
private static boolean passLocalCheck(FlowRule rule, Context context, DefaultNode node, int acquireCount,
boolean prioritized) {
// 根据限流规则中的limitApp、strategy参数值，选择合适的节点（DefaultNode、ClusterNode）
Node selectedNode = selectNodeByRequesterAndStrategy(rule, context, node);
// 如果选择的节点为空，则返回true，表示校验成功
if (selectedNode == null) {
return true;
}
// 根据具体的TrafficShapingController接口的实现类进行校验
return rule.getRater().canPass(selectedNode, acquireCount, prioritized);
}
selectNodeByRequesterAndStrategy
static Node selectNodeByRequesterAndStrategy(/*@NonNull*/ FlowRule rule, Context context, DefaultNode node) {
String limitApp = rule.getLimitApp();
int strategy = rule.getStrategy();
String origin = context.getOrigin();
// 如果限流规则中的limitApp参数值等于上下文记录的请求来源，并且上下文记录的请求来源不是DEFAULT、OTHER
if (limitApp.equals(origin) && filterOrigin(origin)) {
// 如果调用关系是根据调用方限流
if (strategy == RuleConstant.STRATEGY_DIRECT) {
// 返回上下文记录的起源节点
return context.getOriginNode();
}
return selectReferenceNode(rule, context, node);
// 如果限流规则中的limitApp参数值等于DEFAULT
} else if (RuleConstant.LIMIT_APP_DEFAULT.equals(limitApp)) {
// 如果调用关系是根据调用方限流
if (strategy == RuleConstant.STRATEGY_DIRECT) {
// 返回对应的集群节点
return node.getClusterNode();
}
return selectReferenceNode(rule, context, node);
// 如果限流规则中的limitApp参数值等于OTHER
} else if (RuleConstant.LIMIT_APP_OTHER.equals(limitApp)
&& FlowRuleManager.isOtherOrigin(origin, rule.getResource())) {
// 如果调用关系是根据调用方限流
if (strategy == RuleConstant.STRATEGY_DIRECT) {
// 返回上下文记录的起源节点
return context.getOriginNode();
}
return selectReferenceNode(rule, context, node);
}
// 其余情况返回空
return null;
}
isOtherOrigin
public static boolean isOtherOrigin(String origin, String resourceName) {
if (StringUtil.isEmpty(origin)) {
return false;
}
List<FlowRule> rules = flowRules.get().get(resourceName);
if (rules != null) {
for (FlowRule rule : rules) {
if (origin.equals(rule.getLimitApp())) {
return false;
}
}
}
return true;
}
selectReferenceNode
static Node selectReferenceNode(FlowRule rule, Context context, DefaultNode node) {
// 获取流量控制规则的refResource参数值
String refResource = rule.getRefResource();
// 获取流量控制规则的strategy参数值
int strategy = rule.getStrategy();
// 如果refResource参数值为空，则返回空
if (StringUtil.isEmpty(refResource)) {
return null;
}
// 如果是关联限流
if (strategy == RuleConstant.STRATEGY_RELATE) {
// 获取refResource参数值对应的集群节点
return ClusterBuilderSlot.getClusterNode(refResource);
}
// 如果是链路限流
if (strategy == RuleConstant.STRATEGY_CHAIN) {
// 如果refResource参数值不等于上下文名称，则返回空
if (!refResource.equals(context.getName())) {
return null;
}
// 返回DefaultNode
return node;
}
// 返回空
return null;
}
According to the effect of traffic control, there are four corresponding implementation classes of the TrafficShapingController interface.
all
Analysis of the implementation class of the TrafficShapingController interface.