Introduction

Based on the real-time statistics collected in Sentinel, FlowSlot will use pre-set rules to decide whether the incoming requests should be controlled.

One resource can have multiple flow control rules. The FlowSlot will traverse these rules until one of them is triggered or all rules have been passed.

A flow control rule mainly consists of the following items, and you can use a combination of different items to satisfy different flow control needs.

  • resource：resource name
  • count: the threshold
  • grade: metric type (QPS or concurrency)
  • strategy: select flow control strategy based on invocation relation (invocation chain)
  • controlBehavior: effect of traffic shaping

Flow Control by Thread/QPS

Metric Type

You can control either by QPS and concurrency. It is defined by the grade field in FlowRule. Both concurrent thread count and request count are collected at runtime.

Concurrency

This mode is usually used to prevent thread pools from being occupied. If an invocation takes a long time to finish, threads blocked on the invocation will be congested. The longer the response takes, the more threads might be occupied.

Besides concurrency (aka. semaphore isolation), there are other ways to achieve this: thread pool isolation.

  • Thread pool isolation: Allocate managed thread pools to handle different invocations. When there are no more idle threads in the pool, the request is rejected without affecting other resources. When an invocation exceeds the timeout, the invocation will be cut down.
  • Semaphore isolation: Limit the concurrency count of the invocation.

The benefit of using thread pools is that you can isolate business logic completely by pre-allocating thread pools. But it also brings us extra costs of context switch and additional threads. If the incoming request is already handled in a separate thread, for instance, a servlet request, it will almost double the thread overhead if using the thread pool mode.

So we recommend using concurrent thread count flow control, which represents lightweight semaphore isolation.

QPS

When QPS exceeds the threshold, we will take actions to control the incoming requests, and this can be done by configuring the controlBehavior field in FlowRule.

1.　Immediately reject (RuleConstant.CONTROL_BEHAVIOR_DEFAULT)

This is the default behavior. The exceeded requests are rejected immediately and the FlowException is thrown.

2.　WarmUp（RuleConstant.CONTROL_BEHAVIOR_WARM_UP)

If the usage of the system has been low for a while, but all of a sudden a large amount of requests comes in, the system might not be able to handle all these requests at once. However, if we steadily increase the incoming requests and allows the system to warm up, it may be able to handle all the requests eventually.

This warm-up period can be configured by setting the warmUpPeriodSec field in FlowRule.

WarmUp

3.Rate limiter (RuleConstant.CONTROL_BEHAVIOR_RATE_LIMITER)

This strategy strictly controls the interval between requests. In other words, it allows requests to pass at a uniform rate.

Uniform rate

This strategy is an implementation of leaky bucket. It is used to handle requests at a stable rate and is often used to process burst requests instead of rejecting them. For instance, a sudden inflow of messages. When a large number of requests arrive at the same time, the system can handle all these incoming requests at a fixed rate.

Flow Control by Call Path

We use the NodeSelectorSlot to establish the invocation chain of resources, and the ClusterBuilderSlot to collect the caller’s runtime statistic data.

By caller

When calling ContextUtil.enter(resourceName, origin), the parameter origin indicates the identity of the caller (e.g. service name of peer side). The ClusterBuilderSlot will collect this info, and use it to perform flow control.

This information can be displayed by the following command:

  1. id: nodeA
  2. idx origin  threadNum passedQps blockedQps totalQps aRt   1m-passed 1m-blocked 1m-total 
  3. 1   caller1 0         0         0          0        0     0         0          0        
  4. 2   caller2 0         0         0          0        0     0         0          0

The origin can be defined by the field limitApp in FlowRule. This field has the following values:

  • default: No specific caller. If the total value of this resource exceeds the threshold defined in this rule, the incoming request will be blocked.
  • <<origin>>: A specific caller has exceeded the threshold defined in the rule.
  • other: This rule applies to requests from a caller that is not defined explicitly in the origin field for this resource.

By entrance

The path is maintained in NodeSelectorSlot. For example, the resource nodes can come from either entrance1 or entrance2.

  1.                    machine-root
  2.                     /       \   
  3.                    /         \
  4.              Entrance1     Entrance2
  5.                 /             \
  6.                /               \
  7.       DefaultNode(nodeA)   DefaultNode(nodeA)

We can shape the flow by setting the field strategy to RuleConstant.CHAIN, and ref_identity to a specified entrance.

By related resource

For instance, two resources will access the same database record. ResourceA will read records from the database, resourceB will write records to the database. The frequency of ResourceA accessing the database depends on ResourceB. We can achieve this by configuring a rule for ResourceA with the value of the strategy field as RuleConstant.RELATE, and the value of ref_identity as ResourceB.