How to configure local rate limit

Installing the demo program

If you haven’t installed the demo program yet, Please refer to Quick Start to install Aeraki, Istio, and the demo.

After installation, you can see that the following two NSs have been added to the cluster, and the Dubbo and Thrift demo applications are installed in these two NSs. You can choose either of them to test.

  1. ➜  ~ kubectl get ns|grep meta
  2. meta-dubbo        Active   16m
  3. meta-thrift       Active   16m

Aeraki’s rate limiting rules are designed to be flexible and easy to use, supporting both rate limiting for all inbound requests to a service and fine-grained rate limiting control based on specified conditions.

Enforce rate limiting on all inbound requests of a service

The following rule limits all inbound requests to the thrift-sample-server.meta-thrift.svc.cluster.local service to 2 requests / minute.

  1. kubectl apply -f- <<EOF
  2. apiVersion: metaprotocol.aeraki.io/v1alpha1
  3. kind: MetaRouter
  4. metadata:
  5.   name: test-metaprotocol-thrift-route
  6.   namespace: meta-thrift
  7. spec:
  8.   hosts:
  9.   - thrift-sample-server.meta-thrift.svc.cluster.local
  10.   localRateLimit:
  11.     tokenBucket:
  12.       fillInterval: 60s
  13.       maxTokens: 2
  14.       tokensPerFill: 2
  15. EOF

Note: Because local rate limiting is handled separately on each service instance, when the service has multiple instances, the actual rate limiting effect is the number of rate limiting times the number of instances.

Using the aerakictl command to view the client-side log, you can see that the client can only successfully send out 4 requests per minute (with two service instances, each limited to 2 requests per minute).

  1. ➜  ~ aerakictl_app_log client meta-thrift -f --tail 10
  2. Hello Aeraki, response from thrift-sample-server-v1-5c8476684-842l6/172.17.0.40
  3. Hello Aeraki, response from thrift-sample-server-v2-6d5bcc885-hpx7n/172.17.0.41
  4. Hello Aeraki, response from thrift-sample-server-v1-5c8476684-842l6/172.17.0.40
  5. Hello Aeraki, response from thrift-sample-server-v2-6d5bcc885-hpx7n/172.17.0.41
  6. org.apache.thrift.TApplicationException: meta protocol local rate limit: request '5' has been rate limited
  7.         at org.apache.thrift.TServiceClient.receiveBase(TServiceClient.java:79)
  8.         at org.aeraki.HelloService$Client.recv_sayHello(HelloService.java:61)
  9.         at org.aeraki.HelloService$Client.sayHello(HelloService.java:48)
  10.         at org.aeraki.HelloClient.main(HelloClient.java:44)
  11. Connected to thrift-sample-server
  12. org.apache.thrift.TApplicationException: meta protocol local rate limit: request '1' has been rate limited
  13. ...

Rate limiting by condition

Aeraki supports setting multiple rate limiting rules for a service, each with different conditions. This is super useful if you need to enforce fine-grained rate limiting on a service. For example, you can set different quota for users based on their SLA(Service Level Agreement).

Just like routing condtions, any attributes that can be extracted from the request packet can be used for the matching conditions of the rate limiting rule.

For example, the following rules set different rate limiting for the sayHello and ping interfaces.

  1. apiVersion: metaprotocol.aeraki.io/v1alpha1
  2. kind: MetaRouter
  3. metadata:
  4.   name: test-metaprotocol-thrift-route
  5.   namespace: meta-thrift
  6. spec:
  7.   hosts:
  8.   - thrift-sample-server.meta-thrift.svc.cluster.local
  9.   localRateLimit:
  10.     conditions:
  11.     - match:
  12.         attributes:
  13.           method:
  14.             exact: sayHello
  15.       tokenBucket:
  16.         fillInterval: 60s
  17.         maxTokens: 10
  18.         tokensPerFill: 10
  19.     - match:
  20.         attributes:
  21.           method:
  22.             exact: ping
  23.       tokenBucket:
  24.         fillInterval: 60s
  25.         maxTokens: 100
  26.         tokensPerFill: 100

Set rate limiting at both the service level and condition level

It is possible to set both service level and condition level rate limiting, which is useful when you need to set a global quota for the whole service, but with a few exceptions.

For example, the following rate limiting rule sets an overall rate limiting rule for the service of 1000 messages/minute, with an exception of 100 messages/minute for the ping interface.

  1. apiVersion: metaprotocol.aeraki.io/v1alpha1
  2. kind: MetaRouter
  3. metadata:
  4.   name: test-metaprotocol-thrift-route
  5.   namespace: meta-thrift
  6. spec:
  7.   hosts:
  8.   - thrift-sample-server.meta-thrift.svc.cluster.local
  9.   localRateLimit:
  10.     tokenBucket:
  11.       fillInterval: 60s
  12.       maxTokens: 1000
  13.       tokensPerFill: 1000
  14.     conditions:
  15.     - match:
  16.         attributes:
  17.           method:
  18.             exact: ping
  19.       tokenBucket:
  20.         fillInterval: 60s
  21.         maxTokens: 100
  22.         tokensPerFill: 100

Understand what happened

In the configuration sent to the Sidecar Proxy, Aeraki sets the MetaProtocol Proxy in the FilterChain corresponding to the service in the VirtualInbound Listener.

Aeraki translates the rate limiting rules configured in the MetaRouter into configuration for the local rate limit filter, and distributes the configuration to sidecar proxies.

The configuration of the service’s sidecar proxy can be viewed with the following command.

  1. aerakictl_sidecar_config server-v1 meta-thrift |fx

The configuration of the MetaProtocol Proxy in the Inbound Listener of the Thrift service is shown below.

  1. {
  2.  "name": "envoy.filters.network.meta_protocol_proxy",
  3.  "typed_config": {
  4.   "@type": "type.googleapis.com/udpa.type.v1.TypedStruct",
  5.   "type_url": "type.googleapis.com/aeraki.meta_protocol_proxy.v1alpha.MetaProtocolProxy",
  6.   "value": {
  7.    "stat_prefix": "inbound|9090||",
  8.    "application_protocol": "thrift",
  9.    "route_config": {
  10.     "name": "inbound|9090||",
  11.     "routes": [
  12.      {
  13.       "route": {
  14.        "cluster": "inbound|9090||"
  15.       }
  16.      }
  17.     ]
  18.    },
  19.    "codec": {
  20.     "name": "aeraki.meta_protocol.codec.thrift"
  21.    },
  22.    "meta_protocol_filters": [
  23.     {
  24.      "name": "aeraki.meta_protocol.filters.local_ratelimit",
  25.      "config": {
  26.       "@type": "type.googleapis.com/aeraki.meta_protocol_proxy.filters.local_ratelimit.v1alpha.LocalRateLimit",
  27.       "stat_prefix": "thrift-sample-server.meta-thrift.svc.cluster.local",
  28.       "token_bucket": {
  29.        "max_tokens": 2,
  30.        "tokens_per_fill": 2,
  31.        "fill_interval": "60s"
  32.       }
  33.      }
  34.     },
  35.     {
  36.      "name": "aeraki.meta_protocol.filters.router"
  37.     }
  38.    ]
  39.   }
  40.  }
  41. }

Last modified May 9, 2022: Update local-rate-limit.md (02fc310)