Enabling Rate Limits using Envoy

This task shows you how to use Envoy’s native rate limiting to dynamically limit the traffic to an Istio service. In this task, you will apply a global rate-limit for the productpage service through ingress gateway that allows 1 requests per minute across all instances of the service. Additionally, you will apply a local rate-limit for each individual productpage instance that will allow 4 requests per minute. In this way, you will ensure that the productpage service handles a maximum of 1 request per minute through the ingress gateway, but each productpage instance can handle up to 4 requests per minute, allowing for any in-mesh traffic.

Before you begin

  1. Setup Istio in a Kubernetes cluster by following the instructions in the Installation Guide.

  2. Deploy the Bookinfo sample application.

Rate limits

Envoy supports two kinds of rate limiting: global and local. Global rate limiting uses a global gRPC rate limiting service to provide rate limiting for the entire mesh. Local rate limiting is used to limit the rate of requests per service instance. Local rate limiting can be used in conjunction with global rate limiting to reduce load on the global rate limiting service.

In this task you will configure Envoy to rate limit traffic to a specific path of a service using both global and local rate limits.

Global rate limit

Envoy can be used to set up global rate limits for your mesh. Global rate limiting in Envoy uses a gRPC API for requesting quota from a rate limiting service. A reference implementation of the API, written in Go with a Redis backend, is used below.

  1. Use the following configmap to configure the reference implementation to rate limit requests to the path /productpage at 1 req/min, a value api for the coming advanced example, and all other requests at 100 req/min.

    1. $ kubectl apply -f - <<EOF
    2. apiVersion: v1
    3. kind: ConfigMap
    4. metadata:
    5.   name: ratelimit-config
    6. data:
    7.   config.yaml: |
    8.     domain: ratelimit
    9.     descriptors:
    10.       - key: PATH
    11.         value: "/productpage"
    12.         rate_limit:
    13.           unit: minute
    14.           requests_per_unit: 1
    15.       - key: PATH
    16.         value: "api"
    17.         rate_limit:
    18.           unit: minute
    19.           requests_per_unit: 2
    20.       - key: PATH
    21.         rate_limit:
    22.           unit: minute
    23.           requests_per_unit: 100
    24. EOF

  2. Create a global rate limit service which implements Envoy’s rate limit service protocol. As a reference, a demo configuration can be found here, which is based on a reference implementation provided by Envoy.

    Zip

    1. $ kubectl apply -f @samples/ratelimit/rate-limit-service.yaml@

  3. Apply an EnvoyFilter to the ingressgateway to enable global rate limiting using Envoy’s global rate limit filter.

    The patch inserts the envoy.filters.http.ratelimit global envoy filter into the HTTP_FILTER chain. The rate_limit_service field specifies the external rate limit service, outbound|8081||ratelimit.default.svc.cluster.local in this case.

    1. $ kubectl apply -f - <<EOF
    2. apiVersion: networking.istio.io/v1alpha3
    3. kind: EnvoyFilter
    4. metadata:
    5.   name: filter-ratelimit
    6.   namespace: istio-system
    7. spec:
    8.   workloadSelector:
    9.     # select by label in the same namespace
    10.     labels:
    11.       istio: ingressgateway
    12.   configPatches:
    13.     # The Envoy config you want to modify
    14.     - applyTo: HTTP_FILTER
    15.       match:
    16.         context: GATEWAY
    17.         listener:
    18.           filterChain:
    19.             filter:
    20.               name: "envoy.filters.network.http_connection_manager"
    21.               subFilter:
    22.                 name: "envoy.filters.http.router"
    23.       patch:
    24.         operation: INSERT_BEFORE
    25.         # Adds the Envoy Rate Limit Filter in HTTP filter chain.
    26.         value:
    27.           name: envoy.filters.http.ratelimit
    28.           typed_config:
    29.             "@type": type.googleapis.com/envoy.extensions.filters.http.ratelimit.v3.RateLimit
    30.             # domain can be anything! Match it to the ratelimter service config
    31.             domain: ratelimit
    32.             failure_mode_deny: true
    33.             timeout: 10s
    34.             rate_limit_service:
    35.               grpc_service:
    36.                 envoy_grpc:
    37.                   cluster_name: outbound|8081||ratelimit.default.svc.cluster.local
    38.                   authority: ratelimit.default.svc.cluster.local
    39.               transport_api_version: V3
    40. EOF

  4. Apply another EnvoyFilter to the ingressgateway that defines the route configuration on which to rate limit. This adds rate limit actions for any route from a virtual host named bookinfo.com:80.

    1. $ kubectl apply -f - <<EOF
    2. apiVersion: networking.istio.io/v1alpha3
    3. kind: EnvoyFilter
    4. metadata:
    5.   name: filter-ratelimit-svc
    6.   namespace: istio-system
    7. spec:
    8.   workloadSelector:
    9.     labels:
    10.       istio: ingressgateway
    11.   configPatches:
    12.     - applyTo: VIRTUAL_HOST
    13.       match:
    14.         context: GATEWAY
    15.         routeConfiguration:
    16.           vhost:
    17.             name: ""
    18.             route:
    19.               action: ANY
    20.       patch:
    21.         operation: MERGE
    22.         # Applies the rate limit rules.
    23.         value:
    24.           rate_limits:
    25.             - actions: # any actions in here
    26.               - request_headers:
    27.                   header_name: ":path"
    28.                   descriptor_key: "PATH"
    29. EOF

Global rate limit advanced case

This example uses regex to match /api/* uri and defines a rate limit action inserted at the route level using the VirtualService http name. The PATH value api inserted in the prior example comes into play.

  1. Change VirtualService so the prefix /api/v1/products is moved to a route called api:

    1. $ kubectl apply -f - <<EOF
    2. apiVersion: networking.istio.io/v1
    3. kind: VirtualService
    4. metadata:
    5.   name: bookinfo
    6. spec:
    7.   gateways:
    8.   - bookinfo-gateway
    9.   hosts:
    10.   - '*'
    11.   http:
    12.   - match:
    13.     - uri:
    14.         exact: /productpage
    15.     - uri:
    16.         prefix: /static
    17.     - uri:
    18.         exact: /login
    19.     - uri:
    20.         exact: /logout
    21.     route:
    22.     - destination:
    23.         host: productpage
    24.         port:
    25.           number: 9080
    26.   - match:
    27.     - uri:
    28.         prefix: /api/v1/products
    29.     route:
    30.     - destination:
    31.         host: productpage
    32.         port:
    33.           number: 9080
    34.     name: api
    35. EOF

  2. Apply an EnvoyFilter to add the rate limits action at the route level on any 1 to 99 product:

    1. $ kubectl apply -f - <<EOF
    2. apiVersion: networking.istio.io/v1alpha3
    3. kind: EnvoyFilter
    4. metadata:
    5.   name: filter-ratelimit-svc-api
    6.   namespace: istio-system
    7. spec:
    8.   workloadSelector:
    9.     labels:
    10.       istio: ingressgateway
    11.   configPatches:
    12.     - applyTo: HTTP_ROUTE
    13.       match:
    14.         context: GATEWAY
    15.         routeConfiguration:
    16.           vhost:
    17.             name: "*:8080"
    18.             route:
    19.               name: "api"
    20.       patch:
    21.         operation: MERGE
    22.         value:
    23.           route:
    24.             rate_limits:
    25.             - actions:
    26.               - header_value_match:
    27.                   descriptor_key: "PATH"
    28.                   descriptor_value: "api"
    29.                   headers:
    30.                     - name: ":path"
    31.                       safe_regex_match:
    32.                         google_re2: {}
    33.                         regex: "/api/v1/products/[1-9]{1,2}"
    34. EOF

Local rate limit

Envoy supports local rate limiting of L4 connections and HTTP requests. This allows you to apply rate limits at the instance level, in the proxy itself, without calling any other service.

The following EnvoyFilter enables local rate limiting for any traffic through the productpage service. The HTTP_FILTER patch inserts the envoy.filters.http.local_ratelimit local envoy filter into the HTTP connection manager filter chain. The local rate limit filter’s token bucket is configured to allow 4 requests/min. The filter is also configured to add an x-local-rate-limit response header to requests that are blocked.

The statistics mentioned on the Envoy rating limiting page are disabled by default. You can enable them with the following annotations during deployment:

  1. template:
  2.   metadata:
  3.     annotations:
  4.       proxy.istio.io/config: |-
  5.         proxyStatsMatcher:
  6.           inclusionRegexps:
  7.           - ".*http_local_rate_limit.*"
  1. $ kubectl apply -f - <<EOF
  2. apiVersion: networking.istio.io/v1alpha3
  3. kind: EnvoyFilter
  4. metadata:
  5.   name: filter-local-ratelimit-svc
  6.   namespace: istio-system
  7. spec:
  8.   workloadSelector:
  9.     labels:
  10.       app: productpage
  11.   configPatches:
  12.     - applyTo: HTTP_FILTER
  13.       match:
  14.         context: SIDECAR_INBOUND
  15.         listener:
  16.           filterChain:
  17.             filter:
  18.               name: "envoy.filters.network.http_connection_manager"
  19.       patch:
  20.         operation: INSERT_BEFORE
  21.         value:
  22.           name: envoy.filters.http.local_ratelimit
  23.           typed_config:
  24.             "@type": type.googleapis.com/udpa.type.v1.TypedStruct
  25.             type_url: type.googleapis.com/envoy.extensions.filters.http.local_ratelimit.v3.LocalRateLimit
  26.             value:
  27.               stat_prefix: http_local_rate_limiter
  28.               token_bucket:
  29.                 max_tokens: 4
  30.                 tokens_per_fill: 4
  31.                 fill_interval: 60s
  32.               filter_enabled:
  33.                 runtime_key: local_rate_limit_enabled
  34.                 default_value:
  35.                   numerator: 100
  36.                   denominator: HUNDRED
  37.               filter_enforced:
  38.                 runtime_key: local_rate_limit_enforced
  39.                 default_value:
  40.                   numerator: 100
  41.                   denominator: HUNDRED
  42.               response_headers_to_add:
  43.                 - append: false
  44.                   header:
  45.                     key: x-local-rate-limit
  46.                     value: 'true'
  47. EOF

The above configuration applies local rate limiting to all vhosts/routes. Alternatively, you can restrict it to a specific route.

The following EnvoyFilter enables local rate limiting for any traffic to port 9080 of the productpage service. Unlike the previous configuration, there is no token_bucket included in the HTTP_FILTER patch. The token_bucket is instead defined in the second (HTTP_ROUTE) patch which includes a typed_per_filter_config for the envoy.filters.http.local_ratelimit local envoy filter, for routes to virtual host inbound|http|9080.

  1. $ kubectl apply -f - <<EOF
  2. apiVersion: networking.istio.io/v1alpha3
  3. kind: EnvoyFilter
  4. metadata:
  5.   name: filter-local-ratelimit-svc
  6.   namespace: istio-system
  7. spec:
  8.   workloadSelector:
  9.     labels:
  10.       app: productpage
  11.   configPatches:
  12.     - applyTo: HTTP_FILTER
  13.       match:
  14.         context: SIDECAR_INBOUND
  15.         listener:
  16.           filterChain:
  17.             filter:
  18.               name: "envoy.filters.network.http_connection_manager"
  19.       patch:
  20.         operation: INSERT_BEFORE
  21.         value:
  22.           name: envoy.filters.http.local_ratelimit
  23.           typed_config:
  24.             "@type": type.googleapis.com/udpa.type.v1.TypedStruct
  25.             type_url: type.googleapis.com/envoy.extensions.filters.http.local_ratelimit.v3.LocalRateLimit
  26.             value:
  27.               stat_prefix: http_local_rate_limiter
  28.     - applyTo: HTTP_ROUTE
  29.       match:
  30.         context: SIDECAR_INBOUND
  31.         routeConfiguration:
  32.           vhost:
  33.             name: "inbound|http|9080"
  34.             route:
  35.               action: ANY
  36.       patch:
  37.         operation: MERGE
  38.         value:
  39.           typed_per_filter_config:
  40.             envoy.filters.http.local_ratelimit:
  41.               "@type": type.googleapis.com/udpa.type.v1.TypedStruct
  42.               type_url: type.googleapis.com/envoy.extensions.filters.http.local_ratelimit.v3.LocalRateLimit
  43.               value:
  44.                 stat_prefix: http_local_rate_limiter
  45.                 token_bucket:
  46.                   max_tokens: 4
  47.                   tokens_per_fill: 4
  48.                   fill_interval: 60s
  49.                 filter_enabled:
  50.                   runtime_key: local_rate_limit_enabled
  51.                   default_value:
  52.                     numerator: 100
  53.                     denominator: HUNDRED
  54.                 filter_enforced:
  55.                   runtime_key: local_rate_limit_enforced
  56.                   default_value:
  57.                     numerator: 100
  58.                     denominator: HUNDRED
  59.                 response_headers_to_add:
  60.                   - append: false
  61.                     header:
  62.                       key: x-local-rate-limit
  63.                       value: 'true'
  64. EOF

Verify the results

Verify global rate limit

Send traffic to the Bookinfo sample. Visit http://$GATEWAY_URL/productpage in your web browser or issue the following command:

  1. $ for i in {1..2}; do curl -s "http://$GATEWAY_URL/productpage" -o /dev/null -w "%{http_code}\n"; sleep 3; done
  2. 200
  3. 429
  1. $ for i in {1..3}; do curl -s "http://$GATEWAY_URL/api/v1/products/${i}" -o /dev/null -w "%{http_code}\n"; sleep 3; done
  2. 200
  3. 200
  4. 429

$GATEWAY_URL is the value set in the Bookinfo example.

For /productpage, you will see the first request go through but every following request within a minute will get a 429 response. And for /api/v1/products/* you will need to hit twice, with any number in between 1-99, until you get the 429 response within a minute.

Verify local rate limit

Although the global rate limit at the ingress gateway limits requests to the productpage service at 1 req/min, the local rate limit for productpage instances allows 4 req/min. To confirm this, send internal productpage requests, from the ratings pod, using the following curl command:

  1. $ kubectl exec "$(kubectl get pod -l app=ratings -o jsonpath='{.items[0].metadata.name}')" -c ratings -- bash -c 'for i in {1..5}; do curl -s productpage:9080/productpage -o /dev/null -w "%{http_code}\n"; sleep 1; done'
  3. 200
  4. 200
  5. 200
  6. 200
  7. 429

You should see no more than 4 req/min go through per productpage instance.

Cleanup

Zip

  1. $ kubectl delete envoyfilter filter-ratelimit -nistio-system
  2. $ kubectl delete envoyfilter filter-ratelimit-svc -nistio-system
  3. $ kubectl delete envoyfilter filter-ratelimit-svc-api -nistio-system
  4. $ kubectl delete envoyfilter filter-local-ratelimit-svc -nistio-system
  5. $ kubectl delete cm ratelimit-config
  6. $ kubectl delete -f @samples/ratelimit/rate-limit-service.yaml@