Function Description

The ai-token-ratelimit plugin implements token rate limiting based on specific key values. The key values can come from URL parameters, HTTP request headers, client IP addresses, consumer names, or key names in cookies.

Notice

For this plugin to function, the AI Observability Plugin must also be enabled to achieve token count statistics.

Runtime Attributes

Plugin execution phase: default phase
Plugin execution priority: 600

Configuration Description

Configuration ItemTypeRequiredDefault ValueDescription
rule_namestringYes-Name of the rate limiting rule, used to assemble the redis key based on the rule name + rate limiting type + rate limiting key name + actual value corresponding to the rate limiting key
rule_itemsarray of objectYes-Rate limiting rule items. After matching the first rule_item, subsequent rules will be ignored based on the order in rule_items
rejected_codeintNo429The HTTP status code returned when the request is rate limited
rejected_msgstringNoToo many requestsThe response body returned when the request is rate limited
redisobjectYes-Redis related configuration

Field descriptions for each item in rule_items

Configuration ItemTypeRequiredDefault ValueDescription
limitby_headerstringNo, optionally select one in limit_by-Configure the source HTTP header name for obtaining the rate limiting key value
limitby_paramstringNo, optionally select one in limit_by-Configure the source URL parameter name for obtaining the rate limiting key value
limitby_consumerstringNo, optionally select one in limit_by-Rate limit by consumer name, no actual value needs to be added
limitby_cookiestringNo, optionally select one in limit_by-Configure the source key name in cookies for obtaining the rate limiting key value
limitby_per_headerstringNo, optionally select one in limit_by-Match specific HTTP request headers according to rules and calculate rate limiting separately for each header. Configure the source HTTP header name for obtaining the rate limiting key value. Supports regular expressions or when configuring limitkeys
limit_by_per_paramstringNo, optionally select one in limit_by-Match specific URL parameters according to rules and calculate rate limiting separately for each parameter. Configure the source URL parameter name for obtaining the rate limiting key value. Supports regular expressions or when configuring limitkeys
limit_by_per_consumerstringNo, optionally select one in limit_by-Match specific consumers according to rules and calculate rate limiting separately for each consumer. Rate limit by consumer name, no actual value needs to be added. Supports regular expressions or when configuring limitkeys
limit_by_per_cookiestringNo, optionally select one in limit_by-Match specific cookies according to rules and calculate rate limiting separately for each cookie. Configure the source key name in cookies for obtaining the rate limiting key value. Supports regular expressions or when configuring limitkeys
limit_by_per_ipstringNo, optionally select one in limit_by*-Match specific IPs according to rules and calculate rate limiting separately for each IP. Configure the source IP parameter name for obtaining the rate limiting key value from request headers, from-header-<header name>, such as from-header-x-forwarded-for. Directly get the remote socket IP by configuring from-remote-addr
limit_keysarray of objectYes-Configure the number of rate limit requests after matching keys

Field descriptions for each item in limit_keys

Configuration ItemTypeRequiredDefault ValueDescription
keystringYes-Matched key value. Types limit_by_per_header, limit_by_per_param, limit_by_per_consumer, limit_by_per_cookie support configuring regular expressions (beginning with regexp: followed by the regex) or (representing all). Example regex: regexp:^d. (all strings starting with d); limit_by_per_ip supports configuring IP addresses or IP segments
token_per_secondintNo, optionally select one in token_per_second, token_per_minute, token_per_hour, token_per_day-Allowed number of token requests per second
token_per_minuteintNo, optionally select one in token_per_second, token_per_minute, token_per_hour, token_per_day-Allowed number of token requests per minute
token_per_hourintNo, optionally select one in token_per_second, token_per_minute, token_per_hour, token_per_day-Allowed number of token requests per hour
token_per_dayintNo, optionally select one in token_per_second, token_per_minute, token_per_hour, token_per_day-Allowed number of token requests per day

Field descriptions for each item in redis

Configuration ItemTypeRequiredDefault ValueDescription
service_namestringRequired-Full FQDN name of the redis service, including service type, e.g., my-redis.dns, redis.my-ns.svc.cluster.local
service_portintNoDefault value for static addresses (static service) is 80; otherwise, it is 6379Input the service port of the redis service
usernamestringNo-Redis username
passwordstringNo-Redis password
timeoutintNo1000Redis connection timeout in milliseconds

Configuration Examples

Identify request parameter apikey for differentiated rate limiting

  1. rule_name: default_rule
  2. rule_items:
  3.   - limit_by_param: apikey
  4.     limit_keys:
  5.       - key: 9a342114-ba8a-11ec-b1bf-00163e1250b5
  6.         token_per_minute: 10
  7.       - key: a6a6d7f2-ba8a-11ec-bec2-00163e1250b5
  8.         token_per_hour: 100
  9.   - limit_by_per_param: apikey
  10.     limit_keys:
  11.       # Regular expression, matches all strings starting with a, each apikey corresponds to 10 qds
  12.       - key: “regexp:^a.“
  13.         token_per_second: 10
  14.       # Regular expression, matches all strings starting with b, each apikey corresponds to 100 qd
  15.       - key: “regexp:^b.“
  16.         token_per_minute: 100
  17.       # Fallback, matches all requests, each apikey corresponds to 1000 qdh
  18.       - key: “*”
  19.         token_per_hour: 1000
  20. redis:
  21.   service_name: redis.static

Identify request header x-ca-key for differentiated rate limiting

  1. rule_name: default_rule
  2. rule_items:
  3.   - limit_by_header: x-ca-key
  4.     limit_keys:
  5.       - key: 102234
  6.         token_per_minute: 10
  7.       - key: 308239
  8.         token_per_hour: 10
  9.   - limit_by_per_header: x-ca-key
  10.     limit_keys:
  11.       # Regular expression, matches all strings starting with a, each apikey corresponds to 10 qds
  12.       - key: “regexp:^a.“
  13.         token_per_second: 10
  14.       # Regular expression, matches all strings starting with b, each apikey corresponds to 100 qd
  15.       - key: “regexp:^b.“
  16.         token_per_minute: 100
  17.       # Fallback, matches all requests, each apikey corresponds to 1000 qdh
  18.       - key: “*”
  19.         token_per_hour: 1000
  20. redis:
  21.   service_name: redis.static

Get the peer IP using the request header x-forwarded-for for differentiated rate limiting

  1. rule_name: default_rule
  2. rule_items:
  3.   - limit_by_per_ip: from-header-x-forwarded-for
  4.     limit_keys:
  5.       # Exact IP
  6.       - key: 1.1.1.1
  7.         token_per_day: 10
  8.       # IP segment, matching IPs in this segment, each IP 100 qpd
  9.       - key: 1.1.1.0/24
  10.         token_per_day: 100
  11.       # Fallback, i.e., default each IP 1000 qpd
  12.       - key: 0.0.0.0/0
  13.         token_per_day: 1000
  14. redis:
  15.   service_name: redis.static

Identify consumer for differentiated rate limiting

  1. rule_name: default_rule
  2. rule_items:
  3.   - limit_by_consumer: ‘’
  4.     limit_keys:
  5.       - key: consumer1
  6.         token_per_second: 10
  7.       - key: consumer2
  8.         token_per_hour: 100
  9.   - limit_by_per_consumer: ‘’
  10.     limit_keys:
  11.       # Regular expression, matches all strings starting with a, each consumer corresponds to 10 qds
  12.       - key: “regexp:^a.“
  13.         token_per_second: 10
  14.       # Regular expression, matches all strings starting with b, each consumer corresponds to 100 qd
  15.       - key: “regexp:^b.“
  16.         token_per_minute: 100
  17.       # Fallback, matches all requests, each consumer corresponds to 1000 qdh
  18.       - key: “*”
  19.         token_per_hour: 1000
  20. redis:
  21.   service_name: redis.static

Identify key-value pairs in cookies for differentiated rate limiting

  1. rule_name: default_rule
  2. rule_items:
  3.   - limit_by_cookie: key1
  4.     limit_keys:
  5.       - key: value1
  6.         token_per_minute: 10
  7.       - key: value2
  8.         token_per_hour: 100
  9.   - limit_by_per_cookie: key1
  10.     limit_keys:
  11.       # Regular expression, matches all strings starting with a, each value in cookie corresponds to 10 qds
  12.       - key: “regexp:^a.“
  13.         token_per_second: 10
  14.       # Regular expression, matches all strings starting with b, each value in cookie corresponds to 100 qd
  15.       - key: “regexp:^b.“
  16.         token_per_minute: 100
  17.       # Fallback, matches all requests, each value in cookie corresponds to 1000 qdh
  18.       - key: “*”
  19.         token_per_hour: 1000
  20. rejected_code: 200
  21. rejected_msg: ‘{“code”:-1,”msg”:”Too many requests”}’
  22. redis:
  23.   service_name: redis.static