traffic-split

Description

The traffic-split Plugin can be used to dynamically direct portions of traffic to various Upstream services.

This is done by configuring match, which are custom rules for splitting traffic, and weighted_upstreams which is a set of Upstreams to direct traffic to.

When a request is matched based on the match attribute configuration, it will be directed to the Upstreams based on their configured weights. You can also omit using the match attribute and direct all traffic based on weighted_upstreams.

traffic-split - 图1note

The traffic ratio between Upstream services may be less accurate since round robin algorithm is used to direct traffic (especially when the state is reset).

Attributes

NameTypeRequiredDefaultValid valuesDescription
rules.matcharray[object]FalseRules to match for conditional traffic split. By default the list is empty and the traffic will be split unconditionally.
rules.match.varsarray[array]FalseList of variables to match for filtering requests for conditional traffic split. It is in the format {variable operator value}. For example, {“arg_name”, “==”, “json”}. The variables here are consistent with NGINX internal variables. For details on supported operators, lua-resty-expr.
rules.weighted_upstreamsarray[object]FalseList of Upstream configurations.
weighted_upstreams.upstream_idstring/integerFalseID of the configured Upstream object.
weighted_upstreams.upstreamobjectFalseConfiguration of the Upstream.
upstream.typeenumFalseroundrobin[roundrobin, chash]Type of mechanism to use for traffic splitting. roundobin supports weighted load and chash does consistent hashing.
upstream.hash_onenumFalsevarsOnly valid if the type is chash. Supported vars (Nginx variables), header (custom header), cookie, consumer, and vars_combinations. For more details, refer Upstream.
upstream.keystringFalseOnly valid if the type is chash. Finds the corresponding node id according to hash_on and key values. For more details, refer Upstream.
upstream.nodesobjectFalseIP addresses (with optional ports) of the Upstream nodes represented as a hash table. In the hash table, the key is the IP address and the value is the weight of the node. Setting weight to 0 means that a request is never forwarded to that node.
upstream.timeoutobjectFalse15Timeout in seconds for connecting, sending and receiving messages.
upstream.pass_hostenumFalse“pass”[“pass”, “node”, “rewrite”]Configures the host when the request is forwarded to the upstream. Can be one of pass, node or rewrite. pass- transparently passes the client’s host to the Upstream. node- uses the host configured in the node of the Upstream. rewrite- Uses the value configured in upstream_host.
upstream.namestringFalseIdentifier for the Upstream for specifying service name, usage scenarios etc.
upstream.upstream_hoststringFalseHost of the Upstream request. Only valid when pass_host attribute is set to rewrite.
weighted_upstreams.weightintegerFalseweight = 1Weight to give to each Upstream node for splitting traffic.
traffic-split - 图2note

Some of the configuration fields supported in Upstream are not supported in weighted_upstreams.upstream. These fields are service_name, discovery_type, checks, retries, retry_timeout, desc, scheme, labels, create_time, and update_time.

As a workaround, you can create an Upstream object and configure it in weighted_upstreams.upstream_id to achieve these functionalities.

traffic-split - 图3IMPORTANT

In the match attribute configuration, the expression in variable is related as AND whereas multiple variables are related by OR.

If only the weight attribute is configured, it corresponds to the weight of the Upstream service configured on the Route or Service. You can see this in action below.

Enable Plugin

You can configure the Plugin on a Route as shown below:

traffic-split - 图4note

You can fetch the admin_key from config.yaml and save to an environment variable with the following command:

  1. admin_key=$(yq '.deployment.admin.admin_key[0].key' conf/config.yaml | sed 's/"//g')
  1. curl http://127.0.0.1:9180/apisix/admin/routes/1 \
  2. -H "X-API-KEY: $admin_key" -X PUT -d '
  3. {
  4.     "uri": "/index.html",
  5.     "plugins": {
  6.         "traffic-split": {
  7.             "rules": [
  8.                 {
  9.                     "weighted_upstreams": [
  10.                         {
  11.                             "upstream": {
  12.                                 "name": "upstream_A",
  13.                                 "type": "roundrobin",
  14.                                 "nodes": {
  15.                                     "127.0.0.1:1981":10
  16.                                 },
  17.                                 "timeout": {
  18.                                     "connect": 15,
  19.                                     "send": 15,
  20.                                     "read": 15
  21.                                 }
  22.                             },
  23.                             "weight": 1
  24.                         },
  25.                         {
  26.                             "weight": 1
  27.                         }
  28.                     ]
  29.                 }
  30.             ]
  31.         }
  32.     },
  33.     "upstream": {
  34.             "type": "roundrobin",
  35.             "nodes": {
  36.                 "127.0.0.1:1980": 1
  37.             }
  38.     }
  39. }'

Alternatively, you can configure upstream_id if you have already configured an Upstream object:

  1. curl http://127.0.0.1:9180/apisix/admin/routes/1 \
  2. -H "X-API-KEY: $admin_key" -X PUT -d '
  3. {
  4.     "uri": "/index.html",
  5.     "plugins": {
  6.         "traffic-split": {
  7.             "rules": [
  8.                 {
  9.                     "weighted_upstreams": [
  10.                         {
  11.                             "upstream_id": 1,
  12.                             "weight": 1
  13.                         },
  14.                         {
  15.                             "weight": 1
  16.                         }
  17.                     ]
  18.                 }
  19.             ]
  20.         }
  21.     },
  22.     "upstream": {
  23.             "type": "roundrobin",
  24.             "nodes": {
  25.                 "127.0.0.1:1980": 1
  26.             }
  27.     }
  28. }'
traffic-split - 图5tip

Configure via upstream_id to reuse Upstream’s health detection, retires, and other functions.

traffic-split - 图6note

You can use both upstream configuration and upstream_id configuration together.

Example usage

The examples below shows different use cases for using the traffic-split Plugin.

Canary release

This is the process of gradually rolling out a release by splitting an increasing percentage of traffic to the new release until all traffic is directed to the new release.

To set this up, you can configure the weight attribute of your weighted_upstreams as shown below:

  1. curl http://127.0.0.1:9180/apisix/admin/routes/1 \
  2. -H "X-API-KEY: $admin_key" -X PUT -d '
  3. {
  4.     "uri": "/index.html",
  5.     "plugins": {
  6.         "traffic-split": {
  7.             "rules": [
  8.                 {
  9.                     "weighted_upstreams": [
  10.                         {
  11.                             "upstream": {
  12.                                 "name": "upstream_A",
  13.                                 "type": "roundrobin",
  14.                                 "nodes": {
  15.                                     "127.0.0.1:1981":10
  16.                                 },
  17.                                 "timeout": {
  18.                                     "connect": 15,
  19.                                     "send": 15,
  20.                                     "read": 15
  21.                                 }
  22.                             },
  23.                             "weight": 3
  24.                         },
  25.                         {
  26.                             "weight": 2
  27.                         }
  28.                     ]
  29.                 }
  30.             ]
  31.         }
  32.     },
  33.     "upstream": {
  34.             "type": "roundrobin",
  35.             "nodes": {
  36.                 "127.0.0.1:1980": 1
  37.             }
  38.     }
  39. }'

Here, the weights are in the ratio 3:2 which means that 60% of the traffic reaches the Upstream service running on :1981 (Plugin’s Upstream) and 40% reaches the service running on :1980 which is the Route’s Upstream service.

Now to test this configuration, if you make 5 requests, 3 will hit one service and 2 will hit the other:

  1. curl http://127.0.0.1:9080/index.html -i
  1. HTTP/1.1 200 OK
  2. Content-Type: text/html; charset=utf-8
  3. ...
  4. hello 1980
  1. curl http://127.0.0.1:9080/index.html -i
  1. HTTP/1.1 200 OK
  2. Content-Type: text/html; charset=utf-8
  3. ...
  4. world 1981

Blue-green release

In this setup, user traffic is shifted from the “green” (production) environment to the “blue” (staging) environment once the new changes have been tested and accepted within the blue environment.

To set this up, you can configure match rules based on the request headers as shown below:

  1. curl http://127.0.0.1:9180/apisix/admin/routes/1 \
  2. -H "X-API-KEY: $admin_key" -X PUT -d '
  3. {
  4.     "uri": "/index.html",
  5.     "plugins": {
  6.         "traffic-split": {
  7.             "rules": [
  8.                 {
  9.                     "match": [
  10.                         {
  11.                             "vars": [
  12.                                 ["http_release","==","new_release"]
  13.                             ]
  14.                         }
  15.                     ],
  16.                     "weighted_upstreams": [
  17.                         {
  18.                             "upstream": {
  19.                                 "name": "upstream_A",
  20.                                 "type": "roundrobin",
  21.                                 "nodes": {
  22.                                     "127.0.0.1:1981":10
  23.                                 }
  24.                             }
  25.                         }
  26.                     ]
  27.                 }
  28.             ]
  29.         }
  30.     },
  31.     "upstream": {
  32.             "type": "roundrobin",
  33.             "nodes": {
  34.                 "127.0.0.1:1980": 1
  35.             }
  36.     }
  37. }'

Here, if the request comes with a release header with value new_release it is directed to the new Upstream.

Now if you send a request with new_release as the value for the release header, it will be directed to one Upstream and other requests will be directed to another Upstream.

  1. curl http://127.0.0.1:9080/index.html -H 'release: new_release' -i
  1. HTTP/1.1 200 OK
  2. Content-Type: text/html; charset=utf-8
  3. ...
  4. world 1981
  1. curl http://127.0.0.1:9080/index.html -H 'release: old_release' -i
  1. HTTP/1.1 200 OK
  2. Content-Type: text/html; charset=utf-8
  3. ...
  4. hello 1980

Custom release

You can also make custom releases by configuring rules and setting weights.

In the example below, only one vars rule is configured and the multiple expressions in the rule have an AND relationship. The weights are configured in 3:2 ratio and traffic not matching the vars will be redirected to the Upstream configured on the Route.

  1. curl http://127.0.0.1:9180/apisix/admin/routes/1 \
  2. -H "X-API-KEY: $admin_key" -X PUT -d '
  3. {
  4.     "uri": "/index.html",
  5.     "plugins": {
  6.         "traffic-split": {
  7.             "rules": [
  8.                 {
  9.                     "match": [
  10.                         {
  11.                             "vars": [
  12.                                 ["arg_name","==","jack"],
  13.                                 ["http_user-id",">","23"],
  14.                                 ["http_apisix-key","~~","[a-z]+"]
  15.                             ]
  16.                         }
  17.                     ],
  18.                     "weighted_upstreams": [
  19.                         {
  20.                             "upstream": {
  21.                                 "name": "upstream_A",
  22.                                 "type": "roundrobin",
  23.                                 "nodes": {
  24.                                     "127.0.0.1:1981":10
  25.                                 }
  26.                             },
  27.                             "weight": 3
  28.                         },
  29.                         {
  30.                             "weight": 2
  31.                         }
  32.                     ]
  33.                 }
  34.             ]
  35.         }
  36.     },
  37.     "upstream": {
  38.             "type": "roundrobin",
  39.             "nodes": {
  40.                 "127.0.0.1:1980": 1
  41.             }
  42.     }
  43. }'

After the rules are matched, 60% of the traffic hit the Upstream on port 1981 and 40% hit the one on 1980.

  1. curl 'http://127.0.0.1:9080/index.html?name=jack' \
  2. -H 'user-id:30' -H 'apisix-key: hello' -i
  1. HTTP/1.1 200 OK
  2. Content-Type: text/html; charset=utf-8
  3. ...
  4. world 1981

If the rule fails to match, then the request is directed to the service on 1980:

  1. curl 'http://127.0.0.1:9080/index.html?name=jack' \
  2. -H 'user-id:30' -H 'apisix-key: hello' -i
  1. HTTP/1.1 200 OK
  2. Content-Type: text/html; charset=utf-8
  3. ...
  4. hello 1980

In the example below, multiple vars rules are configured and they have an OR relationship.

  1. curl http://127.0.0.1:9180/apisix/admin/routes/1 \
  2. -H "X-API-KEY: $admin_key" -X PUT -d '
  3. {
  4.     "uri": "/index.html",
  5.     "plugins": {
  6.         "traffic-split": {
  7.             "rules": [
  8.                 {
  9.                     "match": [
  10.                         {
  11.                             "vars": [
  12.                                 ["arg_name","==","jack"],
  13.                                 ["http_user-id",">","23"],
  14.                                 ["http_apisix-key","~~","[a-z]+"]
  15.                             ]
  16.                         },
  17.                         {
  18.                             "vars": [
  19.                                 ["arg_name2","==","rose"],
  20.                                 ["http_user-id2","!",">","33"],
  21.                                 ["http_apisix-key2","~~","[a-z]+"]
  22.                             ]
  23.                         }
  24.                     ],
  25.                     "weighted_upstreams": [
  26.                         {
  27.                             "upstream": {
  28.                                 "name": "upstream_A",
  29.                                 "type": "roundrobin",
  30.                                 "nodes": {
  31.                                     "127.0.0.1:1981":10
  32.                                 }
  33.                             },
  34.                             "weight": 3
  35.                         },
  36.                         {
  37.                             "weight": 2
  38.                         }
  39.                     ]
  40.                 }
  41.             ]
  42.         }
  43.     },
  44.     "upstream": {
  45.             "type": "roundrobin",
  46.             "nodes": {
  47.                 "127.0.0.1:1980": 1
  48.             }
  49.     }
  50. }'

In the example below, both the vars rules are matched. After the rules are matched, 60% of the traffic is directed to the service on 1981 and 40% to the service on 1980:

  1. curl 'http://127.0.0.1:9080/index.html?name=jack&name2=rose' \
  2. -H 'user-id:30' -H 'user-id2:22' -H 'apisix-key: hello' -H 'apisix-key2: world' -i
  1. HTTP/1.1 200 OK
  2. Content-Type: text/html; charset=utf-8
  3. ...
  4. world 1981
  1. curl 'http://127.0.0.1:9080/index.html?name=jack&name2=rose' \
  2. -H 'user-id:30' -H 'user-id2:22' -H 'apisix-key: hello' -H 'apisix-key2: world' -i
  1. HTTP/1.1 200 OK
  2. Content-Type: text/html; charset=utf-8
  3. ...
  4. hello 1980

In the example below, the second vars rule fail to match. But since it is an OR relationship, the rules are matched and traffic is directed as configured:

  1. curl 'http://127.0.0.1:9080/index.html?name=jack' \
  2. -H 'user-id:30' -H 'user-id2:22' -H 'apisix-key: hello' -H 'apisix-key2: world' -i
  1. HTTP/1.1 200 OK
  2. Content-Type: text/html; charset=utf-8
  3. ...
  4. world 1981
  1. curl 'http://127.0.0.1:9080/index.html?name=jack' \
  2. -H 'user-id:30' -H 'user-id2:22' -H 'apisix-key: hello' -H 'apisix-key2: world' -i
  1. HTTP/1.1 200 OK
  2. Content-Type: text/html; charset=utf-8
  3. ...
  4. hello 1980

In the example below the required headers are missing and both the vars rules fail to match and the request is directed to the default Upstream of the Route (1980):

  1. curl 'http://127.0.0.1:9080/index.html?name=jack' -i
  1. HTTP/1.1 200 OK
  2. Content-Type: text/html; charset=utf-8
  3. ...
  4. hello 1980

Multiple rules to correspond to Upstream

You can achieve one-to-one correspondence between rules and Upstream by configuring multiple rules:

For example, when the request header x-api-id is equal to 1 it should be directed to Upstream on port 1981 and if it is equal to 2 it should be directed to Upstream on port 1982. And in other cases, it should default to the Upstream on port 1980. You can configure this as shown below:

  1. curl http://127.0.0.1:9180/apisix/admin/routes/1 \
  2. -H "X-API-KEY: $admin_key" -X PUT -d '
  3. {
  4.     "uri": "/hello",
  5.     "plugins": {
  6.         "traffic-split": {
  7.             "rules": [
  8.                 {
  9.                     "match": [
  10.                         {
  11.                             "vars": [
  12.                                 ["http_x-api-id","==","1"]
  13.                             ]
  14.                         }
  15.                     ],
  16.                     "weighted_upstreams": [
  17.                         {
  18.                             "upstream": {
  19.                                 "name": "upstream-A",
  20.                                 "type": "roundrobin",
  21.                                 "nodes": {
  22.                                     "127.0.0.1:1981":1
  23.                                 }
  24.                             },
  25.                             "weight": 3
  26.                         }
  27.                     ]
  28.                 },
  29.                 {
  30.                     "match": [
  31.                         {
  32.                             "vars": [
  33.                                 ["http_x-api-id","==","2"]
  34.                             ]
  35.                         }
  36.                     ],
  37.                     "weighted_upstreams": [
  38.                         {
  39.                             "upstream": {
  40.                                 "name": "upstream-B",
  41.                                 "type": "roundrobin",
  42.                                 "nodes": {
  43.                                     "127.0.0.1:1982":1
  44.                                 }
  45.                             },
  46.                             "weight": 3
  47.                         }
  48.                     ]
  49.                 }
  50.             ]
  51.         }
  52.     },
  53.     "upstream": {
  54.             "type": "roundrobin",
  55.             "nodes": {
  56.                 "127.0.0.1:1980": 1
  57.             }
  58.     }
  59. }'

Now, when the request header x-api-id is equal to 1, it will hit the Upstream on 1981:

  1. curl http://127.0.0.1:9080/hello -H 'x-api-id: 1'
  1. 1981

If request header x-api-id is equal to 2, it will hit the Upstream on 1982:

  1. curl http://127.0.0.1:9080/hello -H 'x-api-id: 2'
  1. 1982

If request header x-api-id is equal to 3, the rules do not match, and it will hit the Upstream on 1980:

  1. curl http://127.0.0.1:9080/hello -H 'x-api-id: 3'
  1. 1980

Delete Plugin

To remove the traffic-split Plugin, you can delete the corresponding JSON configuration from the Plugin configuration. APISIX will automatically reload and you do not have to restart for this to take effect.

  1. curl http://127.0.0.1:9180/apisix/admin/routes/1 \
  2. -H "X-API-KEY: $admin_key" -X PUT -d '
  3. {
  4.     "uri": "/index.html",
  5.     "plugins": {},
  6.     "upstream": {
  7.         "type": "roundrobin",
  8.         "nodes": {
  9.             "127.0.0.1:1980": 1
  10.         }
  11.     }
  12. }'