Transformation Plugins - Request Response Transformation - 《Higress v2.0 Documentation》

Function Description
Execution Attributes
Configuration Fields
Transformation Operation Types
Configuration Example

Function Description

The transformer plugin can transform request/response headers, request query parameters, and request/response body parameters. Supported transformation operation types include deletion, renaming, updating, adding, appending, mapping, and deduplication.

Execution Attributes

Plugin execution phase: authentication phase
Plugin execution priority: 410

Configuration Fields

Name	Data Type	Fill Requirement	Default Value	Description
reqRules	string	Optional, at least one of reqRules or respRules must be filled	-	Request transformer configuration, specifying the transformation operation type and rules for transforming request headers, request query parameters, and request body
respRules	string	Optional, at least one of reqRules or respRules must be filled	-	Response transformer configuration, specifying the transformation operation type and rules for transforming response headers and response body

The configuration fields for each item in reqRules and respRules are as follows:

Name	Data Type	Fill Requirement	Default Value	Description
operate	string	Required, optional values are `remove`, `rename`, `replace`, `add`, `append`, `map`, `dedupe`	-	Specifies the transformation operation type. Supported operation types include remove (remove), rename (rename), replace (replace), add (add), append (append), map (map), dedupe (dedupe). When there are multiple transformation rules of different types, they are executed in the order of the above operation types.
mapSource	string	Optional, optional values are `headers`, `querys`, `body`	-	Valid only when operate is `map`. Specifies the mapping source. If this field is not filled, the default mapping source is itself.
headers	array of object	Optional	-	Specifies transformation rules for request/response headers.
querys	array of object	Optional	-	Specifies transformation rules for request query parameters.
body	array of object	Optional	-	Specifies transformation rules for request/response body parameters. Request body transformations allow content-types of `application/json`, `application/x-www-form-urlencoded`, and `multipart/form-data` while response body transformations only allow content-type of `application/json`.

The configuration fields for each item in headers, querys, body are as follows:

Name	Data Type	Fill Requirement	Default Value	Description
key	string	Optional	-	Used when operate is `remove`, see Transformation Operation Types for details.
oldKey	string	Optional	-	Used when operate is `rename`, see Transformation Operation Types for details.
newKey	string	Optional	-	Used when operate is `rename`, see Transformation Operation Types for details.
key	string	Optional	-	Used when operate is `replace`, see Transformation Operation Types for details.
newValue	string	Optional	-	Used when operate is `replace`, see Transformation Operation Types for details.
key	string	Optional	-	Used when operate is `add`, see Transformation Operation Types for details.
value	string	Optional	-	Used when operate is `add`, see Transformation Operation Types for details.
key	string	Optional	-	Used when operate is `append`, see Transformation Operation Types for details.
appendValue	string	Optional	-	Used when operate is `append`, see Transformation Operation Types for details.
fromKey	string	Optional	-	Used when operate is `map`, see Transformation Operation Types for details.
toKey	string	Optional	-	Used when operate is `map`, see Transformation Operation Types for details.
key	string	Optional	-	Used when operate is `dedupe`, see Transformation Operation Types for details.
strategy	string	Optional	-	Used when operate is `dedupe`, see Transformation Operation Types for details.
value_type	string	Optional, optional values are `object`, `boolean`, `number`, `string`	string	When `content-type: application/json`, this field specifies the value type of request/response body parameters.
host_pattern	string	Optional	-	Specifies the request hostname matching rule. Valid when transformation operation type is `replace`, `add`, `append`.
path_pattern	string	Optional	-	Specifies the request path matching rule. Valid when transformation operation type is `replace`, `add`, `append`.

Note:

request transformer supports the following transformation objects: request headers, request query parameters, request body (application/json, application/x-www-form-urlencoded, multipart/form-data).
response transformer supports the following transformation objects: response headers, response body (application/json).
The plugin supports bidirectional conversion capability, meaning that a single plugin can perform transformations on both requests and responses.
The execution order of transformation operation types is the order written in the configuration file, e.g., remove → rename → replace → add → append → map → dedupe or dedupe → map → append → add → replace → rename → remove.
When the transformation object is headers, key is case-insensitive. When headers are operated and are rename or map, value is also case-insensitive (as this field has a key meaning). However, key and value fields in querys and body are case-sensitive.
value_type is only effective for content type application/json for request/response bodies.
host_pattern and path_pattern support RE2 syntax, valid only for replace, add, append operations. In a transformation rule, only one of the two can be optionally filled. If both are filled, then host_pattern takes effect while path_pattern becomes ineffective.

Transformation Operation Types

Operation Type	Key Field Meaning	Value Field Meaning	Description
Remove remove	Target key	Not required	If the specified `key` exists, delete it; otherwise, no operation
Rename rename	Target oldKey	New key name newKey	If the specified `oldKey:value` exists, rename its key to `newKey`, resulting in `newKey:value`; otherwise, no operation
Replace replace	Target key	New value newValue	If the specified `key:value` exists, update its value to `newValue`, resulting in `key:newValue`; otherwise, no operation
Add add	Added key	Added value	If the specified `key:value` does not exist, add it; otherwise, no operation
Append append	Target key	Appending value appendValue	If the specified `key:value` exists, append appendValue to get `key:[value…, appendValue]`; otherwise, it is equivalent to performing add operation, resulting in `key:appendValue`.
Map map	Mapping source fromKey	Mapping target toKey	If the specified `fromKey:fromValue` exists, map its value fromValue to the value of toKey, resulting in `toKey:fromValue`, while retaining `fromKey:fromValue` (note: if toKey already exists, its value will be overwritten); otherwise, no operation.
Deduplicate dedupe	Target key	Specified deduplication strategy strategy	`strategy` optional values include: `RETAIN_UNIQUE`: Retain all unique values in order, e.g., `k1:[v1,v2,v3,v3,v2,v1]`, deduplication results in `k1:[v1,v2,v3]`. `RETAIN_LAST`: Retain the last value, e.g., `k1:[v1,v2,v3]`, deduplication results in `k1:v3`. `RETAIN_FIRST` (default): Retain the first value, e.g., `k1:[v1,v2,v3]`, deduplication results in `k1:v1`. (Note: When deduplication results in only one element v1, the key-value pair becomes `k1:v1`, not `k1:[v1]`.)

Configuration Example

Implement Routing Based on Body Parameters

Configuration example:

reqRules:
- operate: map
  headers:
  - fromKey: userId
    toKey: x-user-id
  mapSource: body

This rule extracts the userId from the request body and sets it in the request header x-user-id. This allows routing based on body parameters using Higress’s ability to match on request headers.

This configuration supports both application/json and application/x-www-form-urlencoded types of request bodies.

For example: For application/json type body

curl localhost -d ‘{“userId”:12, “userName”:”johnlanni”}’ -H ‘content-type:application/json’

The value of the userId field will be extracted from the JSON and set to x-user-id. The backend service will receive a request header with: x-user-id: 12.

After the plugin adds this header, the gateway will recalculate the routes, allowing the routing configuration to match the specific target service based on this request header.

For application/x-www-form-urlencoded type body

curl localhost -d ‘userId=12&userName=johnlanni’

The value of the userId field will be extracted from the form format k1=v1&k2=v2 and set to x-user-id. The backend service will receive a request header with: x-user-id: 12.

After the plugin adds this header, the gateway will recalculate the routes, allowing the routing configuration to match the specific target service based on this request header.

JSON Path Support

You can extract fields from complex JSON according to GJSON Path syntax.

Common operations include, for the following JSON:

{
  “name”: {“first”: “Tom”, “last”: “Anderson”},
  “age”: 37,
  “children”: [“Sara”,”Alex”,”Jack”],
  “fav.movie”: “Deer Hunter”,
  “friends”: [
    {“first”: “Dale”, “last”: “Murphy”, “age”: 44, “nets”: [“ig”, “fb”, “tw”]},
    {“first”: “Roger”, “last”: “Craig”, “age”: 68, “nets”: [“fb”, “tw”]},
    {“first”: “Jane”, “last”: “Murphy”, “age”: 47, “nets”: [“ig”, “tw”]}
  ]
}

You can achieve such extractions:

name.last              “Anderson”
name.first             “Tom”
age                    37
children               [“Sara”,”Alex”,”Jack”]
children.0             “Sara”
children.1             “Alex”
friends.1              {“first”: “Roger”, “last”: “Craig”, “age”: 68}
friends.1.first        “Roger”

Now, if you want to extract the first field from the second item in friends from the above JSON formatted body and set it to the header x-first-name, while also extracting the last field to set it to the header x-last-name, you can use this plugin configuration:

reqRules:
- operate: map
  headers:
  - fromKey: friends.1.first
    toKey: x-first-name
  - fromKey: friends.1.last
    toKey: x-last-name
  mapSource: body

Request Transformer

Transforming Request Headers

reqRules:
- operate: remove
  headers:
  - key: X-remove
- operate: rename
  headers:
  - oldKey: X-not-renamed
    newKey: X-renamed
- operate: replace
  headers:
  - key: X-replace
    newValue: replaced
- operate: add
  headers:
  - key: X-add-append
    value: host-\$1
    host_pattern: ^(.).com$
- operate: append
  headers:
  - key: X-add-append
    appendValue: path-\$1
    path_pattern: ^.?\/(\w+)[\?]{0,1}.*$
- operate: map
  headers:
  - fromKey: X-add-append
    toKey: X-map
- operate: dedupe
  headers:
  - key: X-dedupe-first
    strategy: RETAIN_FIRST
  - key: X-dedupe-last
    strategy: RETAIN_LAST
  - key: X-dedupe-unique
    strategy: RETAIN_UNIQUE

Send Request

$ curl -v console.higress.io/get -H ‘host: foo.bar.com’ \
-H ‘X-remove: exist’ -H ‘X-not-renamed:test’ -H ‘X-replace:not-replaced’ \
-H ‘X-dedupe-first:1’ -H ‘X-dedupe-first:2’ -H ‘X-dedupe-first:3’ \
-H ‘X-dedupe-last:a’ -H ‘X-dedupe-last:b’ -H ‘X-dedupe-last:c’ \
-H ‘X-dedupe-unique:1’ -H ‘X-dedupe-unique:2’ -H ‘X-dedupe-unique:3’ \
-H ‘X-dedupe-unique:3’ -H ‘X-dedupe-unique:2’ -H ‘X-dedupe-unique:1’
# httpbin response result
{
  “args”: {},
  “headers”: {
    …
    “X-Add-Append”: “host-foo.bar,path-get”,
    …
    “X-Dedupe-First”: “1”,
    “X-Dedupe-Last”: “c”,
    “X-Dedupe-Unique”: “1,2,3”,
    …
    “X-Map”: “host-foo.bar,path-get”,
    “X-Renamed”: “test”,
    “X-Replace”: “replaced”
  },
  …
}

Transforming Request Query Parameters

reqRules:
- operate: remove
  querys:
  - key: k1
- operate: rename
  querys:
  - oldKey: k2
    newKey: k2-new
- operate: replace
  querys:
  - key: k2-new
    newValue: v2-new
- operate: add
  querys:
  - key: k3
    value: v31-\$1
    path_pattern: ^.?\/(\w+)[\?]{0,1}.$
- operate: append
  querys:
  - key: k3
    appendValue: v32
- operate: map
  querys:
  - fromKey: k3
    toKey: k4
- operate: dedupe
  querys:
  - key: k4
    strategy: RETAIN_FIRST

Send Request

$ curl -v “console.higress.io/get?k1=v11&k1=v12&k2=v2”
# httpbin response result
{
  “args”: {
    “k2-new”: “v2-new”,
    “k3”: [
      “v31-get”,
      “v32”
    ],
    “k4”: “v31-get”
  },
  …
  “url”: “http://foo.bar.com/get?k2-new=v2-new&k3=v31-get&k3=v32&k4=v31-get“
}

Transforming Request Body

reqRules:
- operate: remove
  body:
  - key: a1
- operate: rename
  body:
  - oldKey: a2
    newKey: a2-new
- operate: replace
  body:
  - key: a3
    newValue: t3-new
    value_type: string
- operate: add
  body:
  - key: a1-new
    value: t1-new
    value_type: string
- operate: append
  body:
  - key: a1-new
    appendValue: t1-\$1-append
    value_type: string
    host_pattern: ^(.*).com$
- operate: map
  body:
  - fromKey: a1-new
    toKey: a4
- operate: dedupe
  body:
  - key: a4
    strategy: RETAIN_FIRST

Send Requests: 1. Content-Type: application/json

$ curl -v -X POST console.higress.io/post -H ‘host: foo.bar.com’ \
-H ‘Content-Type: application/json’ -d ‘{“a1”:”t1”,”a2”:”t2”,”a3”:”t3”}’
# httpbin response result
{
  …
  “headers”: {
    …
    “Content-Type”: “application/json”,
    …
  },
  “json”: {
    “a1-new”: [
      “t1-new”,
      “t1-foo.bar-append”
    ],
    “a2-new”: “t2”,
    “a3”: “t3-new”,
    “a4”: “t1-new”
  },
  …
}

2. Content-Type: application/x-www-form-urlencoded

$ curl -v -X POST console.higress.io/post -H ‘host: foo.bar.com’ \
-d ‘a1=t1&a2=t2&a3=t3’
# httpbin response result
{
  …
  “form”: {
    “a1-new”: [
      “t1-new”,
      “t1-foo.bar-append”
    ],
    “a2-new”: “t2”,
    “a3”: “t3-new”,
    “a4”: “t1-new”
  },
  “headers”: {
    …
    “Content-Type”: “application/x-www-form-urlencoded”,
    …
  },
  …
}

3. Content-Type: multipart/form-data

$ curl -v -X POST console.higress.io/post -H ‘host: foo.bar.com’ \
-F a1=t1 -F a2=t2 -F a3=t3
# httpbin response result
{
  …
  “form”: {
    “a1-new”: [
      “t1-new”,
      “t1-foo.bar-append”
    ],
    “a2-new”: “t2”,
    “a3”: “t3-new”,
    “a4”: “t1-new”
  },
  “headers”: {
    …
    “Content-Type”: “multipart/form-data; boundary=————————————1118b3fab5afbc4e”,
    …
  },
  …
}

Response Transformer

Similar to Request Transformer, this only describes the precautions for transforming JSON-formatted request/response bodies:

Key Nesting `.`

In general, a key containing . indicates a nested meaning, as follows:

respRules:
- operate: add
  body:
  - key: foo.bar
    value: value

$ curl -v console.higress.io/get
# httpbin response result
{
 …
 “foo”: {
  “bar”: “value”
 },
 …
}

When using \. to escape . in the key, it indicates a non-nested meaning, as follows:

When enclosing a string with double quotes, use \\. for escaping

respRules:
- operate: add
  body:
  - key: foo.bar
    value: value

$ curl -v console.higress.io/get
# httpbin response result
{
 …
 “foo.bar”: “value”,
 …
}

Accessing Array Elements `.index`

You can access array elements by their index array.index, where the index starts from 0:

{
  “users”: [
    {
      “123”: { “name”: “zhangsan”, “age”: 18 }
    },
    {
      “456”: { “name”: “lisi”, “age”: 19 }
    }
  ]
}

Remove the first element of user:

reqRules:
- operate: remove
  body:
  - key: users.0

$ curl -v -X POST console.higress.io/post \
-H ‘Content-Type: application/json’ \
-d ‘{“users”:[{“123”:{“name”:”zhangsan”}},{“456”:{“name”:”lisi”}}]}’
# httpbin response result
{
  …
  “json”: {
    “users”: [
      {
        “456”: {
          “name”: “lisi”
        }
      }
    ]
  },
  …
}

Rename the key 123 of the first element of users to msg:

reqRules:
- operate: rename
  body:
  - oldKey: users.0.123
    newKey: users.0.first

$ curl -v -X POST console.higress.io/post \
-H ‘Content-Type: application/json’ \
-d ‘{“users”:[{“123”:{“name”:”zhangsan”}},{“456”:{“name”:”lisi”}}]}’
# httpbin response result
{
  …
  “json”: {
    “users”: [
      {
        “msg”: {
          “name”: “zhangsan”
        }
      },
      {
        “456”: {
          “name”: “lisi”
        }
      }
    ]
  },
  …
}

Iterating Array Elements `.#`

You can use array.# to iterate over an array:

❗️This operation can only be used in replace, do not attempt this operation in other transformations to avoid unpredictable results

{
  “users”: [
    {
      “name”: “zhangsan”,
      “age”: 18
    },
    {
      “name”: “lisi”,
      “age”: 19
    }
  ]
}

reqRules:
- operate: replace
  body:
  - key: users.#.age
    newValue: 20

$ curl -v -X POST console.higress.io/post \
-H ‘Content-Type: application/json’ \
-d ‘{“users”:[{“name”:”zhangsan”,”age”:18},{“name”:”lisi”,”age”:19}]}’
# httpbin response result
{
  …
  “json”: {
    “users”: [
      {
        “age”: “20”,
        “name”: “zhangsan”
      },
      {
        “age”: “20”,
        “name”: “lisi”
      }
    ]
  },
  …
}

Request Response Transformation