HeaderToValue

Example

To extract message headers in an event record into the record value, configure the HeaderToValue SMT in the Kafka Connect configuration for a connector. You can configure the transformation to either remove the original headers or to copy them. To remove header fields from the record, configure the SMT to use the move operations. To retain the header fields in the original record, configure the SMT to use the copy operation. For example, to remove the headers event_timestamp and key from an event message, add the following lines to your connector configuration:

  1. transforms=moveHeadersToValue
  2. transforms.moveHeadersToValue.type=io.debezium.transforms.HeaderToValue
  3. transforms.moveHeadersToValue.headers=event_timestamp,key
  4. transforms.moveHeadersToValue.fields=timestamp,source.id
  5. transforms.moveHeadersToValue.operation=move

The following example shows the headers and values of an event record before and after the transformation is applied.

Example 1. Effect of applying the HeaderToValue SMT

Event record before it is processed by the HeaderToValue transformation

  • Header before the SMT processes the event record

    1. {
    2.     "header_x": 0,
    3.     "event_timestamp": 1626102708861,
    4.     "key": 100,
    5. }

    Value before the SMT processes the event record

    1. {
    2.         "before": null,
    3.         "after": {
    4.             "id": 1,
    5.             "first_name": "Anne",
    6.             "last_name": "Kretchmar",
    7.             "email": "annek@noanswer.org"
    8.         },
    9.         "source": {
    10.             "version": "2.1.3.Final",
    11.             "connector": "postgresql",
    12.             "name": "PostgreSQL_server",
    13.             "ts_ms": 1559033904863,
    14.             "snapshot": true,
    15.             "db": "postgres",
    16.             "sequence": "[\"24023119\",\"24023128\"]"
    17.             "schema": "public",
    18.             "table": "customers",
    19.             "txId": 555,
    20.             "lsn": 24023128,
    21.             "xmin": null
    22.         },
    23.         "op": "c",
    24.         "ts_ms": 1559033904863
    25.     }

Event record after it is processed by the HeaderToValue transformation

  • Header after the SMT removes the specified field

    1. {
    2.     "header_x": 0
    3. }

    Value after the SMT moves header fields into the value

    1. {
    2.         "before": null,
    3.         "after": {
    4.             "id": 1,
    5.             "first_name": "Anne",
    6.             "last_name": "Kretchmar",
    7.             "email": "annek@noanswer.org"
    8.         },
    9.         "source": {
    10.             "version": "2.1.3.Final",
    11.             "connector": "postgresql",
    12.             "name": "PostgreSQL_server",
    13.             "ts_ms": 1559033904863,
    14.             "snapshot": true,
    15.             "db": "postgres",
    16.             "sequence": "[\"24023119\",\"24023128\"]"
    17.             "schema": "public",
    18.             "table": "customers",
    19.             "txId": 555,
    20.             "lsn": 24023128,
    21.             "xmin": null,
    22.             "id": 100
    23.         },
    24.         "op": "c",
    25.         "ts_ms": 1559033904863,
    26.         "event_timestamp": 1626102708861
    27.     }

Configuration options

The following table lists the configuration options that you can use with the HeaderToValue SMT.

Table 1. HeaderToValue SMT configuration options

Property

Description

Type

Default

Valid Values

Importance

A comma-separated list of header names in the record whose values are to be copied or moved to the record value.

list

No default value

non-empty list

high

A comma-separated list of field names, in the same order as the header names listed in the headers configuration property. Use dot notation to instruct the SMT to nest fields within specific nodes of the message payload. For information about how to configure the SMT to use dot notation, see the example that appears earlier in this topic.

list

No default value

non-empty list

high

Specifies one of the following options: move:: The SMT moves header fields to values in the event record, and removes the fields from the header. copy:: The SMT copies header field to values in the event record, and retains the original header fields.

string

No default value

move or copy

high