Timezone Converter

Example: Basic configuration

Configure the TimezoneConverter SMT in the Kafka Connect configuration for a connector to convert the time-based fields in an event record to a target timezone.

For example, to convert all timestamp fields in an event record from UTC to the Pacific/Easter timezone, add the following lines to your connector configuration:

  1. transforms=convertTimezone
  2. transforms.convertTimezone.type=io.debezium.transforms.TimezoneConverter
  3. transforms.convertTimezone.converted.timezone=Pacific/Easter

Effect of applying the TimezoneConverter SMT

The following examples show how the TimezoneConverter transformation modifies the timestamp fields in an event record. The first example shows a Debezium event record that is not processed by the transformation; the record retains its original timestamp values. The next example shows the same event record after the transformation is applied. Per the configuration specified in the basic configuration example, the SMT converts the original UTC values of timestamp fields in the source message to Pacific/Easter timezone values.

Example 1. Event record value before processing by the TimezoneConverter transformation

The value of the created_at field shows the UTC time.

  1. {
  2. "before": null,
  3. "after": {
  4. "id": 1,
  5. "first_name": "Anne",
  6. "last_name": "Kretchmar",
  7. "email": "annek@noanswer.org",
  8. "created_at": "2011-01-11T16:40:30.123456789+00:00"
  9. },
  10. "source": {
  11. "version": "2.5.4.Final",
  12. "connector": "postgresql",
  13. "name": "PostgreSQL_server",
  14. "ts_ms": 1559033904863,
  15. "snapshot": true,
  16. "db": "postgres",
  17. "sequence": "[\"24023119\",\"24023128\"]",
  18. "schema": "public",
  19. "table": "customers",
  20. "txId": 555,
  21. "lsn": 24023128,
  22. "xmin": null
  23. },
  24. "op": "c",
  25. "ts_ms": 1559033904863
  26. }

Example 2. Event record value after processing by the TimezoneConverter transformation

The SMT converts the original UTC value of the created_at field to the time in the target Pacific/Easter timezone that is specified in the Basic configuration example. The SMT also adds an event_timestamp field.

  1. {
  2. "before": null,
  3. "after": {
  4. "id": 1,
  5. "first_name": "Anne",
  6. "last_name": "Kretchmar",
  7. "email": "annek@noanswer.org",
  8. "created_at": "2011-01-11T11:40:30.123456789-05:00"
  9. },
  10. "source": {
  11. "version": "2.5.4.Final",
  12. "connector": "postgresql",
  13. "name": "PostgreSQL_server",
  14. "ts_ms": 1559033904863,
  15. "snapshot": true,
  16. "db": "postgres",
  17. "sequence": "[\"24023119\",\"24023128\"]",
  18. "schema": "public",
  19. "table": "customers",
  20. "txId": 555,
  21. "lsn": 24023128,
  22. "xmin": null,
  23. "id": 100
  24. },
  25. "op": "c",
  26. "ts_ms": 1559033904863,
  27. "event_timestamp": 1626102708861
  28. }

Example: Advanced configuration

Instead of converting all timestamp fields in an event record, you can configure the SMT to convert specific fields only. The following example shows how you might use the include.list option in the SMT configuration to convert only the created_at, and updated_at timestamp fields in an event record. The following configuration uses a fixed offset, rather than a geographic timezone designator, to convert the time from UTC to +05:30.

  1. transforms=convertTimezone
  2. transforms.convertTimezone.type=io.debezium.transforms.TimezoneConverter
  3. transforms.convertTimezone.converted.timezone=+05:30
  4. transforms.convertTimezone.include.list=source:customers:created_at,customers:updated_at

In some cases, you might want to exclude specific timestamp fields from timezone conversion. For example, to exclude the updated_at timestamp field in an event record from timezone conversion, use the exclude.list configuration option as in the following example:

  1. transforms=convertTimezone
  2. transforms.convertTimezone.type=io.debezium.transforms.TimezoneConverter
  3. transforms.convertTimezone.converted.timezone=+05:30
  4. transforms.convertTimezone.exclude.list=source:customers:updated_at

Configuration options

The following table lists the configuration options for the TimezoneConverter SMT.

Table 1. TimezoneConverter SMT configuration options

Property

Description

Type

Importance

A string that specifies the target timezone to which the timestamp fields should be converted. The target timezone can be specified as a geographic timezone, such as, America/New_York, or as a UTC offset, for example, +02:00.

string

high

A comma-separated list of rules that specify the fields that the SMT includes for timezone conversion. Specify rules by using one of the following formats:

    source:<tablename>

    Matches Debezium change events with source information blocks that have the specified table name. The SMT converts all time-based fields in the matched table.

    source:<tablename>:<fieldname>

    Matches Debezium change events with source information blocks that have the specified table name. The SMT converts only fields in the specified table that have the specified field name.

    topic:<topicname>

    Matches events from the specified topic name, converting all time-based fields in the event record.

    topic:<topicname>:<fieldname>

    Matches events from the specified topic name, and converts values for the specified fields only.

    <matchname>:<fieldname>

    Applies a heuristic matching algorithm to match against the table name of the source information block, if present; otherwise, matches against the topic name. The SMT converts values for the specified field name only.

list

medium

A comma-separated list of rules that specify the fields to exclude from timezone conversion. Specify rules by using one of the following formats:

    source:<tablename>

    Matches Debezium change events with source information blocks that have the specified table name. The SMT excludes all time-based fields in the matched table from conversion.

    source:<tablename>:<fieldname>

    Matches Debezium change events with source information blocks that have the specified table name. The SMT excludes from conversion fields in the specified table that match the specified field name.

    topic:<topicname>

    Matches events from the specified topic name, and excludes from conversion all time-based fields in the topic.

    topic:<topicname>:<fieldname>

    Matches events from the specified topic name, and excludes from conversion any fields in the topic that have the specified name.

    <matchname>:<fieldname>

    Applies a heuristic matching algorithm to match against the table name of the source information block, if present; otherwise, matches against the topic name. The SMT excludes from conversion only fields that have the specified name.

list

medium