Transformations

Connectors can be configured with transformations to make lightweight per message modifications. Debezium provides several single message transformations (SMTs) that you can use to either modify records before they are sent to Apache Kafka (by applying them to the Debezium connectors), or when they are read from Kafka by a sink connector. Also Debezium Server supports the usage of SMTs.

The following SMTs are provided by Debezium:

TransformDescription

Re-routes records to different topics based on a regular expression applied to the original topic name.

Reroute selected events to other topics, based on the event content.

Extracts the flat structure of field names and values from Debezium change events, facilitating sink connectors which cannot process Debezium’s complex event structure.

The MongoDB-specific counter-part to the New Record State Extraction SMT.

Provides a way to safely and reliably exchange data between multiple (micro) services.

The MongoDB-specific counter-part to the Outbox Event Router SMT.

Applies a filter to the change events emitted by the connectors, based on their content. This lets you propagate only those records that are relevant to you.

Moves or copies headers into the record value.

Re-routes records to specific partitions based on configured payload fields.

Converts Debezium and Kafka Connect timestamp fields in event records to a specified timezone.

Routes and enriches messages that the Debezium PostgreSQL connector captures from a TimescaleDB.

Converts values of records deserialized by Debezium CloudEvents converter to a structure suitable for Debezium JDBC sink connector.

A transformation that reduces the size of VGTIDs that the Vitess connector emits. To reduce the amount of data in the event message, the SMT writes only the VGTID for the shard in which a change occurs. The VGTIDs for other shards are removed. This transformation is designed for use only with the Debezium connector for Vitess.

The Vitess-specific transformation to remove fields to reduce the size of the messages written (e.g., redundant VGTID fields).

A transformation that filters out all messages that the connector emits to the transaction topic (when transaction metadata is enabled). This transformation is designed for use only with the Debezium connector for Vitess.

By means of SMT Predicates you can apply any of the transformations selectively, so that it modifies only that subset of change event messages that share a common characteristic.

Most of the above SMTs are available by default with the Debezium container image, but you need to opt-in for the scripting-based ones (Message Filtering or Content-based Routing). See the README file of the container image for more details.