TiCDC OpenAPI v2

TiCDC provides the OpenAPI feature for querying and operating the TiCDC cluster. The OpenAPI feature is a subset of the cdc cli tool.

TiCDC Open API v2 - 图1

Note

TiCDC OpenAPI v1 will be removed in the future. It is recommended to use TiCDC OpenAPI v2.

You can use the APIs to perform the following maintenance operations on the TiCDC cluster:

The request body and returned values of all APIs are in JSON format. A successful request returns a 200 OK message. The following sections describe the specific usage of the APIs.

In the following examples, the listening IP address of the TiCDC server is 127.0.0.1 and the port is 8300. You can specify the IP address and port bound to TiCDC via --addr=ip:port when starting the TiCDC server.

API error message template

After an API request is sent, if an error occurs, the returned error message is in the following format:

  1. {
  2.     "error_msg": "",
  3.     "error_code": ""
  4. }

In the above JSON output, error_msg describes the error message and error_code is the corresponding error code.

Return format of the API List interface

If an API request returns a list of resources (for example, a list of all Captures), the TiCDC return format is as follows:

  1. {
  2.   "total": 2,
  3.   "items": [
  4.     {
  5.       "id": "d2912e63-3349-447c-90ba-wwww",
  6.       "is_owner": true,
  7.       "address": "127.0.0.1:8300"
  8.     },
  9.     {
  10.       "id": "d2912e63-3349-447c-90ba-xxxx",
  11.       "is_owner": false,
  12.       "address": "127.0.0.1:8302"
  13.     }
  14.   ]
  15. }

In the above example:

  • total: indicates the total number of resources.
  • items: an array that contains all the resources returned by this request. All elements of the array are of the same resource.

Get the status information of a TiCDC node

This API is a synchronous interface. If the request is successful, the status information of the corresponding node is returned.

Request URI

GET /api/v2/status

Example

The following request gets the status information of the TiCDC node whose IP address is 127.0.0.1 and port number is 8300.

  1. curl -X GET http://127.0.0.1:8300/api/v2/status
  1. {
  2.   "version": "v7.1.5",
  3.   "git_hash": "10413bded1bdb2850aa6d7b94eb375102e9c44dc",
  4.   "id": "d2912e63-3349-447c-90ba-72a4e04b5e9e",
  5.   "pid": 1447,
  6.   "is_owner": true,
  7.   "liveness": 0
  8. }

The parameters of the above output are described as follows:

  • version: the current version number of TiCDC.
  • git_hash: the Git hash value.
  • id: the capture ID of the node.
  • pid: the capture process ID (PID) of the node.
  • is_owner: indicates whether the node is an owner.
  • liveness: whether this node is live. 0 means normal. 1 means that the node is in the graceful shutdown state.

Check the health status of a TiCDC cluster

This API is a synchronous interface. If the cluster is healthy, 200 OK is returned.

Request URI

GET /api/v2/health

Example

  1. curl -X GET http://127.0.0.1:8300/api/v2/health

If the cluster is healthy, the response is 200 OK and an empty JSON object:

  1. {}

If the cluster is not healthy, the response is a JSON object containing the error message.

Create a replication task

This interface is used to submit a replication task to TiCDC. If the request is successful, 200 OK is returned. The returned result only means that the server agrees to run the command but does not guarantee that the command will be run successfully.

Request URI

POST /api/v2/changefeeds

Parameter descriptions

  1. {
  2.   "changefeed_id": "string",
  3.   "replica_config": {
  4.     "bdr_mode": true,
  5.     "case_sensitive": false,
  6.     "check_gc_safe_point": true,
  7.     "consistent": {
  8.       "flush_interval": 0,
  9.       "level": "string",
  10.       "max_log_size": 0,
  11.       "storage": "string"
  12.     },
  13.     "enable_old_value": true,
  14.     "enable_sync_point": true,
  15.     "filter": {
  16.       "do_dbs": [
  17.         "string"
  18.       ],
  19.       "do_tables": [
  20.         {
  21.           "database_name": "string",
  22.           "table_name": "string"
  23.         }
  24.       ],
  25.       "event_filters": [
  26.         {
  27.           "ignore_delete_value_expr": "string",
  28.           "ignore_event": [
  29.             "string"
  30.           ],
  31.           "ignore_insert_value_expr": "string",
  32.           "ignore_sql": [
  33.             "string"
  34.           ],
  35.           "ignore_update_new_value_expr": "string",
  36.           "ignore_update_old_value_expr": "string",
  37.           "matcher": [
  38.             "string"
  39.           ]
  40.         }
  41.       ],
  42.       "ignore_dbs": [
  43.         "string"
  44.       ],
  45.       "ignore_tables": [
  46.         {
  47.           "database_name": "string",
  48.           "table_name": "string"
  49.         }
  50.       ],
  51.       "ignore_txn_start_ts": [
  52.         0
  53.       ],
  54.       "rules": [
  55.         "string"
  56.       ]
  57.     },
  58.     "force_replicate": true,
  59.     "ignore_ineligible_table": true,
  60.     "memory_quota": 0,
  61.     "mounter": {
  62.       "worker_num": 0
  63.     },
  64.     "sink": {
  65.       "column_selectors": [
  66.         {
  67.           "columns": [
  68.             "string"
  69.           ],
  70.           "matcher": [
  71.             "string"
  72.           ]
  73.         }
  74.       ],
  75.       "csv": {
  76.         "delimiter": "string",
  77.         "include_commit_ts": true,
  78.         "null": "string",
  79.         "quote": "string"
  80.       },
  81.       "date_separator": "string",
  82.       "dispatchers": [
  83.         {
  84.           "matcher": [
  85.             "string"
  86.           ],
  87.           "partition": "string",
  88.           "topic": "string"
  89.         }
  90.       ],
  91.       "enable_partition_separator": true,
  92.       "encoder_concurrency": 0,
  93.       "protocol": "string",
  94.       "schema_registry": "string",
  95.       "terminator": "string",
  96.       "transaction_atomicity": "string"
  97.     },
  98.     "sync_point_interval": "string",
  99.     "sync_point_retention": "string"
  100.   },
  101.   "sink_uri": "string",
  102.   "start_ts": 0,
  103.   "target_ts": 0
  104. }

The parameters are described as follows:

Parameter nameDescription
changefeed_idSTRING type. The ID of the replication task. (Optional)
replica_configConfiguration parameters for the replication task. (Optional)
sink_uriSTRING type. The downstream address of the replication task. (Required)
start_tsUINT64 type. Specifies the start TSO of the changefeed. The TiCDC cluster will start pulling data from this TSO. The default value is the current time. (Optional)
target_tsUINT64 type. Specifies the target TSO of the changefeed. The TiCDC cluster stops pulling data when reaching this TSO. The default value is empty, meaning TiCDC does not stop automatically. (Optional)

The meaning and format of changefeed_id, start_ts, target_ts, and sink_uri are the same as those described in the Use cdc cli to create a replication task document. For the detailed description of these parameters, see that document. Note that when you specify the certificate path in sink_uri, make sure you have uploaded the corresponding certificate to the corresponding TiCDC server.

The descriptions of the replica_config parameters are as follows.

Parameter nameDescription
bdr_modeBOOLEAN type. Determines whether to enable bidirectional replication. The default value is false. (Optional)
case_sensitiveBOOLEAN type. Determines whether to be case-sensitive when filtering table names. Starting from v6.5.6 and v7.1.3, the default value changes from true to false. (Optional)
check_gc_safe_pointBOOLEAN type. Determines whether to check that the start time of the replication task is earlier than the GC time. The default value is true. (Optional)
consistentThe configuration parameters of redo log. (Optional)
enable_old_valueBOOLEAN type. Determines whether to output the old value (that is, the value before the update). The default value is true. (Optional)
enable_sync_pointBOOLEAN type. Determines whether to enable sync point. (Optional)
filterThe configuration parameters of filter. (Optional)
force_replicateBOOLEAN type. The default value is false. When you set it to true, the replication task forcibly replicates the tables without unique indexes. (Optional)
ignore_ineligible_tableBOOLEAN type. The default value is false. When you set it to true, the replication task ignores the tables that cannot be replicated. (Optional)
memory_quotaUINT64 type. The memory quota for the replication task. (Optional)
mounterThe configuration parameters of mounter. (Optional)
sinkThe configuration parameters of sink. (Optional)
sync_point_intervalSTRING type. Note that the returned value is a time in nanosecond of the UINT64 type. When the sync point feature is enabled, this parameter specifies the interval at which Syncpoint aligns the upstream and downstream snapshots. The default value is 10m and the minimum value is 30s. (Optional)
sync_point_retentionSTRING type. Note that the returned value is a time in nanosecond of the UINT64 type. When the sync point feature is enabled, this parameter specifies how long the data is retained by Syncpoint in the downstream table. When this duration is exceeded, the data is cleaned up. The default value is 24h. (Optional)

The consistent parameters are described as follows:

Parameter nameDescription
flush_intervalUINT64 type. The interval to flush redo log files. (Optional)
levelSTRING type. The consistency level of the replicated data. (Optional)
max_log_sizeUINT64 type. The maximum value of redo log. (Optional)
storageSTRING type. The destination address of the storage. (Optional)
use_file_backendBOOL type. Specifies whether to store the redo log in a local file. (Optional)
encoding_worker_numINT type. The number of encoding and decoding workers in the redo module. (Optional)
flush_worker_numINT type. The number of flushing workers in the redo module. (Optional)
compressionSTRING type. The behavior to compress redo log files. Available options are “” and “lz4”. The default value is “”, which means no compression. (Optional)
flush_concurrencyINT type. The concurrency for uploading a single file. The default value is 1, which means concurrency is disabled. (Optional)

The filter parameters are described as follows:

Parameter nameDescription
do_dbsSTRING ARRAY type. The databases to be replicated. (Optional)
do_tablesThe tables to be replicated. (Optional)
ignore_dbsSTRING ARRAY type. The databases to be ignored. (Optional)
ignore_tablesThe tables to be ignored. (Optional)
event_filtersThe configuration to filter events. (Optional)
ignore_txn_start_tsUINT64 ARRAY type. Specifying this will ignore transactions that specify start_ts, such as [1, 2]. (Optional)
rulesSTRING ARRAY type. The rules for table schema filtering, such as [‘foo.‘, ‘bar.‘]. For more information, see Table Filter. (Optional)

The filter.event_filters parameters are described as follows. For more information, see Changefeed Log Filters.

Parameter nameDescription
ignore_delete_value_exprSTRING ARRAY type. For example, “name = ‘john’” means to filter out DELETE DML statements containing the name = ‘john’ condition. (Optional)
ignore_eventSTRING ARRAY type. For example, [“insert”] indicates that the INSERT events are filtered out. (Optional)
ignore_insert_value_exprSTRING ARRAY type. For example, “id >= 100” means to filter out INSERT DML statements that match the id >= 100 condition. (Optional)
ignore_sqlSTRING ARRAY type. For example, [“^drop”, “add column”] means to filter out DDL statements that start with DROP or contain ADD COLUMN. (Optional)
ignore_update_new_value_exprSTRING ARRAY type. For example, “gender = ‘male’” means to filter out the UPDATE DML statements with the new value gender = ‘male’. (Optional)
ignore_update_old_value_exprSTRING ARRAY type. For example, “age < 18” means to filter out the UPDATE DML statements with the old value age < 18. (Optional)
matcherSTRING ARRAY type. It works as an allowlist. For example, [“test.worker”] means that the filter rule applies only to the worker table in the test database. (Optional)

The mounter parameter is described as follows:

Parameter nameDescription
worker_numINT type. The number of Mounter threads. Mounter is used to decode the data output from TiKV. The default value is 16. (Optional)

The sink parameters are described as follows:

Parameter nameDescription
column_selectorsThe column selector configuration. (Optional)
csvThe CSV configuration. (Optional)
date_separatorSTRING type. Indicates the date separator type of the file directory. Value options are none, year, month, and day. none is the default value and means that the date is not separated. (Optional)
dispatchersAn configuration array for event dispatching. (Optional)
encoder_concurrencyINT type. The number of encoder threads in the MQ sink. The default value is 16. (Optional)
protocolSTRING type. For MQ sinks, you can specify the protocol format of the message. The following protocols are currently supported: canal-json, open-protocol, avro, and maxwell.
schema_registrySTRING type. The schema registry address. (Optional)
terminatorSTRING type. The terminator is used to separate two data change events. The default value is null, which means “\r\n” is used as the terminator. (Optional)
transaction_atomicitySTRING type. The atomicity level of the transaction. (Optional)
only_output_updated_columnsBOOLEAN type. For MQ sinks using the canal-json or open-protocol protocol, you can specify whether only output the modified columns. The default value is false. (Optional)
cloud_storage_configThe storage sink configuration. (Optional)

sink.column_selectors is an array. The parameters are described as follows:

Parameter nameDescription
columnsSTRING ARRAY type. The column array.
matcherSTRING ARRAY type. The matcher configuration. It has the same matching syntax as the filter rule does.

The sink.csv parameters are described as follows:

Parameter nameDescription
delimiterSTRING type. The character used to separate fields in the CSV file. The value must be an ASCII character and defaults to ,.
include_commit_tsBOOLEAN type. Whether to include commit-ts in CSV rows. The default value is false.
nullSTRING type. The character that is displayed when a CSV column is null. The default value is \N.
quoteSTRING type. The quotation character used to surround fields in the CSV file. If the value is empty, no quotation is used. The default value is “.
binary_encoding_methodSTRING type. The encoding method of binary data, which can be “base64” or “hex”. The default value is “base64”.

sink.dispatchers: for the sink of MQ type, you can use this parameter to configure the event dispatcher. The following dispatchers are supported: default, ts, index-value, and table. The dispatcher rules are as follows:

  • default: when multiple unique indexes (including the primary key) exist, events are dispatched in the table mode. When only one unique index (or the primary key) exists, events are dispatched in the rowid mode. If the Old Value feature is enabled, events are dispatched in the table mode.
  • ts: uses the commitTs of the row change to create the hash value and dispatch events.
  • index-value: uses the name and value of the selected HandleKey column to create the hash value and dispatch events.
  • table: uses the schema name of the table and the table name to create the hash value and dispatch events.

sink.dispatchers is an array. The parameters are described as follows:

Parameter nameDescription
matcherSTRING ARRAY type. It has the same matching syntax as the filter rule does.
partitionSTRING type. The target partition for dispatching events.
topicSTRING type. The target topic for dispatching events.

sink.cloud_storage_config parameters are described as follows:

Parameter nameDescription
worker_countINT type. The concurrency for saving data changes to the downstream cloud storage.
flush_intervalSTRING type. The interval for saving data changes to the downstream cloud storage.
file_sizeINT type. A data change file is saved to the cloud storage when the number of bytes in this file exceeds the value of this parameter.
file_expiration_daysINT type. The duration to retain files, which takes effect only when date-separator is configured as day.
file_cleanup_cron_specSTRING type. The running cycle of the scheduled cleanup task, compatible with the crontab configuration, with a format of <Second> <Minute> <Hour> <Day of the month> <Month> <Day of the week (Optional)>.
flush_concurrencyINT type. The concurrency for uploading a single file.

Example

The following request creates a replication task with an ID of test5 and sink_uri of blackhome://.

  1. curl -X POST -H "'Content-type':'application/json'" http://127.0.0.1:8300/api/v2/changefeeds -d '{"changefeed_id":"test5","sink_uri":"blackhole://"}'

If the request is successful, 200 OK is returned. If the request fails, an error message and error code are returned.

Response body format

  1. {
  2.   "admin_job_type": 0,
  3.   "checkpoint_time": "string",
  4.   "checkpoint_ts": 0,
  5.   "config": {
  6.     "bdr_mode": true,
  7.     "case_sensitive": false,
  8.     "check_gc_safe_point": true,
  9.     "consistent": {
  10.       "flush_interval": 0,
  11.       "level": "string",
  12.       "max_log_size": 0,
  13.       "storage": "string"
  14.     },
  15.     "enable_old_value": true,
  16.     "enable_sync_point": true,
  17.     "filter": {
  18.       "do_dbs": [
  19.         "string"
  20.       ],
  21.       "do_tables": [
  22.         {
  23.           "database_name": "string",
  24.           "table_name": "string"
  25.         }
  26.       ],
  27.       "event_filters": [
  28.         {
  29.           "ignore_delete_value_expr": "string",
  30.           "ignore_event": [
  31.             "string"
  32.           ],
  33.           "ignore_insert_value_expr": "string",
  34.           "ignore_sql": [
  35.             "string"
  36.           ],
  37.           "ignore_update_new_value_expr": "string",
  38.           "ignore_update_old_value_expr": "string",
  39.           "matcher": [
  40.             "string"
  41.           ]
  42.         }
  43.       ],
  44.       "ignore_dbs": [
  45.         "string"
  46.       ],
  47.       "ignore_tables": [
  48.         {
  49.           "database_name": "string",
  50.           "table_name": "string"
  51.         }
  52.       ],
  53.       "ignore_txn_start_ts": [
  54.         0
  55.       ],
  56.       "rules": [
  57.         "string"
  58.       ]
  59.     },
  60.     "force_replicate": true,
  61.     "ignore_ineligible_table": true,
  62.     "memory_quota": 0,
  63.     "mounter": {
  64.       "worker_num": 0
  65.     },
  66.     "sink": {
  67.       "column_selectors": [
  68.         {
  69.           "columns": [
  70.             "string"
  71.           ],
  72.           "matcher": [
  73.             "string"
  74.           ]
  75.         }
  76.       ],
  77.       "csv": {
  78.         "delimiter": "string",
  79.         "include_commit_ts": true,
  80.         "null": "string",
  81.         "quote": "string"
  82.       },
  83.       "date_separator": "string",
  84.       "dispatchers": [
  85.         {
  86.           "matcher": [
  87.             "string"
  88.           ],
  89.           "partition": "string",
  90.           "topic": "string"
  91.         }
  92.       ],
  93.       "enable_partition_separator": true,
  94.       "encoder_concurrency": 0,
  95.       "protocol": "string",
  96.       "schema_registry": "string",
  97.       "terminator": "string",
  98.       "transaction_atomicity": "string"
  99.     },
  100.     "sync_point_interval": "string",
  101.     "sync_point_retention": "string"
  102.   },
  103.   "create_time": "string",
  104.   "creator_version": "string",
  105.   "error": {
  106.     "addr": "string",
  107.     "code": "string",
  108.     "message": "string"
  109.   },
  110.   "id": "string",
  111.   "resolved_ts": 0,
  112.   "sink_uri": "string",
  113.   "start_ts": 0,
  114.   "state": "string",
  115.   "target_ts": 0,
  116.   "task_status": [
  117.     {
  118.       "capture_id": "string",
  119.       "table_ids": [
  120.         0
  121.       ]
  122.     }
  123.   ]
  124. }

The parameters are described as follows:

Parameter nameDescription
admin_job_typeINTEGER type. The admin job type.
checkpoint_timeSTRING type. The formatted time of the current checkpoint for the replication task.
checkpoint_tsSTRING type. The TSO of the current checkpoint for the replication task.
configThe replication task configuration. The structure and meaning are the same as that of the replica_config configuration in creating the replication task.
create_timeSTRING type. The time when the replication task is created.
creator_versionSTRING type. The TiCDC version when the replication task is created.
errorThe replication task error.
idSTRING type. The replication task ID.
resolved_tsUINT64 type. The replication task resolved ts.
sink_uriSTRING type. The replication task sink URI.
start_tsUINT64 type. The replication task start ts.
stateSTRING type. The replication task status. It can be normal, stopped, error, failed, or finished.
target_tsUINT64 type. The replication task target ts.
task_statusThe detailed status of dispatching the replication task.

The task_status parameters are described as follows:

Parameter nameDescription
capture_idSTRING type. The capture ID.
table_idsUINT64 ARRAY type. The ID of the table being replicated on this capture.

The error parameters are described as follows:

Parameter nameDescription
addrSTRING type. The capture address.
codeSTRING type. The error code.
messageSTRING type. The details of the error.

Remove a replication task

This API is an idempotent interface (that is, it can be applied multiple times without changing the result beyond the initial application) for removing a replication task. If the request is successful, 200 OK is returned. The returned result only means that the server agrees to run the command but does not guarantee that the command will be run successfully.

Request URI

DELETE /api/v2/changefeeds/{changefeed_id}

Parameter descriptions

Path parameters

Parameter nameDescription
changefeed_idThe ID of the replication task (changefeed) to be removed.

Example

The following request removes the replication task with the ID test1.

  1. curl -X DELETE http://127.0.0.1:8300/api/v2/changefeeds/test1

If the request is successful, 200 OK is returned. If the request fails, an error message and error code are returned.

Update the replication configuration

This API is used for updating a replication task. If the request is successful, 200 OK is returned. The returned result only means that the server agrees to run the command but does not guarantee that the command will be run successfully.

To modify the changefeed configuration, follow the steps of pause the replication task -> modify the configuration -> resume the replication task.

Request URI

PUT /api/v2/changefeeds/{changefeed_id}

Parameter descriptions

Path parameters

Parameter nameDescription
changefeed_idThe ID of the replication task (changefeed) to be updated.

Parameters for the request body

  1. {
  2.   "replica_config": {
  3.     "bdr_mode": true,
  4.     "case_sensitive": false,
  5.     "check_gc_safe_point": true,
  6.     "consistent": {
  7.       "flush_interval": 0,
  8.       "level": "string",
  9.       "max_log_size": 0,
  10.       "storage": "string"
  11.     },
  12.     "enable_old_value": true,
  13.     "enable_sync_point": true,
  14.     "filter": {
  15.       "do_dbs": [
  16.         "string"
  17.       ],
  18.       "do_tables": [
  19.         {
  20.           "database_name": "string",
  21.           "table_name": "string"
  22.         }
  23.       ],
  24.       "event_filters": [
  25.         {
  26.           "ignore_delete_value_expr": "string",
  27.           "ignore_event": [
  28.             "string"
  29.           ],
  30.           "ignore_insert_value_expr": "string",
  31.           "ignore_sql": [
  32.             "string"
  33.           ],
  34.           "ignore_update_new_value_expr": "string",
  35.           "ignore_update_old_value_expr": "string",
  36.           "matcher": [
  37.             "string"
  38.           ]
  39.         }
  40.       ],
  41.       "ignore_dbs": [
  42.         "string"
  43.       ],
  44.       "ignore_tables": [
  45.         {
  46.           "database_name": "string",
  47.           "table_name": "string"
  48.         }
  49.       ],
  50.       "ignore_txn_start_ts": [
  51.         0
  52.       ],
  53.       "rules": [
  54.         "string"
  55.       ]
  56.     },
  57.     "force_replicate": true,
  58.     "ignore_ineligible_table": true,
  59.     "memory_quota": 0,
  60.     "mounter": {
  61.       "worker_num": 0
  62.     },
  63.     "sink": {
  64.       "column_selectors": [
  65.         {
  66.           "columns": [
  67.             "string"
  68.           ],
  69.           "matcher": [
  70.             "string"
  71.           ]
  72.         }
  73.       ],
  74.       "csv": {
  75.         "delimiter": "string",
  76.         "include_commit_ts": true,
  77.         "null": "string",
  78.         "quote": "string"
  79.       },
  80.       "date_separator": "string",
  81.       "dispatchers": [
  82.         {
  83.           "matcher": [
  84.             "string"
  85.           ],
  86.           "partition": "string",
  87.           "topic": "string"
  88.         }
  89.       ],
  90.       "enable_partition_separator": true,
  91.       "encoder_concurrency": 0,
  92.       "protocol": "string",
  93.       "schema_registry": "string",
  94.       "terminator": "string",
  95.       "transaction_atomicity": "string"
  96.     },
  97.     "sync_point_interval": "string",
  98.     "sync_point_retention": "string"
  99.   },
  100.   "sink_uri": "string",
  101.   "target_ts": 0
  102. }

Currently, only the following configurations can be modified via the API.

Parameter nameDescription
target_tsUINT64 type. Specifies the target TSO of the changefeed. (Optional)
sink_uriSTRING type. The downstream address of the replication task. (Optional)
replica_configThe configuration parameters of sink. It must be complete. (Optional)

The meanings of the above parameters are the same as those in the Create a replication task section. See that section for details.

Example

The following request updates the target_ts of the replication task with the ID test1 to 32.

  1.  curl -X PUT -H "'Content-type':'application/json'" http://127.0.0.1:8300/api/v2/changefeeds/test1 -d '{"target_ts":32}'

If the request is successful, 200 OK is returned. If the request fails, an error message and error code are returned. The meanings of the JSON response body are the same as those in the Create a replication task section. See that section for details.

Query the replication task list

This API is a synchronous interface. If the request is successful, the basic information of all replication tasks (changefeed) in the TiCDC cluster is returned.

Request URI

GET /api/v2/changefeeds

Parameter descriptions

Query parameter

Parameter nameDescription
stateWhen this parameter is specified, the information of replication tasks in this specified state is returned. (Optional)

The value options for state are all, normal, stopped, error, failed, and finished.

If this parameter is not specified, the basic information of replication tasks in the normal, stopped, or failed state is returned by default.

Example

The following request queries the basic information of all replication tasks in the normal state.

  1. curl -X GET http://127.0.0.1:8300/api/v2/changefeeds?state=normal
  1. {
  2.   "total": 2,
  3.   "items": [
  4.     {
  5.       "id": "test",
  6.       "state": "normal",
  7.       "checkpoint_tso": 439749918821711874,
  8.       "checkpoint_time": "2023-02-27 23:46:52.888",
  9.       "error": null
  10.     },
  11.     {
  12.       "id": "test2",
  13.       "state": "normal",
  14.       "checkpoint_tso": 439749918821711874,
  15.       "checkpoint_time": "2023-02-27 23:46:52.888",
  16.       "error": null
  17.     }
  18.   ]
  19. }

The parameters in the returned result above are described as follows:

  • id: the ID of the replication task.
  • state: the current state of the replication task.
  • checkpoint_tso: the TSO of the current checkpoint of the replication task.
  • checkpoint_time: the formatted time of the current checkpoint of the replication task.
  • error: the error information of the replication task.

Query a specific replication task

This API is a synchronous interface. If the request is successful, the detailed information of the specified replication task (changefeed) is returned.

Request URI

GET /api/v2/changefeeds/{changefeed_id}

Parameter description

Path parameter

Parameter nameDescription
changefeed_idThe ID of the replication task (changefeed) to be queried.

Example

The following request queries the detailed information of the replication task with the ID test1.

  1. curl -X GET http://127.0.0.1:8300/api/v2/changefeeds/test1

The meanings of the JSON response body are the same as those in the Create a replication task section. See that section for details.

Query whether a specific replication task is completed

This API is a synchronous interface. If the request is successful, it returns the synchronization status of the specified replication task (changefeed), including whether the task is completed and additional details.

Request URI

GET /api/v2/changefeed/{changefeed_id}/synced

Parameter description

Path parameter

Parameter nameDescription
changefeed_idThe ID of the replication task (changefeed) to be queried.

Examples

The following request queries the synchronization status of the replication task with the ID test1.

  1. curl -X GET http://127.0.0.1:8300/api/v2/changefeed/test1/synced

Example 1: The synchronization is completed

  1. {
  2.   "synced": true,
  3.   "sink_checkpoint_ts": "2023-11-30 15:14:11.015",
  4.   "puller_resolved_ts": "2023-11-30 15:14:12.215",
  5.   "last_synced_ts": "2023-11-30 15:08:35.510",
  6.   "now_ts": "2023-11-30 15:14:11.511",
  7.   "info": "Data syncing is finished"
  8. }

The response includes the following fields:

  • synced: whether this replication task is completed. true means the task is completed, and false means potential incompleteness. If it is false, you need to check both the info field and other fields for the specific status.
  • sink_checkpoint_ts: the checkpoint-ts value of the sink module, in PD time.
  • puller_resolved_ts: the resolved-ts value of the puller module, in PD time.
  • last_synced_ts: the commit-ts value of the latest data processed by TiCDC, in PD time.
  • now_ts: current PD time.
  • info: supplementary information to assist in determining the synchronization status, especially when synced is false.

Example 2: The synchronization is not completed

  1. {
  2.   "synced": false,
  3.   "sink_checkpoint_ts": "2023-11-30 15:26:31.519",
  4.   "puller_resolved_ts": "2023-11-30 15:26:23.525",
  5.   "last_synced_ts": "2023-11-30 15:24:30.115",
  6.   "now_ts": "2023-11-30 15:26:31.511",
  7.   "info": "The data syncing is not finished, please wait"
  8. }

This example shows the response of an ongoing replication task. By checking both synced and info fields, you can learn that the replication task is not completed yet and further waiting is expected.

Example 3: The synchronization status needs further check

  1. {
  2.   "synced":false,
  3.   "sink_checkpoint_ts":"2023-12-13 11:45:13.515",
  4.   "puller_resolved_ts":"2023-12-13 11:45:13.525",
  5.   "last_synced_ts":"2023-12-13 11:45:07.575",
  6.   "now_ts":"2023-12-13 11:50:24.875",
  7.   "info":"Please check whether PD is online and TiKV Regions are all available. If PD is offline or some TiKV regions are not available, it means that the data syncing process is complete. To check whether TiKV regions are all available, you can view 'TiKV-Details' > 'Resolved-Ts' > 'Max Leader Resolved TS gap' on Grafana. If the gap is large, such as a few minutes, it means that some regions in TiKV are unavailable. Otherwise, if the gap is small and PD is online, it means the data syncing is incomplete, so please wait"
  8. }

This API enables you to query the synchronization status even when the upstream cluster encounters disasters. In certain situations, you might not be able to directly determine whether the current data replication task of TiCDC is completed or not. In such cases, you can request this API, and then check both the info field in the response and the current status of the upstream cluster to determine the specific status.

In this example, sink_checkpoint_ts is behind now_ts in time, either because TiCDC is still catching up with data replication, or because the PD or TiKV has failed. If this is due to TiCDC still catching up with data replication, it means that the replication task is not completed yet. If this is due to a PD or TiKV failure, it means that the replication task is completed. Therefore, you need to check the info field to assist in determining the cluster status.

Example 4: Query error

  1. {
  2.   "error_msg": "[CDC:ErrPDEtcdAPIError]etcd api call error: context deadline exceeded",
  3.   "error_code": "CDC:ErrPDEtcdAPIError"
  4. }

In cases where PD in the upstream cluster fails for a long period of time, querying this API might return an error similar to the preceding one, which provides no information for further check. Because PD failures directly affect TiCDC data replication, when getting such errors, you can assume that TiCDC has completed the data replication as much as possible, but data loss might still occur in the downstream cluster due to PD failures.

Pause a replication task

This API pauses a replication task. If the request is successful, 200 OK is returned. The returned result only means that the server agrees to run the command but does not guarantee that the command will be run successfully.

Request URI

POST /api/v2/changefeeds/{changefeed_id}/pause

Parameter description

Path parameter

Parameter nameDescription
changefeed_idThe ID of the replication task (changefeed) to be paused.

Example

The following request pauses the replication task with the ID test1.

  1. curl -X POST http://127.0.0.1:8300/api/v2/changefeeds/test1/pause

If the request is successful, 200 OK is returned. If the request fails, an error message and error code are returned.

Resume a replication task

This API resumes a replication task. If the request is successful, 200 OK is returned. The returned result only means that the server agrees to run the command but does not guarantee that the command will be run successfully.

Request URI

POST /api/v2/changefeeds/{changefeed_id}/resume

Parameter description

Path parameter

Parameter nameDescription
changefeed_idThe ID of the replication task (changefeed) to be resumed.

Parameters for the request body

  1. {
  2.   "overwrite_checkpoint_ts": 0
  3. }
Parameter nameDescription
overwrite_checkpoint_tsUINT64 type. Reassign a checkpoint TSO when resuming a replication task (changefeed).

Example

The following request resumes the replication task with the ID test1.

  1. curl -X POST http://127.0.0.1:8300/api/v2/changefeeds/test1/resume -d '{}'

If the request is successful, 200 OK is returned. If the request fails, an error message and error code are returned.

Query the replication subtask list

This API is a synchronous interface. If the request is successful, the basic information of all replication subtasks (processor) is returned.

Request URI

GET /api/v2/processors

Example

  1. curl -X GET http://127.0.0.1:8300/api/v2/processors
  1. {
  2.   "total": 3,
  3.   "items": [
  4.     {
  5.       "changefeed_id": "test2",
  6.       "capture_id": "d2912e63-3349-447c-90ba-72a4e04b5e9e"
  7.     },
  8.     {
  9.       "changefeed_id": "test1",
  10.       "capture_id": "d2912e63-3349-447c-90ba-72a4e04b5e9e"
  11.     },
  12.     {
  13.       "changefeed_id": "test",
  14.       "capture_id": "d2912e63-3349-447c-90ba-72a4e04b5e9e"
  15.     }
  16.   ]
  17. }

The parameters are described as follows:

  • changefeed_id: the changefeed ID.
  • capture_id: the capture ID.

Query a specific replication subtask

This API is a synchronous interface. If the request is successful, the detailed information of the specified replication subtask (processor) is returned.

Request URI

GET /api/v2/processors/{changefeed_id}/{capture_id}

Parameter descriptions

Path parameters

Parameter nameDescription
changefeed_idThe changefeed ID of the replication subtask to be queried.
capture_idThe capture ID of the replication subtask to be queried.

Example

The following request queries the detailed information of a subtask whose changefeed_id is test and capture_id is 561c3784-77f0-4863-ad52-65a3436db6af. A subtask can be identified by changefeed_id and capture_id.

  1. curl -X GET http://127.0.0.1:8300/api/v2/processors/test/561c3784-77f0-4863-ad52-65a3436db6af
  1. {
  2.   "table_ids": [
  3.     80
  4.   ]
  5. }

The parameter is described as follows:

  • table_ids: The table ID to be replicated on this capture.

Query the TiCDC service process list

This API is a synchronous interface. If the request is successful, the basic information of all replication processes (capture) is returned.

Request URI

GET /api/v2/captures

Example

  1. curl -X GET http://127.0.0.1:8300/api/v2/captures
  1. {
  2.   "total": 1,
  3.   "items": [
  4.     {
  5.       "id": "d2912e63-3349-447c-90ba-72a4e04b5e9e",
  6.       "is_owner": true,
  7.       "address": "127.0.0.1:8300"
  8.     }
  9.   ]
  10. }

The parameters are described as follows:

  • id: the capture ID.
  • is_owner: whether the capture is the owner.
  • address: the address of the capture.

Evict an owner node

This API is an asynchronous interface. If the request is successful, 200 OK is returned. The returned result only means that the server agrees to run the command but does not guarantee that the command will be run successfully.

Request URI

POST /api/v2/owner/resign

Example

The following request evicts the current owner node of TiCDC and triggers a new round of elections to generate a new owner node.

  1. curl -X POST http://127.0.0.1:8300/api/v2/owner/resign

If the request is successful, 200 OK is returned. If the request fails, an error message and error code are returned.

Dynamically adjust the log level of the TiCDC server

This API is a synchronous interface. If the request is successful, 200 OK is returned.

Request URI

POST /api/v2/log

Request parameter

Parameter for the request body

Parameter nameDescription
log_levelThe log level you want to set.

log_level supports the log levels provided by zap: “debug”, “info”, “warn”, “error”, “dpanic” , “panic”, and “fatal”.

Example

  1. curl -X POST -H "'Content-type':'application/json'" http://127.0.0.1:8300/api/v2/log -d '{"log_level":"debug"}'

If the request is successful, 200 OK is returned. If the request fails, an error message and error code are returned.