Maintain DM Clusters Using OpenAPI

DM provides the OpenAPI feature for easily querying and operating the DM cluster, which is similar to the feature of dmctl tools.

To enable OpenAPI, perform one of the following operations:

  • If your DM cluster has been deployed directly using binary, add the following configuration to the DM-master configuration file.

    1. openapi = true

  • If your DM cluster has been deployed using TiUP, add the following configuration to the topology file:

    1. server_configs:
    2.   master:
    3.     openapi: true

OpenAPI - 图1

Note

  • DM provides the specification document that meets the OpenAPI 3.0.0 standard. This document contains all the request parameters and returned values. You can copy the document yaml and preview it in Swagger Editor.

  • After you deploy the DM-master nodes, you can access http://{master-addr}/api/v1/docs to preview the documentation online.

  • Some features supported in the configuration file are not supported in OpenAPI. Their capabilities are not fully aligned. In a production environment, it is recommended to use the configuration file.

You can use the APIs to perform the following maintenance operations on the DM cluster:

APIs for managing clusters

APIs for managing data sources

APIs for managing replication tasks

The following sections describe the specific usage of the APIs.

API error message template

After sending an API request, if an error occurs, the returned error message is in the following format:

  1. {
  2.     "error_msg": "",
  3.     "error_code": ""
  4. }

From the above JSON output, error_msg describes the error message and error_code is the corresponding error code.

Get the information of a DM-master node

This API is a synchronous interface. If the request is successful, the information of the corresponding node is returned.

Request URI

GET /api/v1/cluster/masters

Example

  1. curl -X 'GET' \
  2.   'http://127.0.0.1:8261/api/v1/cluster/masters' \
  3.   -H 'accept: application/json'
  1. {
  2.   "total": 1,
  3.   "data": [
  4.     {
  5.       "name": "master1",
  6.       "alive": true,
  7.       "leader": true,
  8.       "addr": "127.0.0.1:8261"
  9.     }
  10.   ]
  11. }

Stop a DM-master node

This API is a synchronous interface. If the request is successful, the status code of the returned body is 204.

Request URI

DELETE /api/v1/cluster/masters/{master-name}

Example

  1. curl -X 'DELETE' \
  2.   'http://127.0.0.1:8261/api/v1/cluster/masters/master1' \
  3.   -H 'accept: */*'

Get the information of a DM-worker node

This API is a synchronous interface. If the request is successful, the information of the corresponding node is returned.

Request URI

GET /api/v1/cluster/workers

Example

  1. curl -X 'GET' \
  2.   'http://127.0.0.1:8261/api/v1/cluster/workers' \
  3.   -H 'accept: application/json'
  1. {
  2.   "total": 1,
  3.   "data": [
  4.     {
  5.       "name": "worker1",
  6.       "addr": "127.0.0.1:8261",
  7.       "bound_stage": "bound",
  8.       "bound_source_name": "mysql-01"
  9.     }
  10.   ]
  11. }

Stop a DM-worker node

This API is a synchronous interface. If the request is successful, the status code of the returned body is 204.

Request URI

DELETE /api/v1/cluster/workers/{worker-name}

Example

  1. curl -X 'DELETE' \
  2.   'http://127.0.0.1:8261/api/v1/cluster/workers/worker1' \
  3.   -H 'accept: */*'

Create a data source

This API is a synchronous interface. If the request is successful, the information of the corresponding data source is returned.

Request URI

POST /api/v1/sources

Example

  1. curl -X 'POST' \
  2.   'http://127.0.0.1:8261/api/v1/sources' \
  3.   -H 'accept: application/json' \
  4.   -H 'Content-Type: application/json' \
  5.   -d '{
  6.   "source_name": "mysql-01",
  7.   "host": "127.0.0.1",
  8.   "port": 3306,
  9.   "user": "root",
  10.   "password": "123456",
  11.   "enable": true,
  12.   "enable_gtid": false,
  13.   "security": {
  14.     "ssl_ca_content": "",
  15.     "ssl_cert_content": "",
  16.     "ssl_key_content": "",
  17.     "cert_allowed_cn": [
  18.       "string"
  19.     ]
  20.   },
  21.   "purge": {
  22.     "interval": 3600,
  23.     "expires": 0,
  24.     "remain_space": 15
  25.   }
  26. }'
  1. {
  2.   "source_name": "mysql-01",
  3.   "host": "127.0.0.1",
  4.   "port": 3306,
  5.   "user": "root",
  6.   "password": "123456",
  7.   "enable": true,
  8.   "enable_gtid": false,
  9.   "security": {
  10.     "ssl_ca_content": "",
  11.     "ssl_cert_content": "",
  12.     "ssl_key_content": "",
  13.     "cert_allowed_cn": [
  14.       "string"
  15.     ]
  16.   },
  17.   "purge": {
  18.     "interval": 3600,
  19.     "expires": 0,
  20.     "remain_space": 15
  21.   },
  22.   "status_list": [
  23.     {
  24.       "source_name": "mysql-replica-01",
  25.       "worker_name": "worker-1",
  26.       "relay_status": {
  27.         "master_binlog": "(mysql-bin.000001, 1979)",
  28.         "master_binlog_gtid": "e9a1fc22-ec08-11e9-b2ac-0242ac110003:1-7849",
  29.         "relay_dir": "./sub_dir",
  30.         "relay_binlog_gtid": "e9a1fc22-ec08-11e9-b2ac-0242ac110003:1-7849",
  31.         "relay_catch_up_master": true,
  32.         "stage": "Running"
  33.       },
  34.       "error_msg": "string"
  35.     }
  36.   ]
  37. }

Get a data source

This API is a synchronous interface. If the request is successful, the information of the corresponding data source is returned.

Request URI

GET /api/v1/sources/{source-name}

Example

  1. curl -X 'GET' \
  2.   'http://127.0.0.1:8261/api/v1/sources/source-1?with_status=true' \
  3.   -H 'accept: application/json'
  1. {
  2.   "source_name": "mysql-01",
  3.   "host": "127.0.0.1",
  4.   "port": 3306,
  5.   "user": "root",
  6.   "password": "123456",
  7.   "enable_gtid": false,
  8.   "enable": false,
  9.   "flavor": "mysql",
  10.   "task_name_list": [
  11.     "task1"
  12.   ],
  13.   "security": {
  14.     "ssl_ca_content": "",
  15.     "ssl_cert_content": "",
  16.     "ssl_key_content": "",
  17.     "cert_allowed_cn": [
  18.       "string"
  19.     ]
  20.   },
  21.   "purge": {
  22.     "interval": 3600,
  23.     "expires": 0,
  24.     "remain_space": 15
  25.   },
  26.   "status_list": [
  27.     {
  28.       "source_name": "mysql-replica-01",
  29.       "worker_name": "worker-1",
  30.       "relay_status": {
  31.         "master_binlog": "(mysql-bin.000001, 1979)",
  32.         "master_binlog_gtid": "e9a1fc22-ec08-11e9-b2ac-0242ac110003:1-7849",
  33.         "relay_dir": "./sub_dir",
  34.         "relay_binlog_gtid": "e9a1fc22-ec08-11e9-b2ac-0242ac110003:1-7849",
  35.         "relay_catch_up_master": true,
  36.         "stage": "Running"
  37.       },
  38.       "error_msg": "string"
  39.     }
  40.   ],
  41.   "relay_config": {
  42.     "enable_relay": true,
  43.     "relay_binlog_name": "mysql-bin.000002",
  44.     "relay_binlog_gtid": "e9a1fc22-ec08-11e9-b2ac-0242ac110003:1-7849",
  45.     "relay_dir": "./relay_log"
  46.   }
  47. }

Delete the data source

This API is a synchronous interface. If the request is successful, the status code of the returned body is 204.

Request URI

DELETE /api/v1/sources/{source-name}

Example

  1. curl -X 'DELETE' \
  2.   'http://127.0.0.1:8261/api/v1/sources/mysql-01?force=true' \
  3.   -H 'accept: application/json'

Update a data source

This API is a synchronous interface. If the request is successful, the information of the corresponding data source is returned.

OpenAPI - 图2

Note

When you use this API to update the data source configuration, make sure that there are no running tasks under the current data source.

Request URI

PUT /api/v1/sources/{source-name}

Example

  1. curl -X 'PUT' \
  2.   'http://127.0.0.1:8261/api/v1/sources/mysql-01' \
  3.   -H 'accept: application/json' \
  4.   -H 'Content-Type: application/json' \
  5.   -d '{
  6.   "source": {
  7.     "source_name": "mysql-01",
  8.     "host": "127.0.0.1",
  9.     "port": 3306,
  10.     "user": "root",
  11.     "password": "123456",
  12.     "enable_gtid": false,
  13.     "enable": false,
  14.     "flavor": "mysql",
  15.     "task_name_list": [
  16.       "task1"
  17.     ],
  18.     "security": {
  19.       "ssl_ca_content": "",
  20.       "ssl_cert_content": "",
  21.       "ssl_key_content": "",
  22.       "cert_allowed_cn": [
  23.         "string"
  24.       ]
  25.     },
  26.     "purge": {
  27.       "interval": 3600,
  28.       "expires": 0,
  29.       "remain_space": 15
  30.     },
  31.     "relay_config": {
  32.       "enable_relay": true,
  33.       "relay_binlog_name": "mysql-bin.000002",
  34.       "relay_binlog_gtid": "e9a1fc22-ec08-11e9-b2ac-0242ac110003:1-7849",
  35.       "relay_dir": "./relay_log"
  36.     }
  37.   }
  38. }'
  1. {
  2.   "source_name": "mysql-01",
  3.   "host": "127.0.0.1",
  4.   "port": 3306,
  5.   "user": "root",
  6.   "password": "123456",
  7.   "enable": true,
  8.   "enable_gtid": false,
  9.   "security": {
  10.     "ssl_ca_content": "",
  11.     "ssl_cert_content": "",
  12.     "ssl_key_content": "",
  13.     "cert_allowed_cn": [
  14.       "string"
  15.     ]
  16.   },
  17.   "purge": {
  18.     "interval": 3600,
  19.     "expires": 0,
  20.     "remain_space": 15
  21.   }
  22. }

Enable a data source

This is a synchronous interface that enables a data source on a successful request and starts all subtasks of the task that rely on this data source in batch.

Request URI

POST /api/v1/sources/{source-name}/enable

Example

  1. curl -X 'POST' \
  2.   'http://127.0.0.1:8261/api/v1/sources/mysql-01/enable' \
  3.   -H 'accept: */*' \
  4.   -H 'Content-Type: application/json'

Disable a data source

This is a synchronous interface that deactivates this data source on a successful request and stops all subtasks of the task that rely on it in batch.

Request URI

POST /api/v1/sources/{source-name}/disable

Example

  1. curl -X 'POST' \
  2.   'http://127.0.0.1:8261/api/v1/sources/mysql-01/disable' \
  3.   -H 'accept: */*' \
  4.   -H 'Content-Type: application/json'

Get the data source list

This API is a synchronous interface. If the request is successful, the data source list is returned.

Request URI

GET /api/v1/sources

Example

  1. curl -X 'GET' \
  2.   'http://127.0.0.1:8261/api/v1/sources?with_status=true' \
  3.   -H 'accept: application/json'
  1. {
  2.   "data": [
  3.     {
  4.       "enable_gtid": false,
  5.       "host": "127.0.0.1",
  6.       "password": "******",
  7.       "port": 3306,
  8.       "purge": {
  9.         "expires": 0,
  10.         "interval": 3600,
  11.         "remain_space": 15
  12.       },
  13.       "security": null,
  14.       "source_name": "mysql-01",
  15.       "user": "root"
  16.     },
  17.     {
  18.       "enable_gtid": false,
  19.       "host": "127.0.0.1",
  20.       "password": "******",
  21.       "port": 3307,
  22.       "purge": {
  23.         "expires": 0,
  24.         "interval": 3600,
  25.         "remain_space": 15
  26.       },
  27.       "security": null,
  28.       "source_name": "mysql-02",
  29.       "user": "root"
  30.     }
  31.   ],
  32.   "total": 2
  33. }

Get the information of a data source

This API is a synchronous interface. If the request is successful, the information of the corresponding node is returned.

Request URI

GET /api/v1/sources/{source-name}/status

Example

  1. curl -X 'GET' \
  2.   'http://127.0.0.1:8261/api/v1/sources/mysql-replica-01/status' \
  3.   -H 'accept: application/json'
  1. {
  2.   "total": 1,
  3.   "data": [
  4.     {
  5.       "source_name": "mysql-replica-01",
  6.       "worker_name": "worker-1",
  7.       "relay_status": {
  8.         "master_binlog": "(mysql-bin.000001, 1979)",
  9.         "master_binlog_gtid": "e9a1fc22-ec08-11e9-b2ac-0242ac110003:1-7849",
  10.         "relay_dir": "./sub_dir",
  11.         "relay_binlog_gtid": "e9a1fc22-ec08-11e9-b2ac-0242ac110003:1-7849",
  12.         "relay_catch_up_master": true,
  13.         "stage": "Running"
  14.       },
  15.       "error_msg": "string"
  16.     }
  17.   ]
  18. }

Start the relay-log feature for data sources

This API is an asynchronous interface. If the request is successful, the status code of the returned body is 200. To learn about its latest status, You can get the information of a data source.

Request URI

POST /api/v1/sources/{source-name}/relay/enable

Example

  1. curl -X 'POST' \
  2.   'http://127.0.0.1:8261/api/v1/sources/mysql-01/relay/enable' \
  3.   -H 'accept: */*' \
  4.   -H 'Content-Type: application/json' \
  5.   -d '{
  6.   "worker_name_list": [
  7.     "worker-1"
  8.   ],
  9.   "relay_binlog_name": "mysql-bin.000002",
  10.   "relay_binlog_gtid": "e9a1fc22-ec08-11e9-b2ac-0242ac110003:1-7849",
  11.   "relay_dir": "./relay_log"
  12. }'

Stop the relay-log feature for data sources

This API is an asynchronous interface. If the request is successful, the status code of the returned body is 200. To learn about its latest status, You can get the information of a data source.

Request URI

POST /api/v1/sources/{source-name}/relay/disable

Example

  1. curl -X 'POST' \
  2.   'http://127.0.0.1:8261/api/v1/sources/mysql-01/relay/disable' \
  3.   -H 'accept: */*' \
  4.   -H 'Content-Type: application/json' \
  5.   -d '{
  6.   "worker_name_list": [
  7.     "worker-1"
  8.   ]
  9. }'

Purge relay log files that are no longer required

This API is an asynchronous interface. If the request is successful, the status code of the returned body is 200. To learn about its latest status, You can get the information of a data source.

Request URI

POST /api/v1/sources/{source-name}/relay/purge

Example

  1. curl -X 'POST' \
  2.   'http://127.0.0.1:8261/api/v1/sources/mysql-01/relay/purge' \
  3.   -H 'accept: */*' \
  4.   -H 'Content-Type: application/json' \
  5.   -d '{
  6.   "relay_binlog_name": "mysql-bin.000002",
  7.   "relay_dir": "string"
  8. }'

Change the bindings between the data source and DM-workers

This API is an asynchronous interface. If the request is successful, the status code of the returned body is 200. To learn about its latest status, You can get the information of a DM-worker node.

Request URI

POST /api/v1/sources/{source-name}/transfer

Example

  1. curl -X 'POST' \
  2.   'http://127.0.0.1:8261/api/v1/sources/mysql-01/transfer' \
  3.   -H 'accept: */*' \
  4.   -H 'Content-Type: application/json' \
  5.   -d '{
  6.   "worker_name": "worker-1"
  7. }'

Get the list of schema names of a data source

This API is a synchronous interface. If the request is successful, the corresponding list is returned.

Request URI

GET /api/v1/sources/{source-name}/schemas

Example

  1. curl -X 'GET' \
  2.   'http://127.0.0.1:8261/api/v1/sources/source-1/schemas' \
  3.   -H 'accept: application/json'
  1. [
  2.   "db1"
  3. ]

Get the list of table names of a specified schema in a data source

This API is a synchronous interface. If the request is successful, the corresponding list is returned.

Request URI

GET /api/v1/sources/{source-name}/schemas/{schema-name}

Example

  1. curl -X 'GET' \
  2.   'http://127.0.0.1:8261/api/v1/sources/source-1/schemas/db1' \
  3.   -H 'accept: application/json'
  1. [
  2.   "table1"
  3. ]

Create a replication task

This API is a synchronous interface. If the request is successful, the status code of the returned body is 200. A successful request will return the information of the corresponding replication task.

Request URI

POST /api/v1/tasks

Example

  1. curl -X 'POST' \
  2.   'http://127.0.0.1:8261/api/v1/tasks' \
  3.   -H 'accept: application/json' \
  4.   -H 'Content-Type: application/json' \
  5.   -d '{
  6.   "task": {
  7.     "name": "task-1",
  8.     "task_mode": "all",
  9.     "shard_mode": "pessimistic",
  10.     "meta_schema": "dm-meta",
  11.     "enhance_online_schema_change": true,
  12.     "on_duplicate": "overwrite",
  13.     "target_config": {
  14.       "host": "127.0.0.1",
  15.       "port": 3306,
  16.       "user": "root",
  17.       "password": "123456",
  18.       "security": {
  19.         "ssl_ca_content": "",
  20.         "ssl_cert_content": "",
  21.         "ssl_key_content": "",
  22.         "cert_allowed_cn": [
  23.           "string"
  24.         ]
  25.       }
  26.     },
  27.     "binlog_filter_rule": {
  28.       "rule-1": {
  29.         "ignore_event": [
  30.           "all dml"
  31.         ],
  32.         "ignore_sql": [
  33.           "^Drop"
  34.         ]
  35.       },
  36.       "rule-2": {
  37.         "ignore_event": [
  38.           "all dml"
  39.         ],
  40.         "ignore_sql": [
  41.           "^Drop"
  42.         ]
  43.       },
  44.       "rule-3": {
  45.         "ignore_event": [
  46.           "all dml"
  47.         ],
  48.         "ignore_sql": [
  49.           "^Drop"
  50.         ]
  51.       }
  52.     },
  53.     "table_migrate_rule": [
  54.       {
  55.         "source": {
  56.           "source_name": "source-name",
  57.           "schema": "db-*",
  58.           "table": "tb-*"
  59.         },
  60.         "target": {
  61.           "schema": "db1",
  62.           "table": "tb1"
  63.         },
  64.         "binlog_filter_rule": [
  65.           "rule-1",
  66.           "rule-2",
  67.           "rule-3",
  68.         ]
  69.       }
  70.     ],
  71.     "source_config": {
  72.       "full_migrate_conf": {
  73.         "export_threads": 4,
  74.         "import_threads": 16,
  75.         "data_dir": "./exported_data",
  76.         "consistency": "auto"
  77.         "import_mode": "physical",
  78.         "sorting_dir": "./sort_dir",
  79.         "disk_quota": "80G",
  80.         "checksum": "required",
  81.         "analyze": "optional",
  82.         "range_concurrency": 0,
  83.         "compress-kv-pairs": "",
  84.         "pd_addr": "",
  85.         "on_duplicate_logical": "error",
  86.         "on_duplicate_physical": "none"
  87.       },
  88.       "incr_migrate_conf": {
  89.         "repl_threads": 16,
  90.         "repl_batch": 100
  91.       },
  92.       "source_conf": [
  93.         {
  94.           "source_name": "mysql-replica-01",
  95.           "binlog_name": "binlog.000001",
  96.           "binlog_pos": 4,
  97.           "binlog_gtid": "03fc0263-28c7-11e7-a653-6c0b84d59f30:1-7041423,05474d3c-28c7-11e7-8352-203db246dd3d:1-170"
  98.         }
  99.       ]
  100.     }
  101.   }
  102. }'
  1. {
  2.   "name": "task-1",
  3.   "task_mode": "all",
  4.   "shard_mode": "pessimistic",
  5.   "meta_schema": "dm-meta",
  6.   "enhance_online_schema_change": true,
  7.   "on_duplicate": "overwrite",
  8.   "target_config": {
  9.     "host": "127.0.0.1",
  10.     "port": 3306,
  11.     "user": "root",
  12.     "password": "123456",
  13.     "security": {
  14.       "ssl_ca_content": "",
  15.       "ssl_cert_content": "",
  16.       "ssl_key_content": "",
  17.       "cert_allowed_cn": [
  18.         "string"
  19.       ]
  20.     }
  21.   },
  22.   "binlog_filter_rule": {
  23.     "rule-1": {
  24.       "ignore_event": [
  25.         "all dml"
  26.       ],
  27.       "ignore_sql": [
  28.         "^Drop"
  29.       ]
  30.     },
  31.     "rule-2": {
  32.       "ignore_event": [
  33.         "all dml"
  34.       ],
  35.       "ignore_sql": [
  36.         "^Drop"
  37.       ]
  38.     },
  39.     "rule-3": {
  40.       "ignore_event": [
  41.         "all dml"
  42.       ],
  43.       "ignore_sql": [
  44.         "^Drop"
  45.       ]
  46.     }
  47.   },
  48.   "table_migrate_rule": [
  49.     {
  50.       "source": {
  51.         "source_name": "source-name",
  52.         "schema": "db-*",
  53.         "table": "tb-*"
  54.       },
  55.       "target": {
  56.         "schema": "db1",
  57.         "table": "tb1"
  58.       },
  59.       "binlog_filter_rule": [
  60.         "rule-1",
  61.         "rule-2",
  62.         "rule-3",
  63.       ]
  64.     }
  65.   ],
  66.   "source_config": {
  67.     "full_migrate_conf": {
  68.       "export_threads": 4,
  69.       "import_threads": 16,
  70.       "data_dir": "./exported_data",
  71.       "consistency": "auto"
  72.       "import_mode": "physical",
  73.       "sorting_dir": "./sort_dir",
  74.       "disk_quota": "80G",
  75.       "checksum": "required",
  76.       "analyze": "optional",
  77.       "range_concurrency": 0,
  78.       "compress-kv-pairs": "",
  79.       "pd_addr": "",
  80.       "on_duplicate_logical": "error",
  81.       "on_duplicate_physical": "none"
  82.     },
  83.     "incr_migrate_conf": {
  84.       "repl_threads": 16,
  85.       "repl_batch": 100
  86.     },
  87.     "source_conf": [
  88.       {
  89.         "source_name": "mysql-replica-01",
  90.         "binlog_name": "binlog.000001",
  91.         "binlog_pos": 4,
  92.         "binlog_gtid": "03fc0263-28c7-11e7-a653-6c0b84d59f30:1-7041423,05474d3c-28c7-11e7-8352-203db246dd3d:1-170"
  93.       }
  94.     ]
  95.   }
  96. }

Get a replication task

This API is a synchronous interface. If the request is successful, the information of the corresponding replication task is returned.

Request URI

GET /api/v1/tasks/{task-name}?with_status=true

Example

  1. curl -X 'GET' \
  2.   'http://127.0.0.1:8261/api/v1/tasks/task-1?with_status=true' \
  3.   -H 'accept: application/json'
  1. {
  2.   "name": "task-1",
  3.   "task_mode": "all",
  4.   "shard_mode": "pessimistic",
  5.   "meta_schema": "dm-meta",
  6.   "enhance_online_schema_change": true,
  7.   "on_duplicate": "overwrite",
  8.   "target_config": {
  9.     "host": "127.0.0.1",
  10.     "port": 3306,
  11.     "user": "root",
  12.     "password": "123456",
  13.     "security": {
  14.       "ssl_ca_content": "",
  15.       "ssl_cert_content": "",
  16.       "ssl_key_content": "",
  17.       "cert_allowed_cn": [
  18.         "string"
  19.       ]
  20.     }
  21.   },
  22.   "binlog_filter_rule": {
  23.     "rule-1": {
  24.       "ignore_event": [
  25.         "all dml"
  26.       ],
  27.       "ignore_sql": [
  28.         "^Drop"
  29.       ]
  30.     },
  31.     "rule-2": {
  32.       "ignore_event": [
  33.         "all dml"
  34.       ],
  35.       "ignore_sql": [
  36.         "^Drop"
  37.       ]
  38.     },
  39.     "rule-3": {
  40.       "ignore_event": [
  41.         "all dml"
  42.       ],
  43.       "ignore_sql": [
  44.         "^Drop"
  45.       ]
  46.     }
  47.   },
  48.   "table_migrate_rule": [
  49.     {
  50.       "source": {
  51.         "source_name": "source-name",
  52.         "schema": "db-*",
  53.         "table": "tb-*"
  54.       },
  55.       "target": {
  56.         "schema": "db1",
  57.         "table": "tb1"
  58.       },
  59.       "binlog_filter_rule": [
  60.         "rule-1",
  61.         "rule-2",
  62.         "rule-3",
  63.       ]
  64.     }
  65.   ],
  66.   "source_config": {
  67.     "full_migrate_conf": {
  68.       "export_threads": 4,
  69.       "import_threads": 16,
  70.       "data_dir": "./exported_data",
  71.       "consistency": "auto",
  72.       "import_mode": "physical",
  73.       "sorting_dir": "./sort_dir",
  74.       "disk_quota": "80G",
  75.       "checksum": "required",
  76.       "analyze": "optional",
  77.       "range_concurrency": 0,
  78.       "compress-kv-pairs": "",
  79.       "pd_addr": "",
  80.       "on_duplicate_logical": "error",
  81.       "on_duplicate_physical": "none"
  82.     },
  83.     "incr_migrate_conf": {
  84.       "repl_threads": 16,
  85.       "repl_batch": 100
  86.     },
  87.     "source_conf": [
  88.       {
  89.         "source_name": "mysql-replica-01",
  90.         "binlog_name": "binlog.000001",
  91.         "binlog_pos": 4,
  92.         "binlog_gtid": "03fc0263-28c7-11e7-a653-6c0b84d59f30:1-7041423,05474d3c-28c7-11e7-8352-203db246dd3d:1-170"
  93.       }
  94.     ]
  95.   }
  96. }

Delete a replication task

This interface is a synchronous interface and the Status Code of the returned body is 204 upon successful request.

Request URI

DELETE /api/v1/tasks/{task-name}

Example

  1. curl -X 'DELETE' \
  2.   'http://127.0.0.1:8261/api/v1/tasks/task-1' \
  3.   -H 'accept: application/json'

Update a replication task

This interface is a synchronous interface and a successful request returns the information of the task.

OpenAPI - 图3

Note

When you use this API to update the task configuration, make sure that the task is stopped and has run into incremental sync and that only some of the fields can be updated.

Request URI

PUT /api/v1/tasks/{task-name}

Example

  1. curl -X 'PUT' \
  2.   'http://127.0.0.1:8261/api/v1/tasks/task-1' \
  3.   -H 'accept: application/json' \
  4.   -H 'Content-Type: application/json' \
  5.   -d '{
  6.   "task": {
  7.     "name": "task-1",
  8.     "task_mode": "all",
  9.     "shard_mode": "pessimistic",
  10.     "meta_schema": "dm-meta",
  11.     "enhance_online_schema_change": true,
  12.     "on_duplicate": "overwrite",
  13.     "target_config": {
  14.       "host": "127.0.0.1",
  15.       "port": 3306,
  16.       "user": "root",
  17.       "password": "123456",
  18.       "security": {
  19.         "ssl_ca_content": "",
  20.         "ssl_cert_content": "",
  21.         "ssl_key_content": "",
  22.         "cert_allowed_cn": [
  23.           "string"
  24.         ]
  25.       }
  26.     },
  27.     "binlog_filter_rule": {
  28.       "rule-1": {
  29.         "ignore_event": [
  30.           "all dml"
  31.         ],
  32.         "ignore_sql": [
  33.           "^Drop"
  34.         ]
  35.       },
  36.       "rule-2": {
  37.         "ignore_event": [
  38.           "all dml"
  39.         ],
  40.         "ignore_sql": [
  41.           "^Drop"
  42.         ]
  43.       },
  44.       "rule-3": {
  45.         "ignore_event": [
  46.           "all dml"
  47.         ],
  48.         "ignore_sql": [
  49.           "^Drop"
  50.         ]
  51.       }
  52.     },
  53.     "table_migrate_rule": [
  54.       {
  55.         "source": {
  56.           "source_name": "source-name",
  57.           "schema": "db-*",
  58.           "table": "tb-*"
  59.         },
  60.         "target": {
  61.           "schema": "db1",
  62.           "table": "tb1"
  63.         },
  64.         "binlog_filter_rule": [
  65.           "rule-1",
  66.           "rule-2",
  67.           "rule-3",
  68.         ]
  69.       }
  70.     ],
  71.     "source_config": {
  72.       "full_migrate_conf": {
  73.         "export_threads": 4,
  74.         "import_threads": 16,
  75.         "data_dir": "./exported_data",
  76.         "consistency": "auto",
  77.         "import_mode": "physical",
  78.         "sorting_dir": "./sort_dir",
  79.         "disk_quota": "80G",
  80.         "checksum": "required",
  81.         "analyze": "optional",
  82.         "range_concurrency": 0,
  83.         "compress-kv-pairs": "",
  84.         "pd_addr": "",
  85.         "on_duplicate_logical": "error",
  86.         "on_duplicate_physical": "none"
  87.       },
  88.       "incr_migrate_conf": {
  89.         "repl_threads": 16,
  90.         "repl_batch": 100
  91.       },
  92.       "source_conf": [
  93.         {
  94.           "source_name": "mysql-replica-01",
  95.           "binlog_name": "binlog.000001",
  96.           "binlog_pos": 4,
  97.           "binlog_gtid": "03fc0263-28c7-11e7-a653-6c0b84d59f30:1-7041423,05474d3c-28c7-11e7-8352-203db246dd3d:1-170"
  98.         }
  99.       ]
  100.     }
  101.   }
  102. }'
  1. {
  2.   "name": "task-1",
  3.   "task_mode": "all",
  4.   "shard_mode": "pessimistic",
  5.   "meta_schema": "dm-meta",
  6.   "enhance_online_schema_change": true,
  7.   "on_duplicate": "overwrite",
  8.   "target_config": {
  9.     "host": "127.0.0.1",
  10.     "port": 3306,
  11.     "user": "root",
  12.     "password": "123456",
  13.     "security": {
  14.       "ssl_ca_content": "",
  15.       "ssl_cert_content": "",
  16.       "ssl_key_content": "",
  17.       "cert_allowed_cn": [
  18.         "string"
  19.       ]
  20.     }
  21.   },
  22.   "binlog_filter_rule": {
  23.     "rule-1": {
  24.       "ignore_event": [
  25.         "all dml"
  26.       ],
  27.       "ignore_sql": [
  28.         "^Drop"
  29.       ]
  30.     },
  31.     "rule-2": {
  32.       "ignore_event": [
  33.         "all dml"
  34.       ],
  35.       "ignore_sql": [
  36.         "^Drop"
  37.       ]
  38.     },
  39.     "rule-3": {
  40.       "ignore_event": [
  41.         "all dml"
  42.       ],
  43.       "ignore_sql": [
  44.         "^Drop"
  45.       ]
  46.     }
  47.   },
  48.   "table_migrate_rule": [
  49.     {
  50.       "source": {
  51.         "source_name": "source-name",
  52.         "schema": "db-*",
  53.         "table": "tb-*"
  54.       },
  55.       "target": {
  56.         "schema": "db1",
  57.         "table": "tb1"
  58.       },
  59.       "binlog_filter_rule": [
  60.         "rule-1",
  61.         "rule-2",
  62.         "rule-3",
  63.       ]
  64.     }
  65.   ],
  66.   "source_config": {
  67.     "full_migrate_conf": {
  68.       "export_threads": 4,
  69.       "import_threads": 16,
  70.       "data_dir": "./exported_data",
  71.       "consistency": "auto"
  72.       "import_mode": "physical",
  73.       "sorting_dir": "./sort_dir",
  74.       "disk_quota": "80G",
  75.       "checksum": "required",
  76.       "analyze": "optional",
  77.       "range_concurrency": 0,
  78.       "compress-kv-pairs": "",
  79.       "pd_addr": "",
  80.       "on_duplicate_logical": "error",
  81.       "on_duplicate_physical": "none"
  82.     },
  83.     "incr_migrate_conf": {
  84.       "repl_threads": 16,
  85.       "repl_batch": 100
  86.     },
  87.     "source_conf": [
  88.       {
  89.         "source_name": "mysql-replica-01",
  90.         "binlog_name": "binlog.000001",
  91.         "binlog_pos": 4,
  92.         "binlog_gtid": "03fc0263-28c7-11e7-a653-6c0b84d59f30:1-7041423,05474d3c-28c7-11e7-8352-203db246dd3d:1-170"
  93.       }
  94.     ]
  95.   }
  96. }

Start a replication task

This API is an asynchronous interface. If the request is successful, the status code of the returned body is 204. To learn the latest status of a task, You can get the information of a replication task.

Request URI

POST /api/v1/tasks/{task-name}/start

Example

  1. curl -X 'POST' \
  2.   'http://127.0.0.1:8261/api/v1/tasks/task-1/start' \
  3.   -H 'accept: */*'

Stop a replication task

This API is an asynchronous interface. If the request is successful, the status code of the returned body is 200. To learn the latest status of a task, You can get the information of a replication task.

Request URI

POST /api/v1/tasks/{task-name}/stop

Example

  1. curl -X 'POST' \
  2.   'http://127.0.0.1:8261/api/v1/tasks/task-1/stop' \
  3.   -H 'accept: */*'

Get the information of a replication task

This API is a synchronous interface. If the request is successful, the information of the corresponding node is returned.

Request URI

GET /api/v1/tasks/task-1/status

Example

  1. curl -X 'GET' \
  2.   'http://127.0.0.1:8261/api/v1/tasks/task-1/status?stage=running' \
  3.   -H 'accept: application/json'
  1. {
  2.   "total": 1,
  3.   "data": [
  4.     {
  5.       "name": "string",
  6.       "source_name": "string",
  7.       "worker_name": "string",
  8.       "stage": "running",
  9.       "unit": "sync",
  10.       "unresolved_ddl_lock_id": "string",
  11.       "load_status": {
  12.         "finished_bytes": 0,
  13.         "total_bytes": 0,
  14.         "progress": "string",
  15.         "meta_binlog": "string",
  16.         "meta_binlog_gtid": "string"
  17.       },
  18.       "sync_status": {
  19.         "total_events": 0,
  20.         "total_tps": 0,
  21.         "recent_tps": 0,
  22.         "master_binlog": "string",
  23.         "master_binlog_gtid": "string",
  24.         "syncer_binlog": "string",
  25.         "syncer_binlog_gtid": "string",
  26.         "blocking_ddls": [
  27.           "string"
  28.         ],
  29.         "unresolved_groups": [
  30.           {
  31.             "target": "string",
  32.             "ddl_list": [
  33.               "string"
  34.             ],
  35.             "first_location": "string",
  36.             "synced": [
  37.               "string"
  38.             ],
  39.             "unsynced": [
  40.               "string"
  41.             ]
  42.           }
  43.         ],
  44.         "synced": true,
  45.         "binlog_type": "string",
  46.         "seconds_behind_master": 0
  47.       }
  48.     }
  49.   ]
  50. }

Get the replication task list

This API is a synchronous interface and a successful request returns a list of the corresponding tasks.

Request URI

GET /api/v1/tasks

Example

  1. curl -X 'GET' \
  2.   'http://127.0.0.1:8261/api/v1/tasks' \
  3.   -H 'accept: application/json'
  1. {
  2.   "total": 2,
  3.   "data": [
  4.     {
  5.       "name": "task-1",
  6.       "task_mode": "all",
  7.       "shard_mode": "pessimistic",
  8.       "meta_schema": "dm-meta",
  9.       "enhance_online_schema_change": true,
  10.       "on_duplicate": "overwrite",
  11.       "target_config": {
  12.         "host": "127.0.0.1",
  13.         "port": 3306,
  14.         "user": "root",
  15.         "password": "123456",
  16.         "security": {
  17.           "ssl_ca_content": "",
  18.           "ssl_cert_content": "",
  19.           "ssl_key_content": "",
  20.           "cert_allowed_cn": [
  21.             "string"
  22.           ]
  23.         }
  24.       },
  25.       "binlog_filter_rule": {
  26.         "rule-1": {
  27.           "ignore_event": [
  28.             "all dml"
  29.           ],
  30.           "ignore_sql": [
  31.             "^Drop"
  32.           ]
  33.         },
  34.         "rule-2": {
  35.           "ignore_event": [
  36.             "all dml"
  37.           ],
  38.           "ignore_sql": [
  39.             "^Drop"
  40.           ]
  41.         },
  42.         "rule-3": {
  43.           "ignore_event": [
  44.             "all dml"
  45.           ],
  46.           "ignore_sql": [
  47.             "^Drop"
  48.           ]
  49.         }
  50.       },
  51.       "table_migrate_rule": [
  52.         {
  53.           "source": {
  54.             "source_name": "source-name",
  55.             "schema": "db-*",
  56.             "table": "tb-*"
  57.           },
  58.           "target": {
  59.             "schema": "db1",
  60.             "table": "tb1"
  61.           },
  62.           "binlog_filter_rule": [
  63.             "rule-1",
  64.             "rule-2",
  65.             "rule-3",
  66.           ]
  67.         }
  68.       ],
  69.       "source_config": {
  70.         "full_migrate_conf": {
  71.           "export_threads": 4,
  72.           "import_threads": 16,
  73.           "data_dir": "./exported_data",
  74.           "consistency": "auto",
  75.           "import_mode": "physical",
  76.           "sorting_dir": "./sort_dir",
  77.           "disk_quota": "80G",
  78.           "checksum": "required",
  79.           "analyze": "optional",
  80.           "range_concurrency": 0,
  81.           "compress-kv-pairs": "",
  82.           "pd_addr": "",
  83.           "on_duplicate_logical": "error",
  84.           "on_duplicate_physical": "none"
  85.         },
  86.         "incr_migrate_conf": {
  87.           "repl_threads": 16,
  88.           "repl_batch": 100
  89.         },
  90.         "source_conf": [
  91.           {
  92.             "source_name": "mysql-replica-01",
  93.             "binlog_name": "binlog.000001",
  94.             "binlog_pos": 4,
  95.             "binlog_gtid": "03fc0263-28c7-11e7-a653-6c0b84d59f30:1-7041423,05474d3c-28c7-11e7-8352-203db246dd3d:1-170"
  96.           }
  97.         ]
  98.       }
  99.     }
  100.   ]
  101. }

Get the migration rules of a replication task

This API is a synchronous interface and a successful request returns a list of the migration rules of this task.

Request URI

GET /api/v1/tasks/{task-name}/sources/{source-name}/migrate_targets

Example

  1. curl -X 'GET' \
  2.   'http://127.0.0.1:8261/api/v1/tasks/task-1/sources/source-1/migrate_targets' \
  3.   -H 'accept: application/json'
  1. {
  2.   "total": 0,
  3.   "data": [
  4.     {
  5.       "source_schema": "db1",
  6.       "source_table": "tb1",
  7.       "target_schema": "db1",
  8.       "target_table": "tb1"
  9.     }
  10.   ]
  11. }

Get the list of schema names of the data source that is associated with a replication task

This API is a synchronous interface. If the request is successful, the corresponding list is returned.

Request URI

GET /api/v1/tasks/{task-name}/sources/{source-name}/schemas

Example

  1. curl -X 'GET' \
  2.   'http://127.0.0.1:8261/api/v1/tasks/task-1/sources/source-1/schemas' \
  3.   -H 'accept: application/json'
  1. [
  2.   "db1"
  3. ]

Get the list of table names of a specified schema in the data source that is associated with a replication task

This API is a synchronous interface. If the request is successful, the corresponding list is returned.

Request URI

GET /api/v1/tasks/{task-name}/sources/{source-name}/schemas/{schema-name}

Example

  1. curl -X 'GET' \
  2.   'http://127.0.0.1:8261/api/v1/tasks/task-1/sources/source-1/schemas/db1' \
  3.   -H 'accept: application/json'
  1. [
  2.   "table1"
  3. ]

Get the CREATE statement for schemas of the data source that is associated with a replication task

This API is a synchronous interface. If the request is successful, the corresponding CREATE statement is returned.

Request URI

GET /api/v1/tasks/{task-name}/sources/{source-name}/schemas/{schema-name}/{table-name}

Example

  1. curl -X 'GET' \
  2.   'http://127.0.0.1:8261/api/v1/tasks/task-1/sources/source-1/schemas/db1/table1' \
  3.   -H 'accept: application/json'
  1. {
  2.   "schema_name": "db1",
  3.   "table_name": "table1",
  4.   "schema_create_sql": "CREATE TABLE `t1` (`id` int(11) NOT NULL AUTO_INCREMENT,PRIMARY KEY (`id`) /*T![clustered_index] CLUSTERED */) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin"
  5. }

Update the CREATE statement for schemas of the data source that is associated with a replication task

This API is a synchronous interface. If the request is successful, the status code of the returned body is 200.

Request URI

POST /api/v1/tasks/{task-name}/sources/{source-name}/schemas/{schema-name}/{table-name}

Example

  1. curl -X 'PUT' \
  2.   'http://127.0.0.1:8261/api/v1/tasks/task-1/sources/task-1/schemas/db1/table1' \
  3.   -H 'accept: */*' \
  4.   -H 'Content-Type: application/json' \
  5.   -d '{
  6.   "sql_content": "CREATE TABLE `t1` ( `c1` int(11) DEFAULT NULL, `c2` int(11) DEFAULT NULL, `c3` int(11) DEFAULT NULL) ENGINE=InnoDB DEFAULT CHARSET=utf8mb4 COLLATE=utf8mb4_bin;",
  7.   "flush": true,
  8.   "sync": true
  9. }'

Delete a schema of the data source that is associated with a replication task

This API is a synchronous interface. If the request is successful, the status code of the returned body is 200.

Request URI

DELETE /api/v1/tasks/{task-name}/sources/{source-name}/schemas/{schema-name}/{table-name}

Example

  1. curl -X 'DELETE' \
  2.   'http://127.0.0.1:8261/api/v1/tasks/task-1/sources/source-1/schemas/db1/table1' \
  3.   -H 'accept: */*'