Data Check in the DM Replication Scenario

When using replication tools such as TiDB Data Migration, you need to check the data consistency before and after the replication process. You can set a specific task-name configuration from DM-master to perform a data check.

The following is a simple configuration example. To learn the complete configuration, refer to Sync-diff-inspector User Guide.

  1. # Diff Configuration.
  3. ######################### Global config #########################
  5. # The number of goroutines created to check data. The number of connections between upstream and downstream databases are slightly greater than this value.
  6. check-thread-count = 4
  8. # If enabled, SQL statements is exported to fix inconsistent tables.
  9. export-fix-sql = true
  11. # Only compares the table structure instead of the data.
  12. check-struct-only = false
  14. # The IP address of dm-master and the format is "http://127.0.0.1:8261".
  15. dm-addr = "http://127.0.0.1:8261"
  17. # Specifies the `task-name` of DM.
  18. dm-task = "test"
  20. ######################### Task config #########################
  21. [task]
  22.     output-dir = "./output"
  24.     # The tables of downstream databases to be compared. Each table needs to contain the schema name and the table name, separated by '.'
  25.     target-check-tables = ["hb_test.*"]

This example is configured in dm-task = “test”, which checks all the tables of hb_test schema under the “test” task. It automatically gets the regular matching of the schemas between upstream and downstream databases to verify the data consistency after DM replication.