TiDB Binlog Deployment Topology

This document describes the deployment topology of TiDB Binlog based on the minimal TiDB topology.

TiDB Binlog is the widely used component for replicating incremental data. It provides near real-time backup and replication.

Topology information

InstanceCountPhysical machine configurationIPConfiguration
TiDB316 VCore 32 GB10.0.1.1
10.0.1.2
10.0.1.3		Default port configuration;
Enable enable_binlog;
Enable ignore-error
PD34 VCore 8 GB10.0.1.4
10.0.1.5
10.0.1.6		Default port configuration
TiKV316 VCore 32 GB10.0.1.7
10.0.1.8
10.0.1.9		Default port configuration
Pump38 VCore 16GB10.0.1.1
10.0.1.7
10.0.1.8		Default port configuration;
Set GC time to 7 days
Drainer18 VCore 16GB10.0.1.12Default port configuration;
Set the default initialization commitTS -1 as the latest timestamp;
Configure the downstream target TiDB as 10.0.1.12:4000

Topology templates

For detailed descriptions of the configuration items in the above TiDB cluster topology file, see Topology Configuration File for Deploying TiDB Using TiUP.

Key parameters

The key parameters in the topology configuration templates are as follows:

  • server_configs.tidb.binlog.enable: true

    • Enables the binlog service.
    • Default value: false.

  • server_configs.tidb.binlog.ignore-error: true

    • It is recommended to enable this configuration in high availability scenarios.
    • If set to true, when an error occurs, TiDB stops writing data into binlog, and adds 1 to the value of the tidb_server_critical_error_total monitoring metric.
    • If set to false, when TiDB fails to write data into binlog, the whole TiDB service is stopped.

  • drainer_servers.config.syncer.db-type

    The downstream type of TiDB Binlog. Currently, mysql, tidb, kafka, and file are supported.

  • drainer_servers.config.syncer.to

    The downstream configuration of TiDB Binlog. Depending on different db-types, you can use this configuration item to configure the connection parameters of the downstream database, the connection parameters of Kafka, and the file save path. For details, refer to TiDB Binlog Configuration File.

TiDB Binlog Topology - 图1

Note

  • When editing the configuration file template, if you do not need custom ports or directories, modify the IP only.
  • You do not need to manually create the tidb user in the configuration file. The TiUP cluster component automatically creates the tidb user on the target machines. You can customize the user, or keep the user consistent with the control machine.
  • If you configure the deployment directory as a relative path, the cluster will be deployed in the home directory of the user.