Sharding

Background

Data sharding YAML configuration is highly readable. The dependencies between sharding rules can be quickly understood through the YAML format. ShardingSphere automatically creates the ShardingSphereDataSource object according to YAML configuration, which can reduce unnecessary coding for users.

Parameters

  1. rules:
  2. - !SHARDING
  3.   tables: # Sharding table configuration
  4.     <logic-table-name> (+): # Logic table name
  5.       actualDataNodes (?): # Describe data source names and actual tables (refer to Inline syntax rules)
  6.       databaseStrategy (?): # Databases sharding strategy, use default databases sharding strategy if absent. sharding strategy below can choose only one.
  7.         standard: # For single sharding column scenario
  8.           shardingColumn: # Sharding column name
  9.           shardingAlgorithmName: # Sharding algorithm name
  10.         complex: # For multiple sharding columns scenario
  11.           shardingColumns: # Sharding column names, multiple columns separated with comma
  12.           shardingAlgorithmName: # Sharding algorithm name
  13.         hint: # Sharding by hint
  14.           shardingAlgorithmName: # Sharding algorithm name
  15.         none: # Do not sharding
  16.       tableStrategy: # Tables sharding strategy, same as database sharding strategy
  17.       keyGenerateStrategy: # Key generator strategy
  18.         column: # Column name of key generator
  19.         keyGeneratorName: # Key generator name
  20.       auditStrategy: # Sharding audit strategy
  21.         auditorNames: # Sharding auditor name
  22.           - <auditor-name>
  23.           - <auditor-name>
  24.         allowHintDisable: true # Enable or disable sharding audit hint
  25.   autoTables: # Auto Sharding table configuration
  26.     t_order_auto: # Logic table name
  27.       actualDataSources (?): # Data source names
  28.       shardingStrategy: # Sharding strategy
  29.         standard: # For single sharding column scenario
  30.           shardingColumn: # Sharding column name
  31.           shardingAlgorithmName: # Auto sharding algorithm name
  32.   bindingTables (+): # Binding tables
  33.     - <logic_table_name_1, logic_table_name_2, ...> 
  34.     - <logic_table_name_1, logic_table_name_2, ...> 
  35.   broadcastTables (+): # Broadcast tables
  36.     - <table-name>
  37.     - <table-name>
  38.   defaultDatabaseStrategy: # Default strategy for database sharding
  39.   defaultTableStrategy: # Default strategy for table sharding
  40.   defaultKeyGenerateStrategy: # Default Key generator strategy
  41.   defaultShardingColumn: # Default sharding column name
  43.   # Sharding algorithm configuration
  44.   shardingAlgorithms:
  45.     <sharding-algorithm-name> (+): # Sharding algorithm name
  46.       type: # Sharding algorithm type
  47.       props: # Sharding algorithm properties
  48.       # ...
  50.   # Key generate algorithm configuration
  51.   keyGenerators:
  52.     <key-generate-algorithm-name> (+): # Key generate algorithm name
  53.       type: # Key generate algorithm type
  54.       props: # Key generate algorithm properties
  55.       # ...
  57.   # Sharding audit algorithm configuration
  58.   auditors:
  59.     <sharding-audit-algorithm-name> (+): # Sharding audit algorithm name
  60.       type: # Sharding audit algorithm type
  61.       props: # Sharding audit algorithm properties
  62.       # ...

Procedure

  1. Configure data sharding rules in YAML files, including data source, sharding rules, and global attributes and other configuration items.
  2. Call createDataSource method of the object YamlShardingSphereDataSourceFactory. Create ShardingSphereDataSource according to the configuration information in YAML files.

Sample

The YAML configuration sample of data sharding is as follows:

  1. dataSources:
  2.   ds_0:
  3.     dataSourceClassName: com.zaxxer.hikari.HikariDataSource
  4.     driverClassName: com.mysql.jdbc.Driver
  5.     jdbcUrl: jdbc:mysql://localhost:3306/demo_ds_0?serverTimezone=UTC&useSSL=false&useUnicode=true&characterEncoding=UTF-8
  6.     username: root
  7.     password:
  8.   ds_1:
  9.     dataSourceClassName: com.zaxxer.hikari.HikariDataSource
  10.     driverClassName: com.mysql.jdbc.Driver
  11.     jdbcUrl: jdbc:mysql://localhost:3306/demo_ds_1?serverTimezone=UTC&useSSL=false&useUnicode=true&characterEncoding=UTF-8
  12.     username: root
  13.     password:
  15. rules:
  16. - !SHARDING
  17.   tables:
  18.     t_order: 
  19.       actualDataNodes: ds_${0..1}.t_order_${0..1}
  20.       tableStrategy: 
  21.         standard:
  22.           shardingColumn: order_id
  23.           shardingAlgorithmName: t-order-inline
  24.       keyGenerateStrategy:
  25.         column: order_id
  26.         keyGeneratorName: snowflake
  27.       auditStrategy:
  28.         auditorNames:
  29.           - sharding_key_required_auditor
  30.         allowHintDisable: true
  31.     t_order_item:
  32.       actualDataNodes: ds_${0..1}.t_order_item_${0..1}
  33.       tableStrategy:
  34.         standard:
  35.           shardingColumn: order_id
  36.           shardingAlgorithmName: t_order-item-inline
  37.       keyGenerateStrategy:
  38.         column: order_item_id
  39.         keyGeneratorName: snowflake
  40.     t_account:
  41.       actualDataNodes: ds_${0..1}.t_account_${0..1}
  42.       tableStrategy:
  43.         standard:
  44.           shardingAlgorithmName: t-account-inline
  45.       keyGenerateStrategy:
  46.         column: account_id
  47.         keyGeneratorName: snowflake
  48.   defaultShardingColumn: account_id
  49.   bindingTables:
  50.     - t_order,t_order_item
  51.   broadcastTables:
  52.     - t_address
  53.   defaultDatabaseStrategy:
  54.     standard:
  55.       shardingColumn: user_id
  56.       shardingAlgorithmName: database-inline
  57.   defaultTableStrategy:
  58.     none:
  60.   shardingAlgorithms:
  61.     database-inline:
  62.       type: INLINE
  63.       props:
  64.         algorithm-expression: ds_${user_id % 2}
  65.     t-order-inline:
  66.       type: INLINE
  67.       props:
  68.         algorithm-expression: t_order_${order_id % 2}
  69.     t_order-item-inline:
  70.       type: INLINE
  71.       props:
  72.         algorithm-expression: t_order_item_${order_id % 2}
  73.     t-account-inline:
  74.       type: INLINE
  75.       props:
  76.         algorithm-expression: t_account_${account_id % 2}
  77.   keyGenerators:
  78.     snowflake:
  79.       type: SNOWFLAKE
  80.   auditors:
  81.     sharding_key_required_auditor:
  82.       type: DML_SHARDING_CONDITIONS
  84. props:
  85.   sql-show: false

Read the YAML configuration to create a data source according to the createDataSource method of YamlShardingSphereDataSourceFactory.

  1. YamlShardingSphereDataSourceFactory.createDataSource(getFile("/META-INF/sharding-databases-tables.yaml"));