Sharding

Background

Data sharding YAML configuration is highly readable. The dependencies between sharding rules can be quickly understood through the YAML format. ShardingSphere automatically creates the ShardingSphereDataSource object according to YAML configuration, which can reduce unnecessary coding for users.

Parameters

  1. rules:
  2. - !SHARDING
  3.   tables: # Sharding table configuration
  4.     <logic_table_name> (+): # Logic table name
  5.       actualDataNodes (?): # Describe data source names and actual tables (refer to Inline syntax rules)
  6.       databaseStrategy (?): # Databases sharding strategy, use default databases sharding strategy if absent. sharding strategy below can choose only one.
  7.         standard: # For single sharding column scenario
  8.           shardingColumn: # Sharding column name
  9.           shardingAlgorithmName: # Sharding algorithm name
  10.         complex: # For multiple sharding columns scenario
  11.           shardingColumns: # Sharding column names, multiple columns separated with comma
  12.           shardingAlgorithmName: # Sharding algorithm name
  13.         hint: # Sharding by hint
  14.           shardingAlgorithmName: # Sharding algorithm name
  15.         none: # Do not sharding
  16.       tableStrategy: # Tables sharding strategy, same as database sharding strategy
  17.       keyGenerateStrategy: # Key generator strategy
  18.         column: # Column name of key generator
  19.         keyGeneratorName: # Key generator name
  20.       auditStrategy: # Sharding audit strategy
  21.         auditorNames: # Sharding auditor name
  22.           - <auditor_name>
  23.           - <auditor_name>
  24.         allowHintDisable: true # Enable or disable sharding audit hint
  25.   autoTables: # Auto Sharding table configuration
  26.     t_order_auto: # Logic table name
  27.       actualDataSources (?): # Data source names
  28.       shardingStrategy: # Sharding strategy
  29.         standard: # For single sharding column scenario
  30.           shardingColumn: # Sharding column name
  31.           shardingAlgorithmName: # Auto sharding algorithm name
  32.   bindingTables (+): # Binding tables
  33.     - <logic_table_name_1, logic_table_name_2, ...> 
  34.     - <logic_table_name_1, logic_table_name_2, ...> 
  35.   defaultDatabaseStrategy: # Default strategy for database sharding
  36.   defaultTableStrategy: # Default strategy for table sharding
  37.   defaultKeyGenerateStrategy: # Default Key generator strategy
  38.   defaultShardingColumn: # Default sharding column name
  40.   # Sharding algorithm configuration
  41.   shardingAlgorithms:
  42.     <sharding_algorithm_name> (+): # Sharding algorithm name
  43.       type: # Sharding algorithm type
  44.       props: # Sharding algorithm properties
  45.       # ...
  47.   # Key generate algorithm configuration
  48.   keyGenerators:
  49.     <key_generate_algorithm_name> (+): # Key generate algorithm name
  50.       type: # Key generate algorithm type
  51.       props: # Key generate algorithm properties
  52.       # ...
  54.   # Sharding audit algorithm configuration
  55.   auditors:
  56.     <sharding_audit_algorithm_name> (+): # Sharding audit algorithm name
  57.       type: # Sharding audit algorithm type
  58.       props: # Sharding audit algorithm properties
  59.       # ...
  61. - !BROADCAST
  62.   tables: # Broadcast tables
  63.     - <table_name>
  64.     - <table_name>

Procedure

  1. Configure data sharding rules in YAML files, including data source, sharding rules, and global attributes and other configuration items.
  2. Call createDataSource method of the object YamlShardingSphereDataSourceFactory. Create ShardingSphereDataSource according to the configuration information in YAML files.

Sample

The YAML configuration sample of data sharding is as follows:

  1. dataSources:
  2.   ds_0:
  3.     dataSourceClassName: com.zaxxer.hikari.HikariDataSource
  4.     driverClassName: com.mysql.jdbc.Driver
  5.     jdbcUrl: jdbc:mysql://localhost:3306/demo_ds_0?serverTimezone=UTC&useSSL=false&useUnicode=true&characterEncoding=UTF-8
  6.     username: root
  7.     password:
  8.   ds_1:
  9.     dataSourceClassName: com.zaxxer.hikari.HikariDataSource
  10.     driverClassName: com.mysql.jdbc.Driver
  11.     jdbcUrl: jdbc:mysql://localhost:3306/demo_ds_1?serverTimezone=UTC&useSSL=false&useUnicode=true&characterEncoding=UTF-8
  12.     username: root
  13.     password:
  15. rules:
  16. - !SHARDING
  17.   tables:
  18.     t_order: 
  19.       actualDataNodes: ds_${0..1}.t_order_${0..1}
  20.       tableStrategy: 
  21.         standard:
  22.           shardingColumn: order_id
  23.           shardingAlgorithmName: t_order_inline
  24.       keyGenerateStrategy:
  25.         column: order_id
  26.         keyGeneratorName: snowflake
  27.       auditStrategy:
  28.         auditorNames:
  29.           - sharding_key_required_auditor
  30.         allowHintDisable: true
  31.     t_order_item:
  32.       actualDataNodes: ds_${0..1}.t_order_item_${0..1}
  33.       tableStrategy:
  34.         standard:
  35.           shardingColumn: order_id
  36.           shardingAlgorithmName: t_order_item_inline
  37.       keyGenerateStrategy:
  38.         column: order_item_id
  39.         keyGeneratorName: snowflake
  40.     t_account:
  41.       actualDataNodes: ds_${0..1}.t_account_${0..1}
  42.       tableStrategy:
  43.         standard:
  44.           shardingAlgorithmName: t_account_inline
  45.       keyGenerateStrategy:
  46.         column: account_id
  47.         keyGeneratorName: snowflake
  48.   defaultShardingColumn: account_id
  49.   bindingTables:
  50.     - t_order,t_order_item
  51.   defaultDatabaseStrategy:
  52.     standard:
  53.       shardingColumn: user_id
  54.       shardingAlgorithmName: database_inline
  55.   defaultTableStrategy:
  56.     none:
  58.   shardingAlgorithms:
  59.     database_inline:
  60.       type: INLINE
  61.       props:
  62.         algorithm-expression: ds_${user_id % 2}
  63.     t_order_inline:
  64.       type: INLINE
  65.       props:
  66.         algorithm-expression: t_order_${order_id % 2}
  67.     t_order_item_inline:
  68.       type: INLINE
  69.       props:
  70.         algorithm-expression: t_order_item_${order_id % 2}
  71.     t_account_inline:
  72.       type: INLINE
  73.       props:
  74.         algorithm-expression: t_account_${account_id % 2}
  75.   keyGenerators:
  76.     snowflake:
  77.       type: SNOWFLAKE
  78.   auditors:
  79.     sharding_key_required_auditor:
  80.       type: DML_SHARDING_CONDITIONS
  82. - !BROADCAST
  83.   tables: # Broadcast tables
  84.     - t_address
  86. props:
  87.   sql-show: false

Read the YAML configuration to create a data source according to the createDataSource method of YamlShardingSphereDataSourceFactory.

  1. YamlShardingSphereDataSourceFactory.createDataSource(getFile("/META-INF/sharding-databases-tables.yaml"));