Sharding Algorithm

Background

ShardingSphere built-in algorithms provide a variety of sharding algorithms, which can be divided into automatic sharding algorithms, standard sharding algorithms, composite sharding algorithms, and hint sharding algorithms, and can meet the needs of most business scenarios of users.

Additionally, considering the complexity of business scenarios, the built-in algorithm also provides a way to customize the sharding algorithm. Users can complete complex sharding logic by writing java code.

It should be noted that the sharding logic of the automatic sharding algorithm is automatically managed by ShardingSphere and needs to be used by configuring the autoTables sharding rules.

Parameters

Auto Sharding Algorithm

Modulo Sharding Algorithm

Type: MOD

Attributes:

NameDataTypeDescription
sharding-countintSharding count

Hash Modulo Sharding Algorithm

Type: HASH_MOD

Attributes:

NameDataTypeDescription
sharding-countintSharding count

Volume Based Range Sharding Algorithm

Type: VOLUME_RANGE

Attributes:

NameDataTypeDescription
range-lowerlongRange lower bound, throw exception if lower than bound
range-upperlongRange upper bound, throw exception if upper than bound
sharding-volumelongSharding volume

Boundary Based Range Sharding Algorithm

Type: BOUNDARY_RANGE

Attributes:

NameDataTypeDescription
sharding-rangesStringRange of sharding border, multiple boundaries separated by commas

Auto Interval Sharding Algorithm

Type: AUTO_INTERVAL

Attributes:

NameDataTypeDescription
datetime-lowerStringShard datetime begin boundary, pattern: yyyy-MM-dd HH:mm:ss
datetime-upperStringShard datetime end boundary, pattern: yyyy-MM-dd HH:mm:ss
sharding-secondslongMax seconds for the data in one shard, allows sharding key timestamp format seconds with time precision, but time precision after seconds is automatically erased

Standard Sharding Algorithm

Apache ShardingSphere built-in standard sharding algorithm are:

Inline Sharding Algorithm

With Groovy expressions that uses the default implementation of the InlineExpressionParser SPI, InlineShardingStrategy provides single-key support for the sharding operation of = and IN in SQL. Simple sharding algorithms can be used through a simple configuration to avoid laborious Java code developments. For example, t_user_$->{u_id % 8} means table t_user is divided into 8 tables according to u_id, with table names from t_user_0 to t_user_7. Please refer to Inline Expression for more details.

Type: INLINE

Attributes:

NameDataTypeDescriptionDefault Value
algorithm-expressionStringInline expression sharding algorithm-
allow-range-query-with-inline-sharding (?)booleanWhether range query is allowed. Note: range query will ignore sharding strategy and conduct full routingfalse

Interval Sharding Algorithm

This algorithm actively ignores the time zone information of datetime-pattern. This means that when datetime-lower, datetime-upper and the incoming shard key contain time zone information, time zone conversion will not occur due to time zone inconsistencies. When the incoming sharding key is java.time.Instant, there is a special case, which will carry the time zone information of the system and convert it into the string format of datetime-pattern, and then proceed to the next sharding.

Type: INTERVAL

Attributes:

NameDataTypeDescriptionDefault Value
datetime-patternStringTimestamp pattern of sharding value, must can be transformed to Java LocalDateTime. For example: yyyy-MM-dd HH:mm:ss, yyyy-MM-dd or HH:mm:ss etc. But Gy-MM etc. related to java.time.chrono.JapaneseDate are not supported-
datetime-lowerStringDatetime sharding lower boundary, pattern is defined datetime-pattern-
datetime-upper (?)StringDatetime sharding upper boundary, pattern is defined datetime-patternNow
sharding-suffix-patternStringSuffix pattern of sharding data sources or tables, must can be transformed to Java LocalDateTime, must be consistent with datetime-interval-unit. For example: yyyyMM-
datetime-interval-amount (?)intInterval of sharding value, after which the next shard will be entered1
datetime-interval-unit (?)StringUnit of sharding value interval, must can be transformed to Java ChronoUnit’s Enum value. For example: MONTHSDAYS

Complex Sharding Algorithm

Complex Inline Sharding Algorithm

Please refer to Inline Expression for more details.

Type: COMPLEX_INLINE

NameDataTypeDescriptionDefault Value
sharding-columns (?)Stringsharing column names-
algorithm-expressionStringInline expression sharding algorithm-
allow-range-query-with-inline-sharding (?)booleanWhether range query is allowed. Note: range query will ignore sharding strategy and conduct full routingfalse

Hint Sharding Algorithm

Hint Inline Sharding Algorithm

Please refer to Inline Expression for more details.

Type: COMPLEX_INLINE

NameDataTypeDescriptionDefault Value
algorithm-expressionStringInline expression sharding algorithm${value}

Class Based Sharding Algorithm

Realize custom extension by configuring the sharding strategy type and algorithm class name. CLASS_BASED allows additional custom properties to be passed into the algorithm class. The passed properties can be retrieved through the java.util.Properties class instance with the property name props. Refer to Git’s org.apache.shardingsphere.example.extension.sharding.algortihm.classbased.fixture.ClassBasedStandardShardingAlgorithmFixture.

Type：CLASS_BASED

Attributes：

NameDataTypeDescription
strategyStringSharding strategy type, support STANDARD, COMPLEX or HINT (case insensitive)
algorithmClassNameStringFully qualified name of sharding algorithm

Procedure

  1. When using data sharding, configure the corresponding data sharding algorithm under the shardingAlgorithms attribute.

Sample

  1. rules:
  2. - !SHARDING
  3.   tables:
  4.     t_order: 
  5.       actualDataNodes: ds_${0..1}.t_order_${0..1}
  6.       tableStrategy: 
  7.         standard:
  8.           shardingColumn: order_id
  9.           shardingAlgorithmName: t_order_inline
  10.       keyGenerateStrategy:
  11.         column: order_id
  12.         keyGeneratorName: snowflake
  13.     t_order_item:
  14.       actualDataNodes: ds_${0..1}.t_order_item_${0..1}
  15.       tableStrategy:
  16.         standard:
  17.           shardingColumn: order_id
  18.           shardingAlgorithmName: t_order_item_inline
  19.       keyGenerateStrategy:
  20.         column: order_item_id
  21.         keyGeneratorName: snowflake
  22.     t_account:
  23.       actualDataNodes: ds_${0..1}.t_account_${0..1}
  24.       tableStrategy:
  25.         standard:
  26.           shardingAlgorithmName: t_account_inline
  27.       keyGenerateStrategy:
  28.         column: account_id
  29.         keyGeneratorName: snowflake
  30.   defaultShardingColumn: account_id
  31.   bindingTables:
  32.     - t_order,t_order_item
  33.   defaultDatabaseStrategy:
  34.     standard:
  35.       shardingColumn: user_id
  36.       shardingAlgorithmName: database_inline
  37.   defaultTableStrategy:
  38.     none:
  40.   shardingAlgorithms:
  41.     database_inline:
  42.       type: INLINE
  43.       props:
  44.         algorithm-expression: ds_${user_id % 2}
  45.     t_order_inline:
  46.       type: INLINE
  47.       props:
  48.         algorithm-expression: t_order_${order_id % 2}
  49.     t_order_item_inline:
  50.       type: INLINE
  51.       props:
  52.         algorithm-expression: t_order_item_${order_id % 2}
  53.     t_account_inline:
  54.       type: INLINE
  55.       props:
  56.         algorithm-expression: t_account_${account_id % 2}
  57.   keyGenerators:
  58.     snowflake:
  59.       type: SNOWFLAKE
  61. - !BROADCAST
  62.   tables:
  63.     - t_address