Sharding Algorithm

Background

ShardingSphere built-in algorithms provide a variety of sharding algorithms, which can be divided into automatic sharding algorithms, standard sharding algorithms, composite sharding algorithms, and hint sharding algorithms, and can meet the needs of most business scenarios of users.

Additionally, considering the complexity of business scenarios, the built-in algorithm also provides a way to customize the sharding algorithm. Users can complete complex sharding logic by writing java code.

Parameters

Auto Sharding Algorithm

Modulo Sharding Algorithm

Type: MOD

Attributes:

NameDataTypeDescription
sharding-countintSharding count

Modulo sharding algorithm provided by CosId

Modulo sharding algorithm implemented by the tool class based on me.ahoo.cosid:cosid-core. See the discussion at https://github.com/apache/shardingsphere/issues/14047 .

Type: COSID_MOD

Attributes:

NameDataTypeDescription
modintSharding count
logic-name-prefixStringPrefix pattern of sharding data sources or tables

Hash Modulo Sharding Algorithm

Type: HASH_MOD

Attributes:

NameDataTypeDescription
sharding-countintSharding count

Volume Based Range Sharding Algorithm

Type: VOLUME_RANGE

Attributes:

NameDataTypeDescription
range-lowerlongRange lower bound, throw exception if lower than bound
range-upperlongRange upper bound, throw exception if upper than bound
sharding-volumelongSharding volume

Boundary Based Range Sharding Algorithm

Type: BOUNDARY_RANGE

Attributes:

NameDataTypeDescription
sharding-rangesStringRange of sharding border, multiple boundaries separated by commas

Auto Interval Sharding Algorithm

Type: AUTO_INTERVAL

Attributes:

NameDataTypeDescription
datetime-lowerStringShard datetime begin boundary, pattern: yyyy-MM-dd HH:mm:ss
datetime-upperStringShard datetime end boundary, pattern: yyyy-MM-dd HH:mm:ss
sharding-secondslongMax seconds for the data in one shard, allows sharding key timestamp format seconds with time precision, but time precision after seconds is automatically erased

Standard Sharding Algorithm

Apache ShardingSphere built-in standard sharding algorithm are:

Inline Sharding Algorithm

With Groovy expressions, InlineShardingStrategy provides single-key support for the sharding operation of = and IN in SQL. Simple sharding algorithms can be used through a simple configuration to avoid laborious Java code developments. For example, t_user_$->{u_id % 8} means table t_user is divided into 8 tables according to u_id, with table names from t_user_0 to t_user_7. Please refer to Inline Expression for more details.

Type: INLINE

Attributes:

NameDataTypeDescriptionDefault Value
algorithm-expressionStringInline expression sharding algorithm-
allow-range-query-with-inline-sharding (?)booleanWhether range query is allowed. Note: range query will ignore sharding strategy and conduct full routingfalse

Interval Sharding Algorithm

This algorithm actively ignores the time zone information of datetime-pattern. This means that when datetime-lower, datetime-upper and the incoming shard key contain time zone information, time zone conversion will not occur due to time zone inconsistencies. When the incoming sharding key is java.time.Instant, there is a special case, which will carry the time zone information of the system and convert it into the string format of datetime-pattern, and then proceed to the next sharding.

Type: INTERVAL

Attributes:

NameDataTypeDescriptionDefault Value
datetime-patternStringTimestamp pattern of sharding value, must can be transformed to Java LocalDateTime. For example: yyyy-MM-dd HH:mm:ss, yyyy-MM-dd or HH:mm:ss etc. But Gy-MM etc. related to java.time.chrono.JapaneseDate are not supported-
datetime-lowerStringDatetime sharding lower boundary, pattern is defined datetime-pattern-
datetime-upper (?)StringDatetime sharding upper boundary, pattern is defined datetime-patternNow
sharding-suffix-patternStringSuffix pattern of sharding data sources or tables, must can be transformed to Java LocalDateTime, must be consistent with datetime-interval-unit. For example: yyyyMM-
datetime-interval-amount (?)intInterval of sharding value, after which the next shard will be entered1
datetime-interval-unit (?)StringUnit of sharding value interval, must can be transformed to Java ChronoUnit’s Enum value. For example: MONTHSDAYS

Fixed interval sharding algorithm provided by CosId

A fixed time range sharding algorithm implemented by the tool class based on me.ahoo.cosid:cosid-core. When the sharding key is a JSR-310 containing class or a time-related class, it will be converted to java.time.LocalDateTime before the next sharding. See the discussion at https://github.com/apache/shardingsphere/issues/14047.

Type：COSID_INTERVAL

Attributes：

NameDataTypeDescriptionDefault Value
zone-idStringTime zone, which must follow the contained value of java.time.ZoneId. For example: Asia/Shanghai
logic-name-prefixStringPrefix pattern of sharding data sources or tables
datetime-lowerStringDatetime sharding lower boundary, pattern is consistent with the timestamp format of yyyy-MM-dd HH:mm:ss
datetime-upperStringDatetime sharding upper boundary, pattern is consistent with the timestamp format of yyyy-MM-dd HH:mm:ss
sharding-suffix-patternStringSuffix pattern of sharding data sources or tables, must can be transformed to Java LocalDateTime, must be consistent with datetime-interval-unit. For example: yyyyMM
datetime-interval-unitStringUnit of sharding value interval, must can be transformed to Java ChronoUnit’s Enum value. For example: MONTHS
datetime-interval-amountintInterval of sharding value, after which the next shard will be entered

Snowflake key-based fixed interval sharding algorithm provided by CosId

Snowflake ID sharding algorithm with fixed time range implemented by tool class based on me.ahoo.cosid:cosid-core. When the sharding key is a JSR-310 containing class or a time-related class, it will be converted to java.time.LocalDateTime before the next sharding. See the discussion at https://github.com/apache/shardingsphere/issues/14047.

Type：COSID_INTERVAL_SNOWFLAKE

Attributes：

NameDataTypeDescriptionDefault Value
zone-idStringTime zone, which must follow the contained value of java.time.ZoneId. For example: Asia/Shanghai
logic-name-prefixStringPrefix pattern of sharding data sources or tables
datetime-lowerStringDatetime sharding lower boundary, pattern is consistent with the timestamp format of yyyy-MM-dd HH:mm:ss
datetime-upperStringDatetime sharding upper boundary, pattern is consistent with the timestamp format of yyyy-MM-dd HH:mm:ss
sharding-suffix-patternStringSuffix pattern of sharding data sources or tables, must can be transformed to Java LocalDateTime, must be consistent with datetime-interval-unit. For example: yyyyMM
datetime-interval-unitStringUnit of sharding value interval, must can be transformed to Java ChronoUnit’s Enum value. For example: MONTHS
datetime-interval-amountintInterval of sharding value, after which the next shard will be entered

Complex Sharding Algorithm

Complex Inline Sharding Algorithm

Please refer to Inline Expression for more details.

Type: COMPLEX_INLINE

NameDataTypeDescriptionDefault Value
sharding-columns (?)Stringsharing column names-
algorithm-expressionStringInline expression sharding algorithm-
allow-range-query-with-inline-sharding (?)booleanWhether range query is allowed. Note: range query will ignore sharding strategy and conduct full routingfalse

Hint Sharding Algorithm

Hint Inline Sharding Algorithm

Please refer to Inline Expression for more details.

Type: COMPLEX_INLINE

NameDataTypeDescriptionDefault Value
algorithm-expressionStringInline expression sharding algorithm${value}

Class Based Sharding Algorithm

Realize custom extension by configuring the sharding strategy type and algorithm class name. CLASS_BASED allows additional custom properties to be passed into the algorithm class. The passed properties can be retrieved through the java.util.Properties class instance with the property name props. Refer to Git’s org.apache.shardingsphere.example.extension.sharding.algortihm.classbased.fixture.ClassBasedStandardShardingAlgorithmFixture.

Type：CLASS_BASED

Attributes：

NameDataTypeDescription
strategyStringSharding strategy type, support STANDARD, COMPLEX or HINT (case insensitive)
algorithmClassNameStringFully qualified name of sharding algorithm

Procedure

  1. When using data sharding, configure the corresponding data sharding algorithm under the shardingAlgorithms attribute.

Sample

  1. rules:
  2. - !SHARDING
  3.   tables:
  4.     t_order: 
  5.       actualDataNodes: ds_${0..1}.t_order_${0..1}
  6.       tableStrategy: 
  7.         standard:
  8.           shardingColumn: order_id
  9.           shardingAlgorithmName: t_order_inline
  10.       keyGenerateStrategy:
  11.         column: order_id
  12.         keyGeneratorName: snowflake
  13.     t_order_item:
  14.       actualDataNodes: ds_${0..1}.t_order_item_${0..1}
  15.       tableStrategy:
  16.         standard:
  17.           shardingColumn: order_id
  18.           shardingAlgorithmName: t_order_item_inline
  19.       keyGenerateStrategy:
  20.         column: order_item_id
  21.         keyGeneratorName: snowflake
  22.     t_account:
  23.       actualDataNodes: ds_${0..1}.t_account_${0..1}
  24.       tableStrategy:
  25.         standard:
  26.           shardingAlgorithmName: t_account_inline
  27.       keyGenerateStrategy:
  28.         column: account_id
  29.         keyGeneratorName: snowflake
  30.   defaultShardingColumn: account_id
  31.   bindingTables:
  32.     - t_order,t_order_item
  33.   broadcastTables:
  34.     - t_address
  35.   defaultDatabaseStrategy:
  36.     standard:
  37.       shardingColumn: user_id
  38.       shardingAlgorithmName: database_inline
  39.   defaultTableStrategy:
  40.     none:
  42.   shardingAlgorithms:
  43.     database_inline:
  44.       type: INLINE
  45.       props:
  46.         algorithm-expression: ds_${user_id % 2}
  47.     t_order_inline:
  48.       type: INLINE
  49.       props:
  50.         algorithm-expression: t_order_${order_id % 2}
  51.     t_order_item_inline:
  52.       type: INLINE
  53.       props:
  54.         algorithm-expression: t_order_item_${order_id % 2}
  55.     t_account_inline:
  56.       type: INLINE
  57.       props:
  58.         algorithm-expression: t_account_${account_id % 2}
  59.   keyGenerators:
  60.     snowflake:
  61.       type: SNOWFLAKE