Table Sharding

The sharding of stored data is essential to any distributed database. This document will describe how table’s data in GreptimeDB is being sharded, and distributed.

Partition

In GreptimeDB, logically, data is sharded in partitions. Because GreptimeDB is using “table” to group data and SQL to query them, we borrow the word “partition”, which is a concept commonly used in OLTP databases.

In GreptimeDB, a table can be horizontally partitioned in multiple ways and it uses the same partitioning types (and corresponding syntax) as in MySQL. Currently, GreptimeDB supports “RANGE COLUMNS partitioning”.

Each partition includes only a portion of the data from the table, and is grouped by some column(s) value range. For example, we can partition a table in GreptimeDB like this:

  1. CREATE TABLE (...)
  2. PARTITION ON COLUMNS (<COLUMN LIST>) (
  3.     <RULE LIST>
  4. );

The syntax mainly consists of two parts:

  • PARTITION ON COLUMNS followed by a comma-separated list of column names, which specifies which columns might be used for partitioning. The partition list specified here is only used as an “allow list”, and in reality only a portion of the columns specified here will be used for partitioning.
  • RULE LIST is a list of multiple partition rules, each of which is a combination of a partition name and a partition condition. The expressions here can use =, !=, >, >=, <, <=, AND, OR, column name and literals.

Here is a concrete example:

  1. CREATE TABLE my_table (
  2.   a INT PRIMARY KEY,
  3.   b STRING,
  4.   ts TIMESTAMP TIME INDEX,
  5. )
  6. PARTITION ON COLUMNS (a) (
  7.   a < 10,
  8.   a >= 10 AND a < 20,
  9.   a >= 20,
  10. );

The above my_table has 3 partitions. The first partition contains rows where “a < 10”, the second partition contains rows where “10 <= a < 20”, and the third partition contains all rows where “a >= 20”.

Table Sharding - 图1Important

  1. The ranges of all partitions must not overlap.
  2. The columns used for partitioning must be specified in ON COLUMNS

Table Sharding - 图2Note

Currently expressions are not supported in “PARTITION BY RANGE” syntax, you can only supply column names.

Region

The data within a table is logically split after creating partitions. You may ask the question “ how are the data, after being logically partitioned, stored in the GreptimeDB? The answer is in “Region“s.

Each region is corresponding to a partition, and stores the data in the partition. The regions are distributed among Datanodes. Our metasrv will move regions among Datanodes automatically, according to the states of Datanodes. Also, metasrv can split or merge regions according to their data volume or access pattern.

The relationship between partition and region can be viewed as the following diagram:

  1.                        ┌───────┐
  2.                        │       │
  3.                        │ Table │
  4.                        │       │
  5.                        └───┬───┘
  6.                            │
  7.         Range [Start, end) │ Horizontally Split Data
  8.         ┌──────────────────┼──────────────────┐
  9.         │                  │                  │
  10.         │                  │                  │
  11.   ┌─────▼─────┐      ┌─────▼─────┐      ┌─────▼─────┐
  12.   │           │      │           │      │           │
  13.   │ Partition │      │ Partition │      │ Partition │
  14.   │           │      │           │      │           │
  15.   │    P0     │      │    P1     │      │    Px     │
  16.   └─────┬─────┘      └─────┬─────┘      └─────┬─────┘
  17.         │                  │                  │
  18.         │                  │                  │  One-to-one mapping of
  19. ┌───────┼──────────────────┼───────┐          │  Partition and Region
  20. │       │                  │       │          │
  21. │ ┌─────▼─────┐      ┌─────▼─────┐ │    ┌─────▼─────┐
  22. │ │           │      │           │ │    │           │
  23. │ │   Region  │      │   Region  │ │    │   Region  │
  24. │ │           │      │           │ │    │           │
  25. │ │     R0    │      │     R1    │ │    │     Ry    │
  26. │ └───────────┘      └───────────┘ │    └───────────┘
  27. │                                  │
  28. └──────────────────────────────────┘
  29.   Could be placed in one Datanode