Sharding plugin using SQL parser and replace for splits large tables into smaller ones, redirects Query into sharding tables. Give you a high performance database access.

https://github.com/go-gorm/sharding

Features

  • Non-intrusive design. Load the plugin, specify the config, and all done.
  • Lighting-fast. No network based middlewares, as fast as Go.
  • Multiple database support. PostgreSQL tested, MySQL and SQLite is coming.
  • Allows you custom the Primary Key generator (Built in keygen, Sequence, Snowflake …).

Usage

Config the sharding middleware, register the tables which you want to shard. See Godoc for config details.

  1. import (
  2.   "fmt"
  4.   "gorm.io/driver/postgres"
  5.   "gorm.io/gorm"
  6.   "gorm.io/sharding"
  7. )
  9. dsn := "postgres://localhost:5432/sharding-db?sslmode=disable"
  10. db, err := gorm.Open(postgres.New(postgres.Config{DSN: dsn}))
  12. db.Use(sharding.Register(sharding.Config{
  13.     ShardingKey:         "user_id",
  14.     NumberOfShards:      64,
  15.     PrimaryKeyGenerator: sharding.PKSnowflake,
  16. }, "orders").Register(sharding.Config{
  17.     ShardingKey:         "user_id",
  18.     NumberOfShards:      256,
  19.     PrimaryKeyGenerator: sharding.PKSnowflake,
  20.     // This case for show up give notifications, audit_logs table use same sharding rule.
  21. }, Notification{}, AuditLog{}))

Use the db session as usual. Just note that the query should have the Sharding Key when operate sharding tables.

  1. // Gorm create example, this will insert to orders_02
  2. db.Create(&Order{UserID: 2})
  3. // sql: INSERT INTO orders_2 ...
  5. // Show have use Raw SQL to insert, this will insert into orders_03
  6. db.Exec("INSERT INTO orders(user_id) VALUES(?)", int64(3))
  8. // This will throw ErrMissingShardingKey error, because there not have sharding key presented.
  9. db.Create(&Order{Amount: 10, ProductID: 100})
  10. fmt.Println(err)
  12. // Find, this will redirect query to orders_02
  13. var orders []Order
  14. db.Model(&Order{}).Where("user_id", int64(2)).Find(&orders)
  15. fmt.Printf("%#v\n", orders)
  17. // Raw SQL also supported
  18. db.Raw("SELECT * FROM orders WHERE user_id = ?", int64(3)).Scan(&orders)
  19. fmt.Printf("%#v\n", orders)
  21. // This will throw ErrMissingShardingKey error, because WHERE conditions not included sharding key
  22. err = db.Model(&Order{}).Where("product_id", "1").Find(&orders).Error
  23. fmt.Println(err)
  25. // Update and Delete are similar to create and query
  26. db.Exec("UPDATE orders SET product_id = ? WHERE user_id = ?", 2, int64(3))
  27. err = db.Exec("DELETE FROM orders WHERE product_id = 3").Error
  28. fmt.Println(err) // ErrMissingShardingKey

The full example is here.