Secondary indexes

On top of the key value store immudb provides secondary indexes to help developers to handle complex queries.

Sorted sets

The sorted set data type provides simplest secondary index you can create with immudb. That’s a data structure that represents a set of elements ordered by the score of each element, which is a floating point number. The score type is a float64 to accommodate the maximum number of uses cases. 64-bit floating point gives a lot of flexibility and dynamic range, at the expense of having only 53-bits of integer.

When an integer64 is cast to a float there could be a loss of precision, but the insertion order is guaranteed by the internal database index that is appended to the internal index key.

ZAdd can reference an item by key or by index.

ZScan accepts following arguments:

  • Set: the name of the collection
  • SeekKey: initial key for the first entry in the iteration. Optional
  • SeekScore: the min or max score for the first entry in the iteration, depending on Desc value. Optional
  • SeekAtTx: the tx id for the first entry in the iteration. Optional
  • InclusiveSeek: the element resulting from the combination of the SeekKey SeekScore and SeekAtTx is returned with the result. Optional
  • Desc: DESC or ASC sorting order. Optional
  • SinceTx: immudb will wait that the transaction provided by SinceTx be processed. Optional
  • NoWait: when true scan doesn’t wait that txSinceTx is processed. Optional
  • MinScore: minimum score filter. Optional
  • MaxScore: maximum score filter. Optional
  • Limit: maximum number of returned items. Optional

Having the possibility to get data specifying a transaction id: AtTx, it’s the optimal way to retrieve the data, as it can be done with random access to it. And it can be made immediately after the transaction was committed or at any point in the future. When the transaction ID is unknown by the application and the query is made by key or key prefixes, it will be served through the index, depending on the insertion rate, it can be delayed or up to date with inserted data, using a big number in SinceTx with NoWait in true will mean that the query will be resolved by looking at the most recent indexed data, but if your query needs to be resolved after some transactions has been inserted, you can set SinceTx to specify up to which transaction the index has to be made for resolving it.

  1.     i1, err := client.Set(ctx, []byte(`user1`), []byte(`user1@mail.com`))
  2.     if err != nil{
  3.         log.Fatal(err)
  4.     }
  5.     i2, err := client.Set(ctx, []byte(`user2`), []byte(`user2@mail.com`))
  6.     if err != nil{
  7.         log.Fatal(err)
  8.     }
  9.     i3, err := client.Set(ctx, []byte(`user3`), []byte(`user3@mail.com`))
  10.     if err != nil{
  11.         log.Fatal(err)
  12.     }
  13.     i4, err := client.Set(ctx, []byte(`user3`), []byte(`another-user3@mail.com`))
  14.     if err != nil{
  15.         log.Fatal(err)
  16.     }
  18.     if _ , err = client.ZAddAt(ctx,  []byte(`age`), 25, []byte(`user1`), i1.Id); err != nil {
  19.         log.Fatal(err)
  20.     }
  21.     if _ , err = client.ZAddAt(ctx,  []byte(`age`), 36, []byte(`user2`), i2.Id); err != nil {
  22.         log.Fatal(err)
  23.     }
  24.     if _ , err = client.ZAddAt(ctx,  []byte(`age`), 36, []byte(`user3`), i3.Id); err != nil {
  25.         log.Fatal(err)
  26.     }
  27.     if _ , err = client.ZAddAt(ctx,  []byte(`age`), 54, []byte(`user3`), i4.Id); err != nil {
  28.         log.Fatal(err)
  29.     }
  31.     zscanOpts1 := &schema.ZScanRequest{
  32.         Set:     []byte(`age`),
  33.         SinceTx: math.MaxUint64,
  34.         NoWait: true,
  35.         MinScore: &schema.Score{Score: 36},
  36.     }
  38.     the36YearsOldList, err := client.ZScan(ctx, zscanOpts1)
  39.     if err != nil{
  40.         log.Fatal(err)
  41.     }
  42.     s, _ := json.MarshalIndent(the36YearsOldList, "", "\t")
  43.     fmt.Print(string(s))
  45.     oldestReq := &schema.ZScanRequest{
  46.         Set:           []byte(`age`),
  47.         SeekKey:       []byte{0xFF},
  48.         SeekScore:     math.MaxFloat64,
  49.         SeekAtTx:      math.MaxUint64,
  50.         Limit:         1,
  51.         Desc:          true,
  52.         SinceTx:       math.MaxUint64,
  53.         NoWait:        true,
  54.     }
  56.     oldest, err := client.ZScan(ctx, oldestReq)
  57.     if err != nil{
  58.         log.Fatal(err)
  59.     }
  60.     s, _ = json.MarshalIndent(oldest, "", "\t")
  61.     fmt.Print(string(s))
  1. byte[] value1 = {0, 1, 2, 3};
  2. byte[] value2 = {4, 5, 6, 7};
  4. try {
  5.     immuClient.set("zadd1", value1);
  6.     immuClient.set("zadd2", value2);
  7. } catch (CorruptedDataException e) {
  8.     // ...
  9. }
  11. TxMetadata set1TxMd = null;
  12. try {
  13.     immuClient.zAdd("set1", 1, "zadd1");
  14.     set1TxMd = immuClient.zAdd("set1", 2, "zadd2");
  16.     immuClient.zAddAt("set1", 3, "zadd3", set1TxMd.id);
  18.     immuClient.zAdd("set2", 2, "zadd1");
  19.     immuClient.zAdd("set2", 1, "zadd2");
  20. } catch (CorruptedDataException e) {
  21.     // ...
  22. }
  24. List<KV> zScan1 = immuClient.zScan("set1", set1TxMd.id, 5, false);
  25. // We expect two KVs with key names "zadd1" and "zadd2".
  27. List<KV> zScan2 = immuClient.zScan("set2", 5, false);
  28. // Same as before, we expect two KVs with key names "zadd2" and "zadd1".

This feature is not yet supported or not documented. Do you want to make a feature request or help out? Open an issue on Python sdk github projectSecondary indexes - 图1 (opens new window)

  1. import ImmudbClient from 'immudb-node'
  2. import Parameters from 'immudb-node/types/parameters'
  4. const IMMUDB_HOST = '127.0.0.1'
  5. const IMMUDB_PORT = '3322'
  6. const IMMUDB_USER = 'immudb'
  7. const IMMUDB_PWD = 'immudb'
  9. const cl = new ImmudbClient({ host: IMMUDB_HOST, port: IMMUDB_PORT });
  11. (async () => {
  12.     await cl.login({ user: IMMUDB_USER, password: IMMUDB_PWD })
  14.     const { id: id1 } = await cl.set({ key: 'user1', value: 'user1@mail.com' })
  15.     const { id: id2 } = await cl.set({ key: 'user2', value: 'user2@mail.com' })
  16.     const { id: id3 } = await cl.set({ key: 'user3', value: 'user3@mail.com' })
  17.     const { id: id4 } = await cl.set({ key: 'user3', value: 'another-user3@mail.com' })
  19.     const zAddAtReq1: Parameters.ZAddAt = {
  20.         set: 'age',
  21.         score: 25,
  22.         key: 'user1',
  23.         attx: id1
  24.     }
  25.     const zAddAtRes1 = await cl.zAddAt(zAddAtReq1)
  26.     const zAddAtReq2: Parameters.ZAddAt = {
  27.         set: 'age',
  28.         score: 36,
  29.         key: 'user2',
  30.         attx: id2
  31.     }
  32.     const zAddAtRes2 = await cl.zAddAt(zAddAtReq2)
  33.     const zAddAtReq3: Parameters.ZAddAt = {
  34.         set: 'age',
  35.         score: 36,
  36.         key: 'user3',
  37.         attx: id3
  38.     }
  39.     const zAddAtRes3 = await cl.zAddAt(zAddAtReq3)
  40.     const zAddAtReq4: Parameters.ZAddAt = {
  41.         set: 'age',
  42.         score: 54,
  43.         key: 'user4',
  44.         attx: id4
  45.     }
  46.     const zAddAtRes4 = await cl.zAddAt(zAddAtReq4)
  48.     const zScanReq: Parameters.ZScan = {
  49.         set: 'age',
  50.         sincetx: 0,
  51.         nowait: true,
  52.         minscore: {
  53.             score: 36
  54.         }
  55.     }
  56.     const zScanRes = await cl.zScan(zScanReq)
  57.     console.log('success: zScan all 36-years-old users', zScanRes)
  58. })()

This feature is not yet supported or not documented. Do you want to make a feature request or help out? Open an issue on .Net sdk github projectSecondary indexes - 图2 (opens new window)

If you’re using another development language, please read up on our immugwSecondary indexes - 图3 (opens new window) option.