The etcd v3 API is designed to give users a more efficient and cleaner abstraction compared to etcd v2. There are a number of semantic and protocol changes in this new API. For an overview see Xiang Li’s video.

To prove out the design of the v3 API the team has also built a number of example recipes, there is a video discussing these recipes too.

Design

  1. Flatten binary key-value space

  2. Keep the event history until compaction

    • access to old version of keys
    • user controlled history compaction

  3. Support range query

    • Pagination support with limit argument
    • Support consistency guarantee across multiple range queries

  4. Replace TTL key with Lease

    • more efficient/ low cost keep alive
    • a logical group of TTL keys

  5. Replace CAS/CAD with multi-object Txn

    • MUCH MORE powerful and flexible

  6. Support efficient watching with multiple ranges

  7. RPC API supports the completed set of APIs.

    • more efficient than JSON/HTTP
    • additional txn/lease support

  8. HTTP API supports a subset of APIs.

    • easy for people to try out etcd
    • easy for people to write simple etcd application

Notes

Request Size Limitation

The max request size is around 1MB. Since etcd replicates requests in a streaming fashion, a very large request might block other requests for a long time. The use case for etcd is to store small configuration values, so we prevent user from submitting large requests. This also applies to Txn requests. We might loosen the size in the future a little bit or make it configurable.

Protobuf Defined API

api protobuf

kv protobuf

Examples

Put a key (foo=bar)

  1. // A put is always successful
  2. Put( PutRequest { key = foo, value = bar } )
  4. PutResponse { 
  5.     cluster_id = 0x1000,
  6.     member_id = 0x1,
  7.     revision = 1,
  8.     raft_term = 0x1,
  9. }

Get a key (assume we have foo=bar)

  1. Get ( RangeRequest { key = foo } )
  3. RangeResponse {
  4.     cluster_id = 0x1000,
  5.     member_id = 0x1,
  6.     revision = 1,
  7.     raft_term = 0x1,
  8.     kvs = {
  9.       {
  10.           key = foo,
  11.           value = bar,
  12.           create_revision = 1,
  13.           mod_revision = 1,
  14.           version = 1;
  15.       },
  16.     },
  17. }

Range over a key space (assume we have foo0=bar0… foo100=bar100)

  1. Range ( RangeRequest { key = foo, end_key = foo80, limit = 30  } )
  3. RangeResponse {
  4.     cluster_id = 0x1000,
  5.     member_id = 0x1,
  6.     revision = 100,
  7.     raft_term = 0x1,
  8.     kvs = {
  9.       {
  10.           key = foo0,
  11.           value = bar0,
  12.           create_revision = 1,
  13.           mod_revision = 1,
  14.           version = 1;
  15.       },
  16.          ...,
  17.       {
  18.           key = foo30,
  19.           value = bar30,
  20.           create_revision = 30,
  21.           mod_revision = 30,
  22.           version = 1;
  23.       },
  24.     },
  25. }

Finish a txn (assume we have foo0=bar0, foo1=bar1)

  1. Txn(TxnRequest {
  2.     // mod_revision of foo0 is equal to 1, mod_revision of foo1 is greater than 1
  3.     compare = {
  4.         {compareType = equal, key = foo0, mod_revision = 1}, 
  5.         {compareType = greater, key = foo1, mod_revision = 1}}
  6.     },
  7.     // if the comparison succeeds, put foo2 = bar2
  8.     success = {PutRequest { key = foo2, value = success }},
  9.     // if the comparison fails, put foo2=fail
  10.     failure = {PutRequest { key = foo2, value = failure }},
  11. )
  13. TxnResponse {
  14.     cluster_id = 0x1000,
  15.     member_id = 0x1,
  16.     revision = 3,
  17.     raft_term = 0x1,
  18.     succeeded = true,
  19.     responses = {
  20.       // response of PUT foo2=success
  21.       {
  22.             cluster_id = 0x1000,
  23.             member_id = 0x1,
  24.             revision = 3,
  25.             raft_term = 0x1,
  26.         }
  27.     }
  28. }

Watch on a key/range

  1. Watch( WatchRequest{
  2.            key = foo,
  3.            end_key = fop, // prefix foo
  4.            start_revision = 20,
  5.            end_revision = 10000,
  6.            // server decided notification frequency
  7.            progress_notification = true,
  8.        } 
  9.        … // this can be a watch request stream
  10.       )
  12. // put (foo0=bar0) event at 3
  13. WatchResponse {
  14.     cluster_id = 0x1000,
  15.     member_id = 0x1,
  16.     revision = 3,
  17.     raft_term = 0x1,
  18.     event_type = put,
  19.     kv = {
  20.               key = foo0,
  21.               value = bar0,
  22.               create_revision = 1,
  23.               mod_revision = 1,
  24.               version = 1;
  25.           },
  26.     }
  27.     …
  29.     // a notification at 2000
  30.     WatchResponse {
  31.         cluster_id = 0x1000,
  32.         member_id = 0x1,
  33.         revision = 2000,
  34.         raft_term = 0x1,
  35.         // nil event as notification
  36.     }
  38.     … 
  40.     // put (foo0=bar3000) event at 3000
  41.     WatchResponse {
  42.         cluster_id = 0x1000,
  43.         member_id = 0x1,
  44.         revision = 3000,
  45.         raft_term = 0x1,
  46.         event_type = put,
  47.         kv = {
  48.                 key = foo0,
  49.                 value = bar3000,
  50.                 create_revision = 1,
  51.                 mod_revision = 3000,
  52.                 version = 2;
  53.           },
  54.     }
  55.     …