Concepts Design Doc

This document describes core concepts of ElasticDL.

An ElasticDL model consists of two kinds of parameters:

  1. dense parameters, and
  2. embedding tables.

A dense parameter is a dense tensor with a name. An embedding table is a map from some ID to embedding vectors. An embedding table also has a name. Formalizing the concepts, we have

  • model = {dense parameter} + {embedding table}
  • dense parameter = tensor + name
  • embedding table = {id, tensor} + name

where the curly braces denote zero or more.

To update the model, workers compute and report gradients. Accordingly, we have two kinds of gradients:

  1. dense gradient, and
  2. embedding table gradient

The content of dense gradient is the same as that of the dense parameter. The content of embedding table gradient is the same as that of the embedding table.

On the parameter server, we’d prefer to maintain each embedding table as a map from ID to embedding vectors. With such a data structure, it is efficient to allocate memory for new embedding vectors. On the contrary, we’d concatenate embedding vectors into protobuf messages for parameter pulling and gradient pushing. We cannot use concatenated embedding vectors as the in-memory data structure on the PS, because allocating new embedding vectors involves resize the space of concatenated embedding vectors.

Let’s make a short summary, following is all the core concepts of ElasticDL include:

  • model = {dense parameter} + {embedding table}
  • dense parameter = tensor + name
  • embedding table = tensor + ID + name

Message Representation

There is a tensor proto message defined in TensorFlow, which meets our needs. We could reuse it directly.

We introduce an IndexedSlices proto message to represent the concatenated embedding vectors pulled from PS, and the concatenated embedding vectors of gradient waiting to be pushed to PS.

The definition of elasticdl.proto:

  1. import "tensorflow/tensorflow/core/framework/tensor.proto"
  3. message IndexedSlices {
  4.   tensorflow.Tensor concat_tensors = 1;
  5.   repeated int64 ids = 2;
  6. }
  8. message Model {
  9.   int32 version = 1; // model updated times
  10.   map<string, tensorflow.Tensor> dense_parameters = 2;
  11.   map<string, IndexedSlices> embedding_tables = 3;
  12. }

For in-memory part, we introduce an EmbeddingTable data structure.

  1. type EmbeddingTable struct {
  2.     Name            string
  3.     Dim             int64
  4.     Initializer     string
  5.     EmbeddingVector map[int64]*tensorflow.Tensor
  6. }
  8. type Model struct {
  9.     Version           int32
  10.     InitStatus        bool
  11.     DenseParameters   map[string]*tensorflow.Tensor
  12.     EmbeddingTables   map[string]*EmbeddingTable
  13. }

RPC Service

Following is some auxiliary messages needed by RPC services.

  1. message PullDenseParametersRequest {
  2.   int32 version = 1;
  3. }
  5. message PullDenseParametersResponse {
  6.   bool initialized = 1;
  7.   map<string, tensorflow.Tensor> = 2;
  8. }
  10. message PullEmbeddingTableRequest {
  11.   string name = 1;
  12.   repeated int64 indices = 2;
  13. }
  15. message EmbeddingTableInfo {
  16.   string name = 1;
  17.   int64 dim = 2;
  18.   string initializer = 3;
  19. }
  21. message EmbeddingTableInfos {
  22.   repeated EmbeddingTableInfo embedding_table_infos = 1
  23. }
  25. message PushGradientsResponse {
  26.   bool accepted = 1;
  27.   int32 version = 2;
  28. }

Following is RPC services between PS and worker.

  1. service Pserver {
  2.   rpc pull_dense_parameters(PullDenseParametersRequest) returns (PullDenseParametersResponse);
  3.   rpc pull_embedding_table(PullEmbeddingTableRequest) returns (IndexedSlices);
  4.   rpc push_dense_paramters(Model) returns (google.protobuf.Empty);
  5.   rpc push_embedding_table_infos(EmbeddingTableInfos) returns (google.protobuf.Empty);
  6.   rpc push_gradients(Model) returns (PushGradientsResponse);
  7. }