安装使用

编译

环境依赖

  1. CentOS、ubuntu和Mac OS都支持（推荐CentOS >= 7.2）

  2. go >= 1.19

  3. gcc >= 5（使用scann模型时，gcc >= 9）

  4. cmake >= 3.17

  5. OpenBLAS

  6. tbb，CentOS可使用yum安装，如：yum install tbb-devel.x86_64

  7. [RocksDB](https://github.com/facebook/rocksdb) == 6.2.2 （可选），你不需要手动安装，脚本自动安装。但是你需要手动安装rocksdb的依赖。请参考如下安装方法：https://github.com/facebook/rocksdb/blob/master/INSTALL.md

  8. CUDA >= 9.0，如果你不使用GPU模型，可忽略。

  9. clang >= 8.0，bazel, python >= 3.7,如果你不使用scann模型，可忽略。

编译

  • 进入 GOPATH 目录, cd $GOPATH/src mkdir -p github.com/vearch cd github.com/vearch

  • 下载源代码: git clone https://github.com/vearch/vearch.git (后续使用$vearch 代表vearch目录绝对路径)

  • 如果使用GPU版本, 修改$vearch/engine/CMakeLists.txt文件中BUILD_WITH_GPU变为on

  • 如果使用SCANN模型, 修改$vearch/engine/CMakeLists.txt文件中BUILD_WITH_SCANN变为on

  • 编译vearch和gamma

    1. cd build

    2. sh build.sh

    生成vearch文件表示编译成功

部署

单机模式:

  • 生成配置文件config.toml(master_server端口使用8817, router_server端口使用9001）
  1. [global]
  2.     # the name will validate join cluster by same name
  3.     name = "vearch"
  4.     # you data save to disk path ,If you are in a production environment, You'd better set absolute paths
  5.     data = ["datas/"]
  6.     # log path , If you are in a production environment, You'd better set absolute paths
  7.     log = "logs/"
  8.     # default log type for any model
  9.     level = "info"
  10.     # master <-> ps <-> router will use this key to send or receive data
  11.     signkey = "vearch"
  12.     skip_auth = true
  14. # if you are master, you'd better set all config for router、ps and router, ps use default config it so cool
  15. [[masters]]
  16.     # name machine name for cluster
  17.     name = "m1"
  18.     # ip or domain
  19.     address = "127.0.0.1"
  20.     # api port for http server
  21.     api_port = 8817
  22.     # port for etcd server
  23.     etcd_port = 2378
  24.     # listen_peer_urls List of comma separated URLs to listen on for peer traffic.
  25.     # advertise_peer_urls List of this member's peer URLs to advertise to the rest of the cluster. The URLs needed to be a comma-separated list.
  26.     etcd_peer_port = 2390
  27.     # List of this member's client URLs to advertise to the public.
  28.     # The URLs needed to be a comma-separated list.
  29.     # advertise_client_urls AND listen_client_urls
  30.     etcd_client_port = 2370
  32. [router]
  33.     # port for server
  34.     port = 9001
  36. [ps]
  37.     # port for server
  38.     rpc_port = 8081
  39.     # raft config begin
  40.     raft_heartbeat_port = 8898
  41.     raft_replicate_port = 8899
  42.     heartbeat-interval = 200 #ms
  43.     raft_retain_logs = 10000
  44.     raft_replica_concurrency = 1
  45.     raft_snap_concurrency = 1
  • 启动

启动vearch前，需要设置LD_LIBRARY_PATH环境变量的值，添加gamma, rocksdb, zfp等依赖的lib包; 执行ldd vearch 和 ldd $vearch/ps/engine/gammacb/lib/lib/libgamma.so.0.1 命令可以查看vearch和gamma依赖的包)。

  1. ./vearch -conf conf.toml all

集群模式:

  • vearch 有三个模块: ps, master, router, run ./vearch -conf config.toml ps/router/master 启动相应模块

假如有5台机器， 2台作为master管理， 2台作为ps计算节点， 1台router请求转发

  • master

    • 192.168.1.1

    • 192.168.1.2

  • ps

    • 192.168.1.3

    • 192.168.1.4

  • router

    • 192.168.1.5

  • 生成toml格式配置文件 config.toml， 作为master的机器ip配置在[[masters]]中，支持多个，router和ps所在机器ip无需配置。

  1. [global]
  2.     name = "vearch"
  3.     data = ["datas/"]
  4.     log = "logs/"
  5.     level = "info"
  6.     signkey = "vearch"
  7.     skip_auth = true
  9. # if you are master, you'd better set all config for router、ps and router, ps use default config it so cool
  10. [[masters]]
  11.     name = "m1"
  12.     address = "192.168.1.1"
  13.     api_port = 8817
  14.     etcd_port = 2378
  15.     etcd_peer_port = 2390
  16.     etcd_client_port = 2370
  17. [[masters]]
  18.     name = "m2"
  19.     address = "192.168.1.2"
  20.     api_port = 8817
  21.     etcd_port = 2378
  22.     etcd_peer_port = 2390
  23.     etcd_client_port = 2370
  24. [router]
  25.     port = 9001
  26.     skip_auth = true
  27. [ps]
  28.     rpc_port = 8081
  29.     raft_heartbeat_port = 8898
  30.     raft_replicate_port = 8899
  31.     heartbeat-interval = 200 #ms
  32.     raft_retain_logs = 10000
  33.     raft_replica_concurrency = 1
  34.     raft_snap_concurrency = 1

  • 启动vearch前，设置LD_LIBRARY_PATH环境变量加载依赖包

  • on 192.168.1.1 , 192.168.1.2 run master

  1. ./vearch -conf config.toml master
  • on 192.168.1.3 , 192.168.1.4 run ps
  1. ./vearch -conf config.toml ps
  • on 192.168.1.5 run router
  1. ./vearch -conf config.toml router