Apache Doris Be 开发调试

前期准备工作

本教程是在 Ubuntu 20.04 下进行的

  1. 下载 doris 源代码

    下载地址为：apache/incubator-doris: Apache Doris (Incubating) (github.com)Doris BE开发调试环境 — vscode - 图1 (opens new window)

  2. 安装 GCC 8.3.1+，Oracle JDK 1.8+，Python 2.7+，确认 gcc, java, python 命令指向正确版本, 设置 JAVA_HOME 环境变量

  3. 安装其他依赖包

  1. sudo apt install build-essential openjdk-8-jdk maven cmake byacc flex automake libtool-bin bison binutils-dev libiberty-dev zip unzip libncurses5-dev curl git ninja-build python brotli
  2. sudo add-apt-repository ppa:ubuntu-toolchain-r/ppa
  3. sudo apt update
  4. sudo apt install gcc-10 g++-10 
  5. sudo apt-get install autoconf automake libtool autopoint
  1. 安装 openssl-devel
  1. sudo apt install -y openssl-devel

编译

以下操作步骤在 /home/workspace 目录下进行

  1. 下载源码
  1. git clone https://github.com/apache/incubator-doris.git
  1. 编译第三方依赖包
  1.  cd /home/workspace/incubator-doris/thirdparty
  2.  ./build-thirdparty.sh
  1. 编译doris产品代码
  1. cd /home/workspace/incubator-doris
  2. ./build.sh

注意：这个编译有以下几条指令：

  1. ./build.sh  #同时编译be 和fe
  2. ./build.sh  --be #只编译be
  3. ./build.sh  --fe #只编译fe
  4. ./build.sh  --fe --be#同时编译be fe
  5. ./build.sh  --fe --be --clean#删除并同时编译be fe
  6. ./build.sh  --fe  --clean#删除并编译fe
  7. ./build.sh  --be  --clean#删除并编译be
  8. ./build.sh  --be --fe  --clean#删除并同时编译be fe

如果不出意外，应该会编译成功，最终的部署文件将产出到 /home/workspace/incubator-doris/output/ 目录下。如果还遇到其他问题，可以参照 doris 的安装文档 http://doris.apache.org。

部署调试

  1. 给be编译结果文件授权
  1. chmod  /home/workspace/incubator-doris/output/be/lib/palo_be

注意： /home/workspace/incubator-doris/output/be/lib/palo_be为be的执行文件。

  1. 创建数据存放目录

通过查看/home/workspace/incubator-doris/output/be/conf/be.conf

  1. # INFO, WARNING, ERROR, FATAL
  2. sys_log_level = INFO
  3. be_port = 9060
  4. be_rpc_port = 9070
  5. webserver_port = 8040
  6. heartbeat_service_port = 9050
  7. brpc_port = 8060
  9. # Note that there should at most one ip match this list.
  10. # If no ip match this rule, will choose one randomly.
  11. # use CIDR format, e.g. 10.10.10.0/
  12. # Default value is empty.
  13. priority_networks = 192.168.59.0/24 # data root path, seperate by ';'
  14. storage_root_path = /soft/be/storage 
  15. # sys_log_dir = ${PALO_HOME}/log
  16. # sys_log_roll_mode = SIZE-MB-
  17. # sys_log_roll_num =
  18. # sys_log_verbose_modules =
  19. # log_buffer_level = -
  20. # palo_cgroups

需要创建一个文件夹，这是be数据存放的地方

  1. mkdir -p /soft/be/storage

  1. 打开 vscode，并打开 be 源码所在目录，在本案例中打开目录为 /home/workspace/incubator-doris/

  2. 安装 vscode ms c++ 调试插件

Doris BE开发调试环境 — vscode - 图2

  1. 创建 launch.json 文件，文件内容如下：
  1. {
  2.     "version": "0.2.0",
  3.     "configurations": [
  4.         {
  5.             "name": "(gdb) Launch",
  6.             "type": "cppdbg",
  7.             "request": "launch",
  8.             "program": "/home/workspace/incubator-doris/output/be/lib/palo_be",
  9.             "args": [],
  10.             "stopAtEntry": false,
  11.             "cwd": "/home/workspace/incubator-doris/",
  12.             "environment": [{"name":"PALO_HOME","value":"/home/workspace/incubator-doris/output/be/"},
  13.                             {"name":"UDF_RUNTIME_DIR","value":"/home/workspace/incubator-doris/output/be/lib/udf-runtime"},
  14.                             {"name":"LOG_DIR","value":"/home/workspace/incubator-doris/output/be/log"},
  15.                             {"name":"PID_DIR","value":"/home/workspace/incubator-doris/output/be/bin"}
  16.                            ],
  17.             "externalConsole": true,
  18.             "MIMode": "gdb",
  19.             "setupCommands": [
  20.                 {
  21.                     "description": "Enable pretty-printing for gdb",
  22.                     "text": "-enable-pretty-printing",
  23.                     "ignoreFailures": true
  24.                 }
  25.             ]
  26.         }
  27.     ]
  28. }

其中，environment 定义了几个环境变量 DORIS_HOME UDF_RUNTIME_DIR LOG_DIR PID_DIR，这是 palo_be 运行时需要的环境变量，如果没有设置，启动会失败。

注意：如果希望是attach(附加进程）调试，配置代码如下：

  1. {
  2.     "version": "0.2.0",
  3.     "configurations": [
  4.           {
  5.           "name": "(gdb) Launch",
  6.           "type": "cppdbg",
  7.           "request": "attach",
  8.           "program": "/home/workspace/incubator-doris/output/lib/palo_be",
  9.           "processId":,
  10.           "MIMode": "gdb",
  11.           "internalConsoleOptions":"openOnSessionStart",
  12.           "setupCommands": [
  13.                 {
  14.                     "description": "Enable pretty-printing for gdb",
  15.                     "text": "-enable-pretty-printing",
  16.                     "ignoreFailures": true
  17.                 }
  18.             ]
  19.         }
  20.     ]
  21. }

配置中 “request”: “attach”， “processId”:PID，这两个配置是重点： 分别设置 gdb 的调试模式为 attach，附加进程的processId，否则会失败。如何查找进程id，可以在命令行中输入以下命令：

  1. ps -ef | grep palo*

如图：

Doris BE开发调试环境 — vscode - 图3

其中的15200即为当前运行的 be 的进程 id.

一个完整的 lainch.json 的例子如下：

  1.  {
  2.     "version": "0.2.0",
  3.     "configurations": [
  4.         {
  5.             "name": "(gdb) Attach",
  6.             "type": "cppdbg",
  7.             "request": "attach",
  8.             "program": "/home/workspace/incubator-doris/output/be/lib/palo_be",
  9.             "processId": 17016,
  10.             "MIMode": "gdb",
  11.             "setupCommands": [
  12.                 {
  13.                     "description": "Enable pretty-printing for gdb",
  14.                     "text": "-enable-pretty-printing",
  15.                     "ignoreFailures": true
  16.                 }
  17.             ]
  18.         },
  19.         {
  20.             "name": "(gdb) Launch",
  21.             "type": "cppdbg",
  22.             "request": "launch",
  23.             "program": "/home/workspace/incubator-doris/output/be/lib/palo_be",
  24.             "args": [],
  25.             "stopAtEntry": false,
  26.             "cwd": "/home/workspace/incubator-doris/output/be",
  27.             "environment": [
  28.                 {
  29.                     "name": "DORIS_HOME",
  30.                     "value": "/home/workspace/incubator-doris/output/be"
  31.                 },
  32.                 {
  33.                     "name": "UDF_RUNTIME_DIR",
  34.                     "value": "/home/workspace/incubator-doris/output/be/lib/udf-runtime"
  35.                 },
  36.                 {
  37.                     "name": "LOG_DIR",
  38.                     "value": "/home/workspace/incubator-doris/output/be/log"
  39.                 },
  40.                 {
  41.                     "name": "PID_DIR",
  42.                     "value": "/home/workspace/incubator-doris/output/be/bin"
  43.                 }
  44.             ],
  45.             "externalConsole": false,
  46.             "MIMode": "gdb",
  47.             "setupCommands": [
  48.                 {
  49.                     "description": "Enable pretty-printing for gdb",
  50.                     "text": "-enable-pretty-printing",
  51.                     "ignoreFailures": true
  52.                 }
  53.             ]
  54.         }
  55.     ]
  56. }

  1. 点击调试即可

    下面就可以开始你的 Doris DEBUG 之旅了

Doris BE开发调试环境 — vscode - 图4