Compiling with LDB-Toolchain

This topic is about how to compile Doris using the LDB toolchain. This method is an alternative to the Docker method so developers and users without a Docker environment can compile Doris from source. The LDB toolchain version currently recommended by Doris is 0.17, which contains clang-16 and gcc-11.

You can still compile the latest code using the Docker development image: apache/doris:build-env-ldb-toolchain-latest

Special thanks to Amos Bird for the contribution.

Prepare the Environment

This works for most Linux distributions (CentOS, Ubuntu, etc.).

  1. Download ldb_toolchain_gen.sh

    The latest ldb_toolchain_gen.sh can be downloaded from here. This script is used to generate the ldb toolchain.

    For more information, you can visit https://github.com/amosbird/ldb_toolchain_gen

  2. Execute the following command to generate the ldb toolchain.

    1. sh ldb_toolchain_gen.sh /path/to/ldb_toolchain/

    /path/to/ldb_toolchain/ is the directory where the toolchain is installed.

    After execution, the following directory structure will be created under /path/to/ldb_toolchain/.

    1. ├── bin
    2. ├── include
    3. ├── lib
    4. ├── share
    5. ├── test
    6. └── usr

  3. Download and install other compilation packages

    1. Java8
    2. Apache Maven 3.6.3
    3. Node v12.13.0

    Different Linux distributions might contain different packages, so you may need to install additional packages. The following instructions describe how to set up a minimal CentOS 6 box to compile Doris. It should work similarly for other Linux distros.

    1. # install required system packages
    2. sudo yum install -y byacc patch automake libtool make which file ncurses-devel gettext-devel unzip bzip2 zip util-linux wget git python2
    4. # install autoconf-2.69
    5. wget http://ftp.gnu.org/gnu/autoconf/autoconf-2.69.tar.gz && \
    6.     tar zxf autoconf-2.69.tar.gz && \
    7.     cd autoconf-2.69 && \
    8.     ./configure && \
    9.     make && \
    10.     make install
    12. # install bison-3.0.4
    13. wget http://ftp.gnu.org/gnu/bison/bison-3.0.4.tar.gz && \
    14.     tar xzf bison-3.0.4.tar.gz && \
    15.     cd bison-3.0.4 && \
    16.     ./configure && \
    17.     make && \
    18.     make install

  4. Download Doris source code

    1. git clone https://github.com/apache/doris.git

    After downloading, create the custom_env.sh, file under the Doris source directory, and set the PATH environment variable:

    1. export JAVA_HOME=/path/to/java/
    2. export PATH=$JAVA_HOME/bin:$PATH
    3. export PATH=/path/to/maven/bin:$PATH
    4. export PATH=/path/to/node/bin:$PATH
    5. export PATH=/path/to/ldb_toolchain/bin:$PATH

Compile Doris

Enter the Doris source code directory and execute:

  1. $ cat /proc/cpuinfo | grep avx2

Check whether the compilation machine supports the avx2 instruction set.

If it is not supported, use the following command to compile:

  1. $ USE_AVX2=0 sh build.sh

If supported, execute sh build.sh directly.

To build debug version for BE, add BUILD_TYPE=Debug.

  1. $ BUILD_TYPE=Debug sh build.sh

This script will compile the third-party libraries first and then the Doris components (FE, BE) later. The compiled output will be in the output/ directory.

Precompile the Third-Party Binaries

The build.sh script will first compile the third-party dependencies. You can also directly download the precompiled three-party binaries:

https://github.com/apache/doris-thirdparty/releases

Here we provide precompiled third-party binaries for Linux X86(with AVX2) and MacOS(X86 Chip). If it is consistent with your compiling and running environment, you can download and use it directly.

After downloading, you will get an installed/ directory after decompression, copy this directory to the thirdparty/ directory, and then run build.sh.