Build Docker Image

This topic is about how to build a running image of Apache Doris through Dockerfile, so that an Apache Doris image can be quickly pulled in a container orchestration tool or during a quick test to complete the cluster creation.

Software and Hardware Requirements

Overview

Prepare the production machine before building a Docker image. The platform architecture of the Docker image will be the same as that of the machine. For example, if you use an X86_64 machine to build a Docker image, you need to download the Doris binary program of X86_64, and the Docker image built can only run on the X86_64 platform. The ARM platform (or M1), likewise.

Hardware Requirements

Minimum configuration: 2C 4G

Recommended configuration: 4C 16G

Software Requirements

Docker Version: 20.10 or newer

Build Docker Image

During Dockerfile scripting, please note that:

  1. Use the official OpenJDK image certified by Docker-Hub as the base parent image (Version: JDK 1.8).
  2. Use the official binary package for download; do not use binary packages from unknown sources.
  3. Use embedded scripts for tasks such as FE startup, multi-FE registration, FE status check, BE startup, registration of BE to FE, and BE status check.
  4. Do not use --daemon to start applications in Docker. Otherwise there will be exceptions during the deployment of orchestration tools such as K8S.

Apache Doris 1.2 and the subsequent versions support JavaUDF, so you also need a JDK environment for BE. The recommended images are as follows:

Doris ProgramRecommended Base Parent Image
Frontendopenjdk:8u342-jdk
Backendopenjdk:8u342-jdk
Brokeropenjdk:8u342-jdk

Script Preparation

In the Dockerfile script for compiling the Docker Image, there are two methods to load the binary package of the Apache Doris program:

  1. Execute the download command via wget / curl when compiling, and then start the docker build process.
  2. Download the binary package to the compilation directory in advance, and then load it into the docker build process through the ADD or COPY command.

Method 1 can produce a smaller Docker image, but if the docker build process fails, the download operation might be repeated and result in longer build time; Method 2 is more suitable for less-than-ideal network environments.

The examples below are based on Method 2. If you prefer to go for Method 1, you may modify the steps accordingly.

Prepare Binary Package

Please noted that if you have a need for custom development, you need to modify the source code, compile and package it, and then place it in the build directory.

If you have no such needs, you can just download the binary package from the official website.

Steps

Build FE

The build environment directory is as follows:

  1. └── docker-build                                       // build root directory
  2.      └── fe                                            // FE build directory
  3.          ├── dockerfile                                // dockerfile script
  4.          └── resource                                  // resource directory
  5.              ├── init_fe.sh                            // startup and registration script
  6.              └── apache-doris-x.x.x-bin-fe.tar.gz      // binary package

  1. Create a build environment directory

    1. mkdir -p ./docker-build/fe/resource

  2. Download official binary package/compiled binary package

    Copy the binary package to the ./docker-build/fe/resource directory

  3. Write the Dockerfile script for FE

    1. # Select the base image
    2. FROM openjdk:8u342-jdk
    4. # Set environment variables
    5. ENV JAVA_HOME="/usr/local/openjdk-8/" \
    6.     PATH="/opt/apache-doris/fe/bin:$PATH"
    8. # Download the software into the image (you can modify based on your own needs)
    9. ADD ./resource/apache-doris-fe-${x.x.x}-bin.tar.gz /opt/
    11. RUN apt-get update && \
    12.     apt-get install -y default-mysql-client && \
    13.     apt-get clean && \
    14.     mkdir /opt/apache-doris && \
    15.     cd /opt && \
    16.     mv apache-doris-fe-${x.x.x}-bin /opt/apache-doris/fe
    18. ADD ./resource/init_fe.sh /opt/apache-doris/fe/bin
    19. RUN chmod 755 /opt/apache-doris/fe/bin/init_fe.sh
    21. ENTRYPOINT ["/opt/apache-doris/fe/bin/init_fe.sh"]

    After writing, name it Dockerfile and save it to the ./docker-build/fe directory.

  4. Write the execution script of FE

    You can refer to init_fe.sh.

    After writing, name it init_fe.sh and save it to the ./docker-build/fe/resouce directory.

  5. Execute the build

    Please note that ${tagName} needs to be replaced with the tag name you want to package and name, such as: apache-doris:1.1.3-fe

    Build FE:

    1. cd ./docker-build/fe
    2. docker build . -t ${fe-tagName}

Build BE

  1. Create a build environment directory
  1. mkdir -p ./docker-build/be/resource

  1. The build environment directory is as follows:

    1. └── docker-build                                            // build root directory
    2.     └── be                                                  // BE build directory
    3.         ├── dockerfile                                      // dockerfile script
    4.         └── resource                                        // resource directory
    5.             ├── init_be.sh                                  // startup and registration script
    6.             └── apache-doris-x.x.x-bin-x86_64/arm-be.tar.gz // binary package

  2. Write the Dockerfile script for BE

    1. # Select the base image
    2. FROM openjdk:8u342-jdk
    4. # Set environment variables
    5. ENV JAVA_HOME="/usr/local/openjdk-8/" \
    6.     PATH="/opt/apache-doris/be/bin:$PATH"
    8. # Download the software into the image (you can modify based on your own needs)
    9. ADD ./resource/apache-doris-be-${x.x.x}-bin-x86_64.tar.gz /opt/
    11. RUN apt-get update && \
    12.     apt-get install -y default-mysql-client && \
    13.     apt-get clean && \
    14.     mkdir /opt/apache-doris && \
    15.     cd /opt && \
    16.     mv apache-doris-be-${x.x.x}-bin-x86_64 /opt/apache-doris/be
    18. ADD ./resource/init_be.sh /opt/apache-doris/be/bin
    19. RUN chmod 755 /opt/apache-doris/be/bin/init_be.sh
    21. ENTRYPOINT ["/opt/apache-doris/be/bin/init_be.sh"]

    After writing, name it Dockerfile and save it to the ./docker-build/be directory

  3. Write the execution script of BE

    You can refer to init_be.sh.

    After writing, name it init_be.sh and save it to the ./docker-build/be/resouce directory.

  4. Execute the build

    Please note that ${tagName} needs to be replaced with the tag name you want to package and name, such as: apache-doris:1.1.3-be

    Build BE:

    1. cd ./docker-build/be
    2. docker build . -t ${be-tagName}

    After the build process is completed, you will see the prompt Success. Then, you can check the built image using the following command.

    1. docker images

Build Broker

  1. Create a build environment directory
  1. mkdir -p ./docker-build/broker/resource

  1. The build environment directory is as follows:

    1. └── docker-build                                     // build root directory
    2.     └── broker                                       // BROKER build directory
    3.         ├── dockerfile                               // dockerfile script
    4.         └── resource                                 // resource directory
    5.             ├── init_broker.sh                       // startup and registration script
    6.             └── apache-doris-x.x.x-bin-broker.tar.gz // binary package

  2. Write the Dockerfile script for Broker

    1. # Select the base image
    2. FROM openjdk:8u342-jdk
    4. # Set environment variables
    5. ENV JAVA_HOME="/usr/local/openjdk-8/" \
    6.     PATH="/opt/apache-doris/broker/bin:$PATH"
    8. # Download the software into the image, where the broker directory is synchronously compressed to the binary package of FE, which needs to be decompressed and repackaged (you can modify based on your own needs)
    9. ADD ./resource/apache_hdfs_broker.tar.gz /opt/
    11. RUN apt-get update && \
    12.     apt-get install -y default-mysql-client && \
    13.     apt-get clean && \
    14.     mkdir /opt/apache-doris && \
    15.     cd /opt && \
    16.     mv apache_hdfs_broker /opt/apache-doris/broker
    18. ADD ./resource/init_broker.sh /opt/apache-doris/broker/bin
    19. RUN chmod 755 /opt/apache-doris/broker/bin/init_broker.sh
    21. ENTRYPOINT ["/opt/apache-doris/broker/bin/init_broker.sh"]

    After writing, name it Dockerfile and save it to the ./docker-build/broker directory

  3. Write the execution script of BE

    You can refer to init_broker.sh.

    After writing, name it init_broker.sh and save it to the ./docker-build/broker/resouce directory.

  4. Execute the build

    Please note that ${tagName} needs to be replaced with the tag name you want to package and name, such as: apache-doris:1.1.3-broker

    Build Broker:

    1. cd ./docker-build/broker
    2. docker build . -t ${broker-tagName}

    After the build process is completed, you will see the prompt Success. Then, you can check the built image using the following command.

    1. docker images

Push Image to DockerHub or Private Warehouse

Log into your DockerHub account

  1. docker login

If the login succeeds, you will see the prompt Success , and then you can push the Docker image to the warehouse.

  1. docker push ${tagName}