Deploying a complete Doris Compute-storage decoupled cluster from scratch on K8s is divided into two parts: the first part deploys Doris-Operator; the second part distributes the relevant resources DorisDisaggregatedMetaService and DorisDisaggregatedCluster for deploying the Compute-storage decoupled cluster. The DorisDisaggregatedMetaService resource deploys the metadata component; the DorisDisaggregatedCluster resource deploys the SQL parsing component and the computing component.

Install Operator

  1. Create CRD：
  1. kubectl create -f https://raw.githubusercontent.com/selectdb/doris-operator/master/config/crd/bases/crds.yaml

Expected Results：

  1. customresourcedefinition.apiextensions.k8s.io/foundationdbclusters.apps.foundationdb.org created
  2. customresourcedefinition.apiextensions.k8s.io/foundationdbbackups.apps.foundationdb.org created
  3. customresourcedefinition.apiextensions.k8s.io/foundationdbrestores.apps.foundationdb.org created
  4. customresourcedefinition.apiextensions.k8s.io/dorisdisaggregatedclusters.disaggregated.cluster.doris.com created
  5. customresourcedefinition.apiextensions.k8s.io/dorisdisaggregatedmetaservices.disaggregated.metaservice.doris.com created
  1. Deploy Doris-Operator and dependent RBAC rules:
  1. kubectl apply -f https://raw.githubusercontent.com/selectdb/doris-operator/master/config/operator/disaggregated-operator.yaml

Expected Results：

  1. kubectl -n doris get pods
  2. NAME                                         READY   STATUS    RESTARTS   AGE
  3. doris-operator-fdb-manager-d75574c47-b2sqx   1/1     Running   0          11s
  4. doris-operator-5b667b4954-d674k              1/1     Running   0          11s

Compute-storage decoupled cluster quick start

Install DorisDisaggregatedMetaService

  1. Download ddm-sample.yaml, a resource that tells Doris-Operator how to deploy metadata components.
  1. curl -O https://raw.githubusercontent.com/selectdb/doris-operator/master/doc/examples/disaggregated/metaservice/ddm-sample.yaml

Deploy resources according to configuration metadata chapters: FDB, ms, recycler Configure resources according to actual needs.

  1. Apply DorisDisaggregatedMetaService
  1. kubectl apply -f ddm-sample.yaml

Expected Results：

  1. kubectl get ddm
  2. NAME                   FDBSTATUS   MSSTATUS   RECYCLERSTATUS
  3. meta-service-release   Available   Ready      Ready

Config object storage

The Compute-storage decoupled cluster uses object storage as persistent storage. You need to use ConfigMap to deliver [object storage information used by the Compute-storage decoupled cluster](install-cluster/config-relation#Register object storage).

  1. Download the ConfigMap resource that contains the object storage information:

Compute-storage decoupled cluster uses object storage as the backend storage, and needs to plan the object storage to be used in advance. Download object-store-info.yaml.

  1. curl -O https://raw.githubusercontent.com/selectdb/doris-operator/master/doc/examples/disaggregated/cluster/object-store-info.yaml
  1. According to the Doris Compute-storage decoupled_interface format, configure the object storage information in JSON format, with instance.conf as the key and the object storage information in JSON format as the value configured in the data of ConfigMap. (Replace the corresponding value in the sample JSON format)

apply object-store-info.yaml :

  1. kubectl apply -f object-store-info.yaml

Compute-storage decoupled quick start - 图1Tip

  • To deploy a Compute-storage decoupled cluster, you need to plan the object storage to be used in advance, and configure the object storage information through ConfigMap to the Namespace where the doris Compute-storage decoupled cluster needs to be deployed.
  • The configuration in the case is mainly to show the basic configuration information required for object storage. All values are fictitious and cannot be used in real scenarios. If you need to build a real and available cluster, please fill in the real data.

Install DorisDisaggregatedCluster

  1. Download ddc-sample.yaml:
  1. curl -O https://raw.githubusercontent.com/selectdb/doris-operator/master/doc/examples/disaggregated/cluster/ddc-sample.yaml
  1. According to the chapters on configuring Compute-storage decoupled cluster: Cluster association configuration, fe, compute cluster, configure resources according to actual needs. Use the following command to deploy resources:
  1. kubectl apply -f ddc-sample.yaml

The expected results of deploying a computing cluster are as follows:

  1. kubectl get ddc
  2. NAME                         CLUSTERHEALTH   FEPHASE   CGCOUNT   CGAVAILABLECOUNT   CGFULLAVAILABLECOUNT
  3. test-disaggregated-cluster   green           Ready     1         1                  1

Compute-storage decoupled quick start - 图2Tip

  • DorisDisaggregatedCluster must be configured to use DorisDisaggregatedMetaService resource information in the specification.
  • To deploy a Compute-storage decoupled cluster, you must plan the object storage you want to use in advance and configure the deployment according to the object storage configuration section in the cluster association configuration.