Cluster Deployment

Cluster deployment is to deploy the DolphinScheduler on multiple machines for running a large number of tasks in production.

If you are a green hand and want to experience DolphinScheduler, we recommended you install follow Standalone. If you want to experience more complete functions or schedule large tasks number, we recommended you install follow pseudo-cluster deployment. If you want to using DolphinScheduler in production, we recommended you follow cluster deployment or kubernetes

Deployment Step

Cluster deployment uses the same scripts and configuration files as we deploy in pseudo-cluster deployment, so the prepare and required are the same as pseudo-cluster deployment. The difference is that pseudo-cluster deployment is for one machine, while cluster deployment (Cluster) for multiple. and the steps of “Modify configuration” are quite different between pseudo-cluster deployment and cluster deployment.

Prepare && DolphinScheduler startup environment

Because of cluster deployment for multiple machine, so you have to run you “Prepare” and “startup” in every machine in pseudo-cluster.md, except section “Configure machine SSH password-free login”, “Start zookeeper”, “Initialize the database”, which is only for deployment or just need an single server

Modify configuration

This is a step that is quite different from pseudo-cluster.md, because the deployment script will transfer the resources required for installation machine to each deployment machine using scp. And we have to declare all machine we want to install DolphinScheduler and then run script install.sh. The configuration file is under the path conf/config/install_config.conf, here we only need to modify section INSTALL MACHINE, DolphinScheduler ENV, Database, Registry Server and keep other same as [pseudo-cluster deployment](pseudo-cluster .md), the following describes the parameters that must be modified

  1. # ---------------------------------------------------------
  2. # INSTALL MACHINE
  3. # ---------------------------------------------------------
  4. # Using IP or machine hostname for server going to deploy master, worker, API server, the IP of the server
  5. # If you using hostname, make sure machine could connect each others by hostname
  6. # As below, the hostname of the machine deploying DolphinScheduler is ds1, ds2, ds3, ds4, ds5, where ds1, ds2 install master server, ds3, ds4, and ds5 installs worker server, the alert server is installed in ds4, and the api server is installed in ds5
  7. ips="ds1,ds2,ds3,ds4,ds5"
  8. masters="ds1,ds2"
  9. workers="ds3:default,ds4:default,ds5:default"
  10. alertServer="ds4"
  11. apiServers="ds5"
  12. pythonGatewayServers="ds5"

Start DolphinScheduler && Login DolphinScheduler && Server Start And Stop

Same as pseudo-cluster.md](http://pseudo-cluster.md)\)