ArangoDB Starter Architecture

What does the Starter do

The ArangoDB Starter is a program used to create ArangoDB database deploymentson bare-metal (or virtual machines) with ease.It enables you to create everything from a simple Single server instanceto a full blown Cluster with datacenter to datacenter replication in under 5 minutes.

The Starter is intended to be used in environments where there is no higherlevel orchestration system (e.g. Kubernetes) available.

Starter versions

The Starter is a separate process in a binary called arangodb (or arangodb.exe on Windows).This binary has its own version number that is independent of a ArangoDB (database)version.

This means that Starter version a.b.c can be used to run deploymentsof ArangoDB databases with different version.For example, the Starter with version 0.11.2 can be used to createArangoDB deployments with ArangoDB version 3.2.<something> as wellas deployments with ArangoDB version 3.3.<something>.

It also means that you can update the Starter independently from the ArangoDBdatabase.

Note that the Starter is also included in all binary ArangoDB packages.

To find the versions of you Starters & ArangoDB database, run the following commands:

  1. # To get the Starter version
  2. arangodb --version
  3. # To get the ArangoDB database version
  4. arangod --version

Starter deployment modes

The Starter supports 3 different modes of ArangoDB deployments:

  • Single server
  • Active failover
  • ClusterNote: Datacenter replication is an option for the cluster deployment mode.

You select one of these modes using the —starter.mode command line option.

Depending on the mode you’ve selected, the Starter launches one or more(arangod / arangosync) server processes.

No matter which mode you select, the Starter always provides youa common directory structure for storing the servers data, configuration & log files.

Starter operating modes

The Starter can run as normal processes directly on the host operating system,or as containers in a docker runtime.

When running as normal process directly on the host operating system,the Starter launches the servers as child processes and monitors those.If one of the server processes terminates, a new one is started automatically.

When running in a docker container, the Starter launches the serversas separate docker containers, that share the volume namespace withthe container that runs the Starter. It monitors those containersand if one terminates, a new container is launched automatically.

Starter data-directory

The Starter uses a single directory with a well known structure to storeall data for its own configuration & logs, as well as the configuration,data & logs of all servers it starts.

This data directory is set using the —starter.data-dir command line option.It contains the following files & sub-directories.

  • setup.json The configuration of the “cluster of Starters”.For details see below. DO NOT edit this file.
  • arangodb.log The log file of the Starter
  • single<port>, agent<port>, coordinator<port>, dbserver<port>: directories forlaunched servers. These directories contain among others the following files:
    • apps: A directory with Foxx applications
    • data: A directory with database data
    • arangod.conf: The configuration file for the server. Editing this file is possible, but not recommended.
    • arangod.log: The log file of the server
    • arangod_command.txt: File containing the exact command line of the started server (for debugging purposes only)

Running on multiple machines

For the activefailover & cluster mode, it is required to run multipleStarters, as every Starter will only launch a subset of all servers neededto form the entire deployment.For example in cluster mode, a Starter will launch a single Agent, a single DB-Serverand a single Coordinator.

It is the responsibility of the user to run the Starter on multiple machines suchthat enough servers are started to form the entire deployment.The minimum number of Starters needed is 3.

The Starters running on those machines need to know about each other’s existence.In order to do so, the Starters form a “cluster” of their own (not to be confusedwith the ArangoDB database cluster).This cluster of Starters is formed from the values given to the —starter.joincommand line option. You should pass the addresses (<host>:<port>) of all Starters.

For example a typical commandline for a cluster deployment looks like this:

  1. arangodb --starter.mode=cluster --starter.join=hostA:8528,hostB:8528,hostC:8528
  2. # this command is run on hostA, hostB and hostC.

The state of the cluster (of Starters) is stored in a configuration file calledsetup.json in the data directory of every Starter and the ArangoDBAgency is used to elect a master among all Starters.

The master Starter is responsible for maintaining the list of all Startersinvolved in the cluster and their addresses. The slave Starters (all Startersexcept the elected master) fetch this list from the master Starter on regularbasis and store it to its own setup.json config file.

Note: The setup.json config file MUST NOT be edited manually.

Running on multiple machines (under the hood)

As mentioned above, when the Starter is used to create an activefailoveror cluster deployment, it first creates a “cluster” of Starters.

These are the steps taken by the Starters to bootstrap such a deploymentfrom scratch.

  • All Starters are started (either manually or by some supervisor)
  • All Starters try to read their config from setup.json.If that file exists and is valid, this bootstrap-from-scratch processis aborted and all Starters go directly to the running phase described below.
  • All Starters create a unique ID
  • The list of —starter.join arguments is sorted
  • All Starters request the unique ID from the first server in the sorted —starter.join list,and compares the result with its own unique ID.
  • The Starter that finds its own unique ID, is continuing as bootstrap masterthe other Starters are continuing as bootstrap slaves.
  • The bootstrap master waits for at least 2 bootstrap slaves to join it.
  • The bootstrap slaves contact the bootstrap master to join its cluster of Starters.
  • Once the bootstrap master has received enough (at least 2) requeststo join its cluster of Starters, it continues with the running phase.
  • The bootstrap slaves keep asking the bootstrap master about its state.As soon as they receive confirmation to do so, they also continue with the running phase.In the running phase all Starters launch the desired servers and keeps monitoring thoseservers. Once a functional Agency is detected, all Starters will try to berunning master by trying to write their ID in a well known location in the Agency.The first Starter to succeed in doing so wins this master election.

The running master will keep writing its ID in the Agency in order to remainingthe running master. Since this ID is written with a short time-to-live,other Starters are able to detect when the current running master has been stoppedor is no longer responsible. In that case the remaining Starters will performanother master election to decide who will be the next running master.

API requests that involve the state of the cluster of Starters are always answeredby the current running master. All other Starters will refer the request tothe current running master.