Monitoring datacenter to datacenter replication

Datacenter to datacenter replication is only available in the Enterprise Edition.

This section includes information related to the monitoring of the datacenter to datacenter replication.

For a general introduction to the datacenter to datacenter replication, please refer to the Datacenter to datacenter replication chapter.

Metrics

ArangoSync (master & worker) provide metrics that can be used for monitoring the datacenter to datacenter replication solution. These metrics are available using the following HTTPS endpoints:

  • GET /metrics: Provides metrics in a format supported by Prometheus.
  • GET /metrics.json: Provides the same metrics in JSON format.

Both endpoints include help information per metrics.

Note: Both endpoints require authentication. Besides the usual authentication methods these endpoints are also accessible using a special bearer token specified using the --monitoring.token command line option.

The Prometheus output (/metrics) looks like this:

  1. ...
  2. # HELP arangosync_master_worker_registrations Total number of registrations
  3. # TYPE arangosync_master_worker_registrations counter
  4. arangosync_master_worker_registrations 2
  5. # HELP arangosync_master_worker_storage Number of times worker info is stored, loaded
  6. # TYPE arangosync_master_worker_storage counter
  7. arangosync_master_worker_storage{kind="",op="save",result="success"} 20
  8. arangosync_master_worker_storage{kind="empty",op="load",result="success"} 1
  9. ...

The JSON output (/metrics.json) looks like this:

  1. {
  2.   ...
  3.   "arangosync_master_worker_registrations": {
  4.     "help": "Total number of registrations",
  5.     "type": "counter",
  6.     "samples": [
  7.       {
  8.         "value": 2
  9.       }
  10.     ]
  11.   },
  12.   "arangosync_master_worker_storage": {
  13.     "help": "Number of times worker info is stored, loaded",
  14.     "type": "counter",
  15.     "samples": [
  16.       {
  17.         "value": 8,
  18.         "labels": {
  19.           "kind": "",
  20.           "op": "save",
  21.           "result": "success"
  22.         }
  23.       },
  24.       {
  25.         "value": 1,
  26.         "labels": {
  27.           "kind": "empty",
  28.           "op": "load",
  29.           "result": "success"
  30.         }
  31.       }
  32.     ]
  33.   }
  34.   ...
  35. }

Hint: To get a list of a metrics and their help information, run:

  1. alias jq='docker run --rm -i realguess/jq jq'
  2. curl -sk -u "<user>:<password>" https://<syncmaster-IP>:8629/metrics.json | \
  3.   jq 'with_entries({key: .key, value:.value.help})'