Dynamic configuration API

This document describes the API endpoints to retrieve and manage dynamic configurations for the Coordinator and Overlord in Apache Druid.

In this topic, http://ROUTER_IP:ROUTER_PORT is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use http://localhost:8888 for quickstart deployments.

Coordinator dynamic configuration

The Coordinator has dynamic configurations to tune certain behavior on the fly, without requiring a service restart. For information on the supported properties, see Coordinator dynamic configuration.

Get dynamic configuration

Retrieves the current Coordinator dynamic configuration. Returns a JSON object with the dynamic configuration properties.

URL

GET /druid/coordinator/v1/config

Responses

  • 200 SUCCESS

Successfully retrieved dynamic configuration

Sample request

  • cURL
  • HTTP
  1. curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config"
  1. GET /druid/coordinator/v1/config HTTP/1.1
  2. Host: http://ROUTER_IP:ROUTER_PORT

Sample response

View the response

  1. {
  2.     "millisToWaitBeforeDeleting": 900000,
  3.     "mergeBytesLimit": 524288000,
  4.     "mergeSegmentsLimit": 100,
  5.     "maxSegmentsToMove": 100,
  6.     "replicantLifetime": 15,
  7.     "replicationThrottleLimit": 500,
  8.     "balancerComputeThreads": 1,
  9.     "killDataSourceWhitelist": [],
  10.     "killPendingSegmentsSkipList": [],
  11.     "maxSegmentsInNodeLoadingQueue": 500,
  12.     "decommissioningNodes": [],
  13.     "decommissioningMaxPercentOfMaxSegmentsToMove": 70,
  14.     "pauseCoordination": false,
  15.     "replicateAfterLoadTimeout": false,
  16.     "maxNonPrimaryReplicantsToLoad": 2147483647,
  17.     "useRoundRobinSegmentAssignment": true,
  18.     "smartSegmentLoading": true,
  19.     "debugDimensions": null
  20. }

Update dynamic configuration

Submits a JSON-based dynamic configuration spec to the Coordinator. For information on the supported properties, see Dynamic configuration.

URL

POST /druid/coordinator/v1/config

Header parameters

The endpoint supports a set of optional header parameters to populate the author and comment fields in the configuration history.

  • X-Druid-Author
    • Type: String
    • Author of the configuration change.
  • X-Druid-Comment
    • Type: String
    • Description for the update.

Responses

  • 200 SUCCESS

Successfully updated dynamic configuration

Sample request

  • cURL
  • HTTP
  1. curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config" \
  2. --header 'Content-Type: application/json' \
  3. --data '{
  4.   "millisToWaitBeforeDeleting": 900000,
  5.   "mergeBytesLimit": 524288000,
  6.   "mergeSegmentsLimit": 100,
  7.   "maxSegmentsToMove": 5,
  8.   "percentOfSegmentsToConsiderPerMove": 100,
  9.   "useBatchedSegmentSampler": true,
  10.   "replicantLifetime": 15,
  11.   "replicationThrottleLimit": 10,
  12.   "balancerComputeThreads": 1,
  13.   "emitBalancingStats": true,
  14.   "killDataSourceWhitelist": [],
  15.   "killPendingSegmentsSkipList": [],
  16.   "maxSegmentsInNodeLoadingQueue": 100,
  17.   "decommissioningNodes": [],
  18.   "decommissioningMaxPercentOfMaxSegmentsToMove": 70,
  19.   "pauseCoordination": false,
  20.   "replicateAfterLoadTimeout": false,
  21.   "maxNonPrimaryReplicantsToLoad": 2147483647,
  22.   "useRoundRobinSegmentAssignment": true
  23. }'
  1. POST /druid/coordinator/v1/config HTTP/1.1
  2. Host: http://ROUTER_IP:ROUTER_PORT
  3. Content-Type: application/json
  4. Content-Length: 683
  6. {
  7.   "millisToWaitBeforeDeleting": 900000,
  8.   "mergeBytesLimit": 524288000,
  9.   "mergeSegmentsLimit": 100,
  10.   "maxSegmentsToMove": 5,
  11.   "percentOfSegmentsToConsiderPerMove": 100,
  12.   "useBatchedSegmentSampler": true,
  13.   "replicantLifetime": 15,
  14.   "replicationThrottleLimit": 10,
  15.   "balancerComputeThreads": 1,
  16.   "emitBalancingStats": true,
  17.   "killDataSourceWhitelist": [],
  18.   "killPendingSegmentsSkipList": [],
  19.   "maxSegmentsInNodeLoadingQueue": 100,
  20.   "decommissioningNodes": [],
  21.   "decommissioningMaxPercentOfMaxSegmentsToMove": 70,
  22.   "pauseCoordination": false,
  23.   "replicateAfterLoadTimeout": false,
  24.   "maxNonPrimaryReplicantsToLoad": 2147483647,
  25.   "useRoundRobinSegmentAssignment": true
  26. }

Sample response

A successful request returns an HTTP 200 OK message code and an empty response body.

Get dynamic configuration history

Retrieves the history of changes to Coordinator dynamic configuration over an interval of time. Returns an empty array if there are no history records available.

URL

GET /druid/coordinator/v1/config/history

Query parameters

The endpoint supports a set of optional query parameters to filter results.

  • interval

    • Type: String
    • Limit the results to the specified time interval in ISO 8601 format delimited with /. For example, 2023-07-13/2023-07-19. The default interval is one week. You can change this period by setting druid.audit.manager.auditHistoryMillis in the runtime.properties file for the Coordinator.

  • count

    • Type: Integer
    • Limit the number of results to the last n entries.

Responses

  • 200 SUCCESS

Successfully retrieved history

Sample request

The following example retrieves the dynamic configuration history between 2022-07-13 and 2024-07-19. The response is limited to 10 entries.

  • cURL
  • HTTP
  1. curl "http://ROUTER_IP:ROUTER_PORT/druid/coordinator/v1/config/history?interval=2022-07-13%2F2024-07-19&count=10"
  1. GET /druid/coordinator/v1/config/history?interval=2022-07-13/2024-07-19&count=10 HTTP/1.1
  2. Host: http://ROUTER_IP:ROUTER_PORT

Sample response

View the response

  1. [
  2.     {
  3.         "key": "coordinator.config",
  4.         "type": "coordinator.config",
  5.         "auditInfo": {
  6.             "author": "",
  7.             "comment": "",
  8.             "ip": "127.0.0.1"
  9.         },
  10.         "payload": "{\"millisToWaitBeforeDeleting\":900000,\"mergeBytesLimit\":524288000,\"mergeSegmentsLimit\":100,\"maxSegmentsToMove\":5,\"replicantLifetime\":15,\"replicationThrottleLimit\":10,\"balancerComputeThreads\":1,\"killDataSourceWhitelist\":[],\"killPendingSegmentsSkipList\":[],\"maxSegmentsInNodeLoadingQueue\":100,\"decommissioningNodes\":[],\"decommissioningMaxPercentOfMaxSegmentsToMove\":70,\"pauseCoordination\":false,\"replicateAfterLoadTimeout\":false,\"maxNonPrimaryReplicantsToLoad\":2147483647,\"useRoundRobinSegmentAssignment\":true,\"smartSegmentLoading\":true,\"debugDimensions\":null}",
  11.         "auditTime": "2023-10-03T20:59:51.622Z"
  12.     }
  13. ]

Overlord dynamic configuration

The Overlord has dynamic configurations to tune how Druid assigns tasks to workers. For information on the supported properties, see Overlord dynamic configuration.

Get dynamic configuration

Retrieves the current Overlord dynamic configuration. Returns a JSON object with the dynamic configuration properties. Returns an empty response body if there is no current Overlord dynamic configuration.

URL

GET /druid/indexer/v1/worker

Responses

  • 200 SUCCESS

Successfully retrieved dynamic configuration

Sample request

  • cURL
  • HTTP
  1. curl "http://ROUTER_IP:ROUTER_PORT/druid/indexer/v1/worker"
  1. GET /druid/indexer/v1/worker HTTP/1.1
  2. Host: http://ROUTER_IP:ROUTER_PORT

Sample response

View the response

  1. {
  2.     "type": "default",
  3.     "selectStrategy": {
  4.         "type": "fillCapacityWithCategorySpec",
  5.         "workerCategorySpec": {
  6.             "categoryMap": {},
  7.             "strong": true
  8.         }
  9.     },
  10.     "autoScaler": null
  11. }

Update dynamic configuration

Submits a JSON-based dynamic configuration spec to the Overlord. For information on the supported properties, see Overlord dynamic configuration.

URL

POST /druid/indexer/v1/worker

Header parameters

The endpoint supports a set of optional header parameters to populate the author and comment fields in the configuration history.

  • X-Druid-Author
    • Type: String
    • Author of the configuration change.
  • X-Druid-Comment
    • Type: String
    • Description for the update.

Responses

  • 200 SUCCESS

Successfully updated dynamic configuration

Sample request

  • cURL
  • HTTP
  1. curl "http://ROUTER_IP:ROUTER_PORT/druid/indexer/v1/worker" \
  2. --header 'Content-Type: application/json' \
  3. --data '{
  4.   "type": "default",
  5.   "selectStrategy": {
  6.     "type": "fillCapacityWithCategorySpec",
  7.     "workerCategorySpec": {
  8.       "categoryMap": {},
  9.       "strong": true
  10.     }
  11.   },
  12.   "autoScaler": null
  13. }'
  1. POST /druid/indexer/v1/worker HTTP/1.1
  2. Host: http://ROUTER_IP:ROUTER_PORT
  3. Content-Type: application/json
  4. Content-Length: 196
  6. {
  7.   "type": "default",
  8.   "selectStrategy": {
  9.     "type": "fillCapacityWithCategorySpec",
  10.     "workerCategorySpec": {
  11.       "categoryMap": {},
  12.       "strong": true
  13.     }
  14.   },
  15.   "autoScaler": null
  16. }

Sample response

A successful request returns an HTTP 200 OK message code and an empty response body.

Get dynamic configuration history

Retrieves the history of changes to Overlord dynamic configuration over an interval of time. Returns an empty array if there are no history records available.

URL

GET /druid/indexer/v1/worker/history

Query parameters

The endpoint supports a set of optional query parameters to filter results.

  • interval

    • Type: String
    • Limit the results to the specified time interval in ISO 8601 format delimited with /. For example, 2023-07-13/2023-07-19. The default interval is one week. You can change this period by setting druid.audit.manager.auditHistoryMillis in the runtime.properties file for the Overlord.

  • count

    • Type: Integer
    • Limit the number of results to the last n entries.

Responses

  • 200 SUCCESS

Successfully retrieved history

Sample request

The following example retrieves the dynamic configuration history between 2022-07-13 and 2024-07-19. The response is limited to 10 entries.

  • cURL
  • HTTP
  1. curl "http://ROUTER_IP:ROUTER_PORT/druid/indexer/v1/worker/history?interval=2022-07-13%2F2024-07-19&count=10"
  1. GET /druid/indexer/v1/worker/history?interval=2022-07-13%2F2024-07-19&count=10 HTTP/1.1
  2. Host: http://ROUTER_IP:ROUTER_PORT

Sample response

View the response

  1. [
  2.     {
  3.         "key": "worker.config",
  4.         "type": "worker.config",
  5.         "auditInfo": {
  6.             "author": "",
  7.             "comment": "",
  8.             "ip": "127.0.0.1"
  9.         },
  10.         "payload": "{\"type\":\"default\",\"selectStrategy\":{\"type\":\"fillCapacityWithCategorySpec\",\"workerCategorySpec\":{\"categoryMap\":{},\"strong\":true}},\"autoScaler\":null}",
  11.         "auditTime": "2023-10-03T21:49:49.991Z"
  12.     }
  13. ]

Get an array of worker nodes in the cluster

Returns an array of all the worker nodes in the cluster along with its corresponding metadata.

GET /druid/indexer/v1/workers

Responses

  • 200 SUCCESS

Successfully retrieved worker nodes

Sample request

  • cURL
  • HTTP
  1. curl "http://ROUTER_IP:ROUTER_PORT/druid/indexer/v1/workers"
  1. GET /druid/indexer/v1/workers HTTP/1.1
  2. Host: http://ROUTER_IP:ROUTER_PORT

Sample response

View the response

  1. [
  2.     {
  3.         "worker": {
  4.             "scheme": "http",
  5.             "host": "localhost:8091",
  6.             "ip": "198.51.100.0",
  7.             "capacity": 2,
  8.             "version": "0",
  9.             "category": "_default_worker_category"
  10.         },
  11.         "currCapacityUsed": 0,
  12.         "currParallelIndexCapacityUsed": 0,
  13.         "availabilityGroups": [],
  14.         "runningTasks": [],
  15.         "lastCompletedTaskTime": "2023-09-29T19:13:05.505Z",
  16.         "blacklistedUntil": null
  17.     }
  18. ]

Get scaling events

Returns Overlord scaling events if autoscaling runners are in use. Returns an empty response body if there are no Overlord scaling events.

URL

GET /druid/indexer/v1/scaling

Responses

  • 200 SUCCESS

Successfully retrieved scaling events

Sample request

  • cURL
  • HTTP
  1. curl "http://ROUTER_IP:ROUTER_PORT/druid/indexer/v1/scaling"
  1. GET /druid/indexer/v1/scaling HTTP/1.1
  2. Host: http://ROUTER_IP:ROUTER_PORT

Sample response

A successful request returns a 200 OK response and an array of scaling events.