Supervisor API

This topic describes the API endpoints to manage and monitor supervisors for Apache Druid.

In this topic, http://ROUTER_IP:ROUTER_PORT is a placeholder for your Router service address and port. Replace it with the information for your deployment. For example, use http://localhost:8888 for quickstart deployments.

Supervisor information

The following table lists the properties of a supervisor object:

PropertyTypeDescription
idStringUnique identifier.
stateStringGeneric state of the supervisor. Available states:UNHEALTHY_SUPERVISOR, UNHEALTHY_TASKS, PENDING, RUNNING, SUSPENDED, STOPPING. See Apache Kafka operations for details.
detailedStateStringDetailed state of the supervisor. This property contains a more descriptive, implementation-specific state that may provide more insight into the supervisor’s activities than the state property. See Apache Kafka ingestion and Amazon Kinesis ingestion for supervisor-specific states.
healthyBooleanSupervisor health indicator.
specObjectContainer object for the supervisor configuration.
suspendedBooleanIndicates whether the supervisor is in a suspended state.

Get an array of active supervisor IDs

Returns an array of strings representing the names of active supervisors. If there are no active supervisors, it returns an empty array.

URL

GET /druid/indexer/v1/supervisor

Responses

  • 200 SUCCESS

Successfully retrieved array of active supervisor IDs

Sample request

  • cURL
  • HTTP
  1. curl "http://ROUTER_IP:ROUTER_PORT/druid/indexer/v1/supervisor"
  1. GET /druid/indexer/v1/supervisor HTTP/1.1
  2. Host: http://ROUTER_IP:ROUTER_PORT

Sample response

Click to show sample response

  1. [
  2.   "wikipedia_stream",
  3.   "social_media"
  4. ]

Get an array of active supervisor objects

Retrieves an array of active supervisor objects. If there are no active supervisors, it returns an empty array. For reference on the supervisor object properties, see the preceding table.

URL

GET /druid/indexer/v1/supervisor?full

Responses

  • 200 SUCCESS

Successfully retrieved supervisor objects

Sample request

  • cURL
  • HTTP
  1. curl "http://ROUTER_IP:ROUTER_PORT/druid/indexer/v1/supervisor?full=null"
  1. GET /druid/indexer/v1/supervisor?full=null HTTP/1.1
  2. Host: http://ROUTER_IP:ROUTER_PORT

Sample response

Click to show sample response

  1. [
  2.   {
  3.       "id": "wikipedia_stream",
  4.       "state": "RUNNING",
  5.       "detailedState": "CONNECTING_TO_STREAM",
  6.       "healthy": true,
  7.       "spec": {
  8.           "type": "kafka",
  9.           "spec": {
  10.               "dataSchema": {
  11.                   "dataSource": "wikipedia_stream",
  12.                   "timestampSpec": {
  13.                       "column": "__time",
  14.                       "format": "iso",
  15.                       "missingValue": null
  16.                   },
  17.                   "dimensionsSpec": {
  18.                       "dimensions": [
  19.                           {
  20.                               "type": "string",
  21.                               "name": "username",
  22.                               "multiValueHandling": "SORTED_ARRAY",
  23.                               "createBitmapIndex": true
  24.                           },
  25.                           {
  26.                               "type": "string",
  27.                               "name": "post_title",
  28.                               "multiValueHandling": "SORTED_ARRAY",
  29.                               "createBitmapIndex": true
  30.                           },
  31.                           {
  32.                               "type": "long",
  33.                               "name": "views",
  34.                               "multiValueHandling": "SORTED_ARRAY",
  35.                               "createBitmapIndex": false
  36.                           },
  37.                           {
  38.                               "type": "long",
  39.                               "name": "upvotes",
  40.                               "multiValueHandling": "SORTED_ARRAY",
  41.                               "createBitmapIndex": false
  42.                           },
  43.                           {
  44.                               "type": "long",
  45.                               "name": "comments",
  46.                               "multiValueHandling": "SORTED_ARRAY",
  47.                               "createBitmapIndex": false
  48.                           },
  49.                           {
  50.                               "type": "string",
  51.                               "name": "edited",
  52.                               "multiValueHandling": "SORTED_ARRAY",
  53.                               "createBitmapIndex": true
  54.                           }
  55.                       ],
  56.                       "dimensionExclusions": [
  57.                           "__time"
  58.                       ],
  59.                       "includeAllDimensions": false,
  60.                       "useSchemaDiscovery": false
  61.                   },
  62.                   "metricsSpec": [],
  63.                   "granularitySpec": {
  64.                       "type": "uniform",
  65.                       "segmentGranularity": "HOUR",
  66.                       "queryGranularity": {
  67.                           "type": "none"
  68.                       },
  69.                       "rollup": false,
  70.                       "intervals": []
  71.                   },
  72.                   "transformSpec": {
  73.                       "filter": null,
  74.                       "transforms": []
  75.                   }
  76.               },
  77.               "ioConfig": {
  78.                   "topic": "social_media",
  79.                   "inputFormat": {
  80.                       "type": "json",
  81.                       "keepNullColumns": false,
  82.                       "assumeNewlineDelimited": false,
  83.                       "useJsonNodeReader": false
  84.                   },
  85.                   "replicas": 1,
  86.                   "taskCount": 1,
  87.                   "taskDuration": "PT3600S",
  88.                   "consumerProperties": {
  89.                       "bootstrap.servers": "localhost:9042"
  90.                   },
  91.                   "autoScalerConfig": null,
  92.                   "pollTimeout": 100,
  93.                   "startDelay": "PT5S",
  94.                   "period": "PT30S",
  95.                   "useEarliestOffset": true,
  96.                   "completionTimeout": "PT1800S",
  97.                   "lateMessageRejectionPeriod": null,
  98.                   "earlyMessageRejectionPeriod": null,
  99.                   "lateMessageRejectionStartDateTime": null,
  100.                   "configOverrides": null,
  101.                   "idleConfig": null,
  102.                   "stream": "social_media",
  103.                   "useEarliestSequenceNumber": true
  104.               },
  105.               "tuningConfig": {
  106.                   "type": "kafka",
  107.                   "appendableIndexSpec": {
  108.                       "type": "onheap",
  109.                       "preserveExistingMetrics": false
  110.                   },
  111.                   "maxRowsInMemory": 150000,
  112.                   "maxBytesInMemory": 0,
  113.                   "skipBytesInMemoryOverheadCheck": false,
  114.                   "maxRowsPerSegment": 5000000,
  115.                   "maxTotalRows": null,
  116.                   "intermediatePersistPeriod": "PT10M",
  117.                   "maxPendingPersists": 0,
  118.                   "indexSpec": {
  119.                       "bitmap": {
  120.                           "type": "roaring"
  121.                       },
  122.                       "dimensionCompression": "lz4",
  123.                       "stringDictionaryEncoding": {
  124.                           "type": "utf8"
  125.                       },
  126.                       "metricCompression": "lz4",
  127.                       "longEncoding": "longs"
  128.                   },
  129.                   "indexSpecForIntermediatePersists": {
  130.                       "bitmap": {
  131.                           "type": "roaring"
  132.                       },
  133.                       "dimensionCompression": "lz4",
  134.                       "stringDictionaryEncoding": {
  135.                           "type": "utf8"
  136.                       },
  137.                       "metricCompression": "lz4",
  138.                       "longEncoding": "longs"
  139.                   },
  140.                   "reportParseExceptions": false,
  141.                   "handoffConditionTimeout": 0,
  142.                   "resetOffsetAutomatically": false,
  143.                   "segmentWriteOutMediumFactory": null,
  144.                   "workerThreads": null,
  145.                   "chatRetries": 8,
  146.                   "httpTimeout": "PT10S",
  147.                   "shutdownTimeout": "PT80S",
  148.                   "offsetFetchPeriod": "PT30S",
  149.                   "intermediateHandoffPeriod": "P2147483647D",
  150.                   "logParseExceptions": false,
  151.                   "maxParseExceptions": 2147483647,
  152.                   "maxSavedParseExceptions": 0,
  153.                   "skipSequenceNumberAvailabilityCheck": false,
  154.                   "repartitionTransitionDuration": "PT120S"
  155.               }
  156.           },
  157.           "dataSchema": {
  158.               "dataSource": "wikipedia_stream",
  159.               "timestampSpec": {
  160.                   "column": "__time",
  161.                   "format": "iso",
  162.                   "missingValue": null
  163.               },
  164.               "dimensionsSpec": {
  165.                   "dimensions": [
  166.                       {
  167.                           "type": "string",
  168.                           "name": "username",
  169.                           "multiValueHandling": "SORTED_ARRAY",
  170.                           "createBitmapIndex": true
  171.                       },
  172.                       {
  173.                           "type": "string",
  174.                           "name": "post_title",
  175.                           "multiValueHandling": "SORTED_ARRAY",
  176.                           "createBitmapIndex": true
  177.                       },
  178.                       {
  179.                           "type": "long",
  180.                           "name": "views",
  181.                           "multiValueHandling": "SORTED_ARRAY",
  182.                           "createBitmapIndex": false
  183.                       },
  184.                       {
  185.                           "type": "long",
  186.                           "name": "upvotes",
  187.                           "multiValueHandling": "SORTED_ARRAY",
  188.                           "createBitmapIndex": false
  189.                       },
  190.                       {
  191.                           "type": "long",
  192.                           "name": "comments",
  193.                           "multiValueHandling": "SORTED_ARRAY",
  194.                           "createBitmapIndex": false
  195.                       },
  196.                       {
  197.                           "type": "string",
  198.                           "name": "edited",
  199.                           "multiValueHandling": "SORTED_ARRAY",
  200.                           "createBitmapIndex": true
  201.                       }
  202.                   ],
  203.                   "dimensionExclusions": [
  204.                       "__time"
  205.                   ],
  206.                   "includeAllDimensions": false,
  207.                   "useSchemaDiscovery": false
  208.               },
  209.               "metricsSpec": [],
  210.               "granularitySpec": {
  211.                   "type": "uniform",
  212.                   "segmentGranularity": "HOUR",
  213.                   "queryGranularity": {
  214.                       "type": "none"
  215.                   },
  216.                   "rollup": false,
  217.                   "intervals": []
  218.               },
  219.               "transformSpec": {
  220.                   "filter": null,
  221.                   "transforms": []
  222.               }
  223.           },
  224.           "tuningConfig": {
  225.               "type": "kafka",
  226.               "appendableIndexSpec": {
  227.                   "type": "onheap",
  228.                   "preserveExistingMetrics": false
  229.               },
  230.               "maxRowsInMemory": 150000,
  231.               "maxBytesInMemory": 0,
  232.               "skipBytesInMemoryOverheadCheck": false,
  233.               "maxRowsPerSegment": 5000000,
  234.               "maxTotalRows": null,
  235.               "intermediatePersistPeriod": "PT10M",
  236.               "maxPendingPersists": 0,
  237.               "indexSpec": {
  238.                   "bitmap": {
  239.                       "type": "roaring"
  240.                   },
  241.                   "dimensionCompression": "lz4",
  242.                   "stringDictionaryEncoding": {
  243.                       "type": "utf8"
  244.                   },
  245.                   "metricCompression": "lz4",
  246.                   "longEncoding": "longs"
  247.               },
  248.               "indexSpecForIntermediatePersists": {
  249.                   "bitmap": {
  250.                       "type": "roaring"
  251.                   },
  252.                   "dimensionCompression": "lz4",
  253.                   "stringDictionaryEncoding": {
  254.                       "type": "utf8"
  255.                   },
  256.                   "metricCompression": "lz4",
  257.                   "longEncoding": "longs"
  258.               },
  259.               "reportParseExceptions": false,
  260.               "handoffConditionTimeout": 0,
  261.               "resetOffsetAutomatically": false,
  262.               "segmentWriteOutMediumFactory": null,
  263.               "workerThreads": null,
  264.               "chatRetries": 8,
  265.               "httpTimeout": "PT10S",
  266.               "shutdownTimeout": "PT80S",
  267.               "offsetFetchPeriod": "PT30S",
  268.               "intermediateHandoffPeriod": "P2147483647D",
  269.               "logParseExceptions": false,
  270.               "maxParseExceptions": 2147483647,
  271.               "maxSavedParseExceptions": 0,
  272.               "skipSequenceNumberAvailabilityCheck": false,
  273.               "repartitionTransitionDuration": "PT120S"
  274.           },
  275.           "ioConfig": {
  276.               "topic": "social_media",
  277.               "inputFormat": {
  278.                   "type": "json",
  279.                   "keepNullColumns": false,
  280.                   "assumeNewlineDelimited": false,
  281.                   "useJsonNodeReader": false
  282.               },
  283.               "replicas": 1,
  284.               "taskCount": 1,
  285.               "taskDuration": "PT3600S",
  286.               "consumerProperties": {
  287.                   "bootstrap.servers": "localhost:9042"
  288.               },
  289.               "autoScalerConfig": null,
  290.               "pollTimeout": 100,
  291.               "startDelay": "PT5S",
  292.               "period": "PT30S",
  293.               "useEarliestOffset": true,
  294.               "completionTimeout": "PT1800S",
  295.               "lateMessageRejectionPeriod": null,
  296.               "earlyMessageRejectionPeriod": null,
  297.               "lateMessageRejectionStartDateTime": null,
  298.               "configOverrides": null,
  299.               "idleConfig": null,
  300.               "stream": "social_media",
  301.               "useEarliestSequenceNumber": true
  302.           },
  303.           "context": null,
  304.           "suspended": false
  305.       },
  306.       "suspended": false
  307.   },
  308.   {
  309.       "id": "social_media",
  310.       "state": "RUNNING",
  311.       "detailedState": "RUNNING",
  312.       "healthy": true,
  313.       "spec": {
  314.           "type": "kafka",
  315.           "spec": {
  316.               "dataSchema": {
  317.                   "dataSource": "social_media",
  318.                   "timestampSpec": {
  319.                       "column": "__time",
  320.                       "format": "iso",
  321.                       "missingValue": null
  322.                   },
  323.                   "dimensionsSpec": {
  324.                       "dimensions": [
  325.                           {
  326.                               "type": "string",
  327.                               "name": "username",
  328.                               "multiValueHandling": "SORTED_ARRAY",
  329.                               "createBitmapIndex": true
  330.                           },
  331.                           {
  332.                               "type": "string",
  333.                               "name": "post_title",
  334.                               "multiValueHandling": "SORTED_ARRAY",
  335.                               "createBitmapIndex": true
  336.                           },
  337.                           {
  338.                               "type": "long",
  339.                               "name": "views",
  340.                               "multiValueHandling": "SORTED_ARRAY",
  341.                               "createBitmapIndex": false
  342.                           },
  343.                           {
  344.                               "type": "long",
  345.                               "name": "upvotes",
  346.                               "multiValueHandling": "SORTED_ARRAY",
  347.                               "createBitmapIndex": false
  348.                           },
  349.                           {
  350.                               "type": "long",
  351.                               "name": "comments",
  352.                               "multiValueHandling": "SORTED_ARRAY",
  353.                               "createBitmapIndex": false
  354.                           },
  355.                           {
  356.                               "type": "string",
  357.                               "name": "edited",
  358.                               "multiValueHandling": "SORTED_ARRAY",
  359.                               "createBitmapIndex": true
  360.                           }
  361.                       ],
  362.                       "dimensionExclusions": [
  363.                           "__time"
  364.                       ],
  365.                       "includeAllDimensions": false,
  366.                       "useSchemaDiscovery": false
  367.                   },
  368.                   "metricsSpec": [],
  369.                   "granularitySpec": {
  370.                       "type": "uniform",
  371.                       "segmentGranularity": "HOUR",
  372.                       "queryGranularity": {
  373.                           "type": "none"
  374.                       },
  375.                       "rollup": false,
  376.                       "intervals": []
  377.                   },
  378.                   "transformSpec": {
  379.                       "filter": null,
  380.                       "transforms": []
  381.                   }
  382.               },
  383.               "ioConfig": {
  384.                   "topic": "social_media",
  385.                   "inputFormat": {
  386.                       "type": "json",
  387.                       "keepNullColumns": false,
  388.                       "assumeNewlineDelimited": false,
  389.                       "useJsonNodeReader": false
  390.                   },
  391.                   "replicas": 1,
  392.                   "taskCount": 1,
  393.                   "taskDuration": "PT3600S",
  394.                   "consumerProperties": {
  395.                       "bootstrap.servers": "localhost:9094"
  396.                   },
  397.                   "autoScalerConfig": null,
  398.                   "pollTimeout": 100,
  399.                   "startDelay": "PT5S",
  400.                   "period": "PT30S",
  401.                   "useEarliestOffset": true,
  402.                   "completionTimeout": "PT1800S",
  403.                   "lateMessageRejectionPeriod": null,
  404.                   "earlyMessageRejectionPeriod": null,
  405.                   "lateMessageRejectionStartDateTime": null,
  406.                   "configOverrides": null,
  407.                   "idleConfig": null,
  408.                   "stream": "social_media",
  409.                   "useEarliestSequenceNumber": true
  410.               },
  411.               "tuningConfig": {
  412.                   "type": "kafka",
  413.                   "appendableIndexSpec": {
  414.                       "type": "onheap",
  415.                       "preserveExistingMetrics": false
  416.                   },
  417.                   "maxRowsInMemory": 150000,
  418.                   "maxBytesInMemory": 0,
  419.                   "skipBytesInMemoryOverheadCheck": false,
  420.                   "maxRowsPerSegment": 5000000,
  421.                   "maxTotalRows": null,
  422.                   "intermediatePersistPeriod": "PT10M",
  423.                   "maxPendingPersists": 0,
  424.                   "indexSpec": {
  425.                       "bitmap": {
  426.                           "type": "roaring"
  427.                       },
  428.                       "dimensionCompression": "lz4",
  429.                       "stringDictionaryEncoding": {
  430.                           "type": "utf8"
  431.                       },
  432.                       "metricCompression": "lz4",
  433.                       "longEncoding": "longs"
  434.                   },
  435.                   "indexSpecForIntermediatePersists": {
  436.                       "bitmap": {
  437.                           "type": "roaring"
  438.                       },
  439.                       "dimensionCompression": "lz4",
  440.                       "stringDictionaryEncoding": {
  441.                           "type": "utf8"
  442.                       },
  443.                       "metricCompression": "lz4",
  444.                       "longEncoding": "longs"
  445.                   },
  446.                   "reportParseExceptions": false,
  447.                   "handoffConditionTimeout": 0,
  448.                   "resetOffsetAutomatically": false,
  449.                   "segmentWriteOutMediumFactory": null,
  450.                   "workerThreads": null,
  451.                   "chatRetries": 8,
  452.                   "httpTimeout": "PT10S",
  453.                   "shutdownTimeout": "PT80S",
  454.                   "offsetFetchPeriod": "PT30S",
  455.                   "intermediateHandoffPeriod": "P2147483647D",
  456.                   "logParseExceptions": false,
  457.                   "maxParseExceptions": 2147483647,
  458.                   "maxSavedParseExceptions": 0,
  459.                   "skipSequenceNumberAvailabilityCheck": false,
  460.                   "repartitionTransitionDuration": "PT120S"
  461.               }
  462.           },
  463.           "dataSchema": {
  464.               "dataSource": "social_media",
  465.               "timestampSpec": {
  466.                   "column": "__time",
  467.                   "format": "iso",
  468.                   "missingValue": null
  469.               },
  470.               "dimensionsSpec": {
  471.                   "dimensions": [
  472.                       {
  473.                           "type": "string",
  474.                           "name": "username",
  475.                           "multiValueHandling": "SORTED_ARRAY",
  476.                           "createBitmapIndex": true
  477.                       },
  478.                       {
  479.                           "type": "string",
  480.                           "name": "post_title",
  481.                           "multiValueHandling": "SORTED_ARRAY",
  482.                           "createBitmapIndex": true
  483.                       },
  484.                       {
  485.                           "type": "long",
  486.                           "name": "views",
  487.                           "multiValueHandling": "SORTED_ARRAY",
  488.                           "createBitmapIndex": false
  489.                       },
  490.                       {
  491.                           "type": "long",
  492.                           "name": "upvotes",
  493.                           "multiValueHandling": "SORTED_ARRAY",
  494.                           "createBitmapIndex": false
  495.                       },
  496.                       {
  497.                           "type": "long",
  498.                           "name": "comments",
  499.                           "multiValueHandling": "SORTED_ARRAY",
  500.                           "createBitmapIndex": false
  501.                       },
  502.                       {
  503.                           "type": "string",
  504.                           "name": "edited",
  505.                           "multiValueHandling": "SORTED_ARRAY",
  506.                           "createBitmapIndex": true
  507.                       }
  508.                   ],
  509.                   "dimensionExclusions": [
  510.                       "__time"
  511.                   ],
  512.                   "includeAllDimensions": false,
  513.                   "useSchemaDiscovery": false
  514.               },
  515.               "metricsSpec": [],
  516.               "granularitySpec": {
  517.                   "type": "uniform",
  518.                   "segmentGranularity": "HOUR",
  519.                   "queryGranularity": {
  520.                       "type": "none"
  521.                   },
  522.                   "rollup": false,
  523.                   "intervals": []
  524.               },
  525.               "transformSpec": {
  526.                   "filter": null,
  527.                   "transforms": []
  528.               }
  529.           },
  530.           "tuningConfig": {
  531.               "type": "kafka",
  532.               "appendableIndexSpec": {
  533.                   "type": "onheap",
  534.                   "preserveExistingMetrics": false
  535.               },
  536.               "maxRowsInMemory": 150000,
  537.               "maxBytesInMemory": 0,
  538.               "skipBytesInMemoryOverheadCheck": false,
  539.               "maxRowsPerSegment": 5000000,
  540.               "maxTotalRows": null,
  541.               "intermediatePersistPeriod": "PT10M",
  542.               "maxPendingPersists": 0,
  543.               "indexSpec": {
  544.                   "bitmap": {
  545.                       "type": "roaring"
  546.                   },
  547.                   "dimensionCompression": "lz4",
  548.                   "stringDictionaryEncoding": {
  549.                       "type": "utf8"
  550.                   },
  551.                   "metricCompression": "lz4",
  552.                   "longEncoding": "longs"
  553.               },
  554.               "indexSpecForIntermediatePersists": {
  555.                   "bitmap": {
  556.                       "type": "roaring"
  557.                   },
  558.                   "dimensionCompression": "lz4",
  559.                   "stringDictionaryEncoding": {
  560.                       "type": "utf8"
  561.                   },
  562.                   "metricCompression": "lz4",
  563.                   "longEncoding": "longs"
  564.               },
  565.               "reportParseExceptions": false,
  566.               "handoffConditionTimeout": 0,
  567.               "resetOffsetAutomatically": false,
  568.               "segmentWriteOutMediumFactory": null,
  569.               "workerThreads": null,
  570.               "chatRetries": 8,
  571.               "httpTimeout": "PT10S",
  572.               "shutdownTimeout": "PT80S",
  573.               "offsetFetchPeriod": "PT30S",
  574.               "intermediateHandoffPeriod": "P2147483647D",
  575.               "logParseExceptions": false,
  576.               "maxParseExceptions": 2147483647,
  577.               "maxSavedParseExceptions": 0,
  578.               "skipSequenceNumberAvailabilityCheck": false,
  579.               "repartitionTransitionDuration": "PT120S"
  580.           },
  581.           "ioConfig": {
  582.               "topic": "social_media",
  583.               "inputFormat": {
  584.                   "type": "json",
  585.                   "keepNullColumns": false,
  586.                   "assumeNewlineDelimited": false,
  587.                   "useJsonNodeReader": false
  588.               },
  589.               "replicas": 1,
  590.               "taskCount": 1,
  591.               "taskDuration": "PT3600S",
  592.               "consumerProperties": {
  593.                   "bootstrap.servers": "localhost:9094"
  594.               },
  595.               "autoScalerConfig": null,
  596.               "pollTimeout": 100,
  597.               "startDelay": "PT5S",
  598.               "period": "PT30S",
  599.               "useEarliestOffset": true,
  600.               "completionTimeout": "PT1800S",
  601.               "lateMessageRejectionPeriod": null,
  602.               "earlyMessageRejectionPeriod": null,
  603.               "lateMessageRejectionStartDateTime": null,
  604.               "configOverrides": null,
  605.               "idleConfig": null,
  606.               "stream": "social_media",
  607.               "useEarliestSequenceNumber": true
  608.           },
  609.           "context": null,
  610.           "suspended": false
  611.       },
  612.       "suspended": false
  613.   }
  614. ]

Get an array of supervisor states

Retrieves an array of objects representing active supervisors and their current state. If there are no active supervisors, it returns an empty array. For reference on the supervisor object properties, see the preceding table.

URL

GET /druid/indexer/v1/supervisor?state=true

Responses

  • 200 SUCCESS

Successfully retrieved supervisor state objects

Sample request

  • cURL
  • HTTP
  1. curl "http://ROUTER_IP:ROUTER_PORT/druid/indexer/v1/supervisor?state=true"
  1. GET /druid/indexer/v1/supervisor?state=true HTTP/1.1
  2. Host: http://ROUTER_IP:ROUTER_PORT

Sample response

Click to show sample response

  1. [
  2.   {
  3.       "id": "wikipedia_stream",
  4.       "state": "UNHEALTHY_SUPERVISOR",
  5.       "detailedState": "UNABLE_TO_CONNECT_TO_STREAM",
  6.       "healthy": false,
  7.       "suspended": false
  8.   },
  9.   {
  10.       "id": "social_media",
  11.       "state": "RUNNING",
  12.       "detailedState": "RUNNING",
  13.       "healthy": true,
  14.       "suspended": false
  15.   }
  16. ]

Get supervisor specification

Retrieves the specification for a single supervisor. The returned specification includes the dataSchema, ioConfig, and tuningConfig objects.

URL

GET /druid/indexer/v1/supervisor/:supervisorId

Responses

  • 200 SUCCESS
  • 404 NOT FOUND

Successfully retrieved supervisor spec

Invalid supervisor ID

Sample request

The following example shows how to retrieve the specification of a supervisor with the name wikipedia_stream.

  • cURL
  • HTTP
  1. curl "http://ROUTER_IP:ROUTER_PORT/druid/indexer/v1/supervisor/wikipedia_stream"
  1. GET /druid/indexer/v1/supervisor/wikipedia_stream HTTP/1.1
  2. Host: http://ROUTER_IP:ROUTER_PORT

Sample response

Click to show sample response

  1. {
  2.   "type": "kafka",
  3.   "spec": {
  4.       "dataSchema": {
  5.           "dataSource": "social_media",
  6.           "timestampSpec": {
  7.               "column": "__time",
  8.               "format": "iso",
  9.               "missingValue": null
  10.           },
  11.           "dimensionsSpec": {
  12.               "dimensions": [
  13.                   {
  14.                       "type": "string",
  15.                       "name": "username",
  16.                       "multiValueHandling": "SORTED_ARRAY",
  17.                       "createBitmapIndex": true
  18.                   },
  19.                   {
  20.                       "type": "string",
  21.                       "name": "post_title",
  22.                       "multiValueHandling": "SORTED_ARRAY",
  23.                       "createBitmapIndex": true
  24.                   },
  25.                   {
  26.                       "type": "long",
  27.                       "name": "views",
  28.                       "multiValueHandling": "SORTED_ARRAY",
  29.                       "createBitmapIndex": false
  30.                   },
  31.                   {
  32.                       "type": "long",
  33.                       "name": "upvotes",
  34.                       "multiValueHandling": "SORTED_ARRAY",
  35.                       "createBitmapIndex": false
  36.                   },
  37.                   {
  38.                       "type": "long",
  39.                       "name": "comments",
  40.                       "multiValueHandling": "SORTED_ARRAY",
  41.                       "createBitmapIndex": false
  42.                   },
  43.                   {
  44.                       "type": "string",
  45.                       "name": "edited",
  46.                       "multiValueHandling": "SORTED_ARRAY",
  47.                       "createBitmapIndex": true
  48.                   }
  49.               ],
  50.               "dimensionExclusions": [
  51.                   "__time"
  52.               ],
  53.               "includeAllDimensions": false,
  54.               "useSchemaDiscovery": false
  55.           },
  56.           "metricsSpec": [],
  57.           "granularitySpec": {
  58.               "type": "uniform",
  59.               "segmentGranularity": "HOUR",
  60.               "queryGranularity": {
  61.                   "type": "none"
  62.               },
  63.               "rollup": false,
  64.               "intervals": []
  65.           },
  66.           "transformSpec": {
  67.               "filter": null,
  68.               "transforms": []
  69.           }
  70.       },
  71.       "ioConfig": {
  72.           "topic": "social_media",
  73.           "inputFormat": {
  74.               "type": "json",
  75.               "keepNullColumns": false,
  76.               "assumeNewlineDelimited": false,
  77.               "useJsonNodeReader": false
  78.           },
  79.           "replicas": 1,
  80.           "taskCount": 1,
  81.           "taskDuration": "PT3600S",
  82.           "consumerProperties": {
  83.               "bootstrap.servers": "localhost:9094"
  84.           },
  85.           "autoScalerConfig": null,
  86.           "pollTimeout": 100,
  87.           "startDelay": "PT5S",
  88.           "period": "PT30S",
  89.           "useEarliestOffset": true,
  90.           "completionTimeout": "PT1800S",
  91.           "lateMessageRejectionPeriod": null,
  92.           "earlyMessageRejectionPeriod": null,
  93.           "lateMessageRejectionStartDateTime": null,
  94.           "configOverrides": null,
  95.           "idleConfig": null,
  96.           "stream": "social_media",
  97.           "useEarliestSequenceNumber": true
  98.       },
  99.       "tuningConfig": {
  100.           "type": "kafka",
  101.           "appendableIndexSpec": {
  102.               "type": "onheap",
  103.               "preserveExistingMetrics": false
  104.           },
  105.           "maxRowsInMemory": 150000,
  106.           "maxBytesInMemory": 0,
  107.           "skipBytesInMemoryOverheadCheck": false,
  108.           "maxRowsPerSegment": 5000000,
  109.           "maxTotalRows": null,
  110.           "intermediatePersistPeriod": "PT10M",
  111.           "maxPendingPersists": 0,
  112.           "indexSpec": {
  113.               "bitmap": {
  114.                   "type": "roaring"
  115.               },
  116.               "dimensionCompression": "lz4",
  117.               "stringDictionaryEncoding": {
  118.                   "type": "utf8"
  119.               },
  120.               "metricCompression": "lz4",
  121.               "longEncoding": "longs"
  122.           },
  123.           "indexSpecForIntermediatePersists": {
  124.               "bitmap": {
  125.                   "type": "roaring"
  126.               },
  127.               "dimensionCompression": "lz4",
  128.               "stringDictionaryEncoding": {
  129.                   "type": "utf8"
  130.               },
  131.               "metricCompression": "lz4",
  132.               "longEncoding": "longs"
  133.           },
  134.           "reportParseExceptions": false,
  135.           "handoffConditionTimeout": 0,
  136.           "resetOffsetAutomatically": false,
  137.           "segmentWriteOutMediumFactory": null,
  138.           "workerThreads": null,
  139.           "chatRetries": 8,
  140.           "httpTimeout": "PT10S",
  141.           "shutdownTimeout": "PT80S",
  142.           "offsetFetchPeriod": "PT30S",
  143.           "intermediateHandoffPeriod": "P2147483647D",
  144.           "logParseExceptions": false,
  145.           "maxParseExceptions": 2147483647,
  146.           "maxSavedParseExceptions": 0,
  147.           "skipSequenceNumberAvailabilityCheck": false,
  148.           "repartitionTransitionDuration": "PT120S"
  149.       }
  150.   },
  151.   "dataSchema": {
  152.       "dataSource": "social_media",
  153.       "timestampSpec": {
  154.           "column": "__time",
  155.           "format": "iso",
  156.           "missingValue": null
  157.       },
  158.       "dimensionsSpec": {
  159.           "dimensions": [
  160.               {
  161.                   "type": "string",
  162.                   "name": "username",
  163.                   "multiValueHandling": "SORTED_ARRAY",
  164.                   "createBitmapIndex": true
  165.               },
  166.               {
  167.                   "type": "string",
  168.                   "name": "post_title",
  169.                   "multiValueHandling": "SORTED_ARRAY",
  170.                   "createBitmapIndex": true
  171.               },
  172.               {
  173.                   "type": "long",
  174.                   "name": "views",
  175.                   "multiValueHandling": "SORTED_ARRAY",
  176.                   "createBitmapIndex": false
  177.               },
  178.               {
  179.                   "type": "long",
  180.                   "name": "upvotes",
  181.                   "multiValueHandling": "SORTED_ARRAY",
  182.                   "createBitmapIndex": false
  183.               },
  184.               {
  185.                   "type": "long",
  186.                   "name": "comments",
  187.                   "multiValueHandling": "SORTED_ARRAY",
  188.                   "createBitmapIndex": false
  189.               },
  190.               {
  191.                   "type": "string",
  192.                   "name": "edited",
  193.                   "multiValueHandling": "SORTED_ARRAY",
  194.                   "createBitmapIndex": true
  195.               }
  196.           ],
  197.           "dimensionExclusions": [
  198.               "__time"
  199.           ],
  200.           "includeAllDimensions": false,
  201.           "useSchemaDiscovery": false
  202.       },
  203.       "metricsSpec": [],
  204.       "granularitySpec": {
  205.           "type": "uniform",
  206.           "segmentGranularity": "HOUR",
  207.           "queryGranularity": {
  208.               "type": "none"
  209.           },
  210.           "rollup": false,
  211.           "intervals": []
  212.       },
  213.       "transformSpec": {
  214.           "filter": null,
  215.           "transforms": []
  216.       }
  217.   },
  218.   "tuningConfig": {
  219.       "type": "kafka",
  220.       "appendableIndexSpec": {
  221.           "type": "onheap",
  222.           "preserveExistingMetrics": false
  223.       },
  224.       "maxRowsInMemory": 150000,
  225.       "maxBytesInMemory": 0,
  226.       "skipBytesInMemoryOverheadCheck": false,
  227.       "maxRowsPerSegment": 5000000,
  228.       "maxTotalRows": null,
  229.       "intermediatePersistPeriod": "PT10M",
  230.       "maxPendingPersists": 0,
  231.       "indexSpec": {
  232.           "bitmap": {
  233.               "type": "roaring"
  234.           },
  235.           "dimensionCompression": "lz4",
  236.           "stringDictionaryEncoding": {
  237.               "type": "utf8"
  238.           },
  239.           "metricCompression": "lz4",
  240.           "longEncoding": "longs"
  241.       },
  242.       "indexSpecForIntermediatePersists": {
  243.           "bitmap": {
  244.               "type": "roaring"
  245.           },
  246.           "dimensionCompression": "lz4",
  247.           "stringDictionaryEncoding": {
  248.               "type": "utf8"
  249.           },
  250.           "metricCompression": "lz4",
  251.           "longEncoding": "longs"
  252.       },
  253.       "reportParseExceptions": false,
  254.       "handoffConditionTimeout": 0,
  255.       "resetOffsetAutomatically": false,
  256.       "segmentWriteOutMediumFactory": null,
  257.       "workerThreads": null,
  258.       "chatRetries": 8,
  259.       "httpTimeout": "PT10S",
  260.       "shutdownTimeout": "PT80S",
  261.       "offsetFetchPeriod": "PT30S",
  262.       "intermediateHandoffPeriod": "P2147483647D",
  263.       "logParseExceptions": false,
  264.       "maxParseExceptions": 2147483647,
  265.       "maxSavedParseExceptions": 0,
  266.       "skipSequenceNumberAvailabilityCheck": false,
  267.       "repartitionTransitionDuration": "PT120S"
  268.   },
  269.   "ioConfig": {
  270.       "topic": "social_media",
  271.       "inputFormat": {
  272.           "type": "json",
  273.           "keepNullColumns": false,
  274.           "assumeNewlineDelimited": false,
  275.           "useJsonNodeReader": false
  276.       },
  277.       "replicas": 1,
  278.       "taskCount": 1,
  279.       "taskDuration": "PT3600S",
  280.       "consumerProperties": {
  281.           "bootstrap.servers": "localhost:9094"
  282.       },
  283.       "autoScalerConfig": null,
  284.       "pollTimeout": 100,
  285.       "startDelay": "PT5S",
  286.       "period": "PT30S",
  287.       "useEarliestOffset": true,
  288.       "completionTimeout": "PT1800S",
  289.       "lateMessageRejectionPeriod": null,
  290.       "earlyMessageRejectionPeriod": null,
  291.       "lateMessageRejectionStartDateTime": null,
  292.       "configOverrides": null,
  293.       "idleConfig": null,
  294.       "stream": "social_media",
  295.       "useEarliestSequenceNumber": true
  296.   },
  297.   "context": null,
  298.   "suspended": false
  299. }

Get supervisor status

Retrieves the current status report for a single supervisor. The report contains the state of the supervisor tasks and an array of recently thrown exceptions.

For additional information about the status report, see the topic for each streaming ingestion methods:

URL

GET /druid/indexer/v1/supervisor/:supervisorId/status

Responses

  • 200 SUCCESS
  • 404 NOT FOUND

Successfully retrieved supervisor status

Invalid supervisor ID

Sample request

The following example shows how to retrieve the status of a supervisor with the name social_media.

  • cURL
  • HTTP
  1. curl "http://ROUTER_IP:ROUTER_PORT/druid/indexer/v1/supervisor/social_media/status"
  1. GET /druid/indexer/v1/supervisor/social_media/status HTTP/1.1
  2. Host: http://ROUTER_IP:ROUTER_PORT

Sample response

Click to show sample response

  1. {
  2.     "id": "social_media",
  3.     "generationTime": "2023-07-05T23:24:43.934Z",
  4.     "payload": {
  5.         "dataSource": "social_media",
  6.         "stream": "social_media",
  7.         "partitions": 1,
  8.         "replicas": 1,
  9.         "durationSeconds": 3600,
  10.         "activeTasks": [
  11.             {
  12.                 "id": "index_kafka_social_media_ab72ae4127c591c_flcbhdlh",
  13.                 "startingOffsets": {
  14.                     "0": 3176381
  15.                 },
  16.                 "startTime": "2023-07-05T23:21:39.321Z",
  17.                 "remainingSeconds": 3415,
  18.                 "type": "ACTIVE",
  19.                 "currentOffsets": {
  20.                     "0": 3296632
  21.                 },
  22.                 "lag": {
  23.                     "0": 3
  24.                 }
  25.             }
  26.         ],
  27.         "publishingTasks": [],
  28.         "latestOffsets": {
  29.             "0": 3296635
  30.         },
  31.         "minimumLag": {
  32.             "0": 3
  33.         },
  34.         "aggregateLag": 3,
  35.         "offsetsLastUpdated": "2023-07-05T23:24:30.212Z",
  36.         "suspended": false,
  37.         "healthy": true,
  38.         "state": "RUNNING",
  39.         "detailedState": "RUNNING",
  40.         "recentErrors": []
  41.     }
  42. }

Audit history

An audit history provides a comprehensive log of events, including supervisor configuration, creation, suspension, and modification history.

Get audit history for all supervisors

Retrieve an audit history of specs for all supervisors.

URL

GET /druid/indexer/v1/supervisor/history

Responses

  • 200 SUCCESS

Successfully retrieved audit history

Sample request

  • cURL
  • HTTP
  1. curl "http://ROUTER_IP:ROUTER_PORT/druid/indexer/v1/supervisor/history"
  1. GET /druid/indexer/v1/supervisor/history HTTP/1.1
  2. Host: http://ROUTER_IP:ROUTER_PORT

Sample response

Click to show sample response

  1. {
  2.   "social_media": [
  3.       {
  4.           "spec": {
  5.               "type": "kafka",
  6.               "spec": {
  7.                   "dataSchema": {
  8.                       "dataSource": "social_media",
  9.                       "timestampSpec": {
  10.                           "column": "__time",
  11.                           "format": "iso",
  12.                           "missingValue": null
  13.                       },
  14.                       "dimensionsSpec": {
  15.                           "dimensions": [
  16.                               {
  17.                                   "type": "string",
  18.                                   "name": "username",
  19.                                   "multiValueHandling": "SORTED_ARRAY",
  20.                                   "createBitmapIndex": true
  21.                               },
  22.                               {
  23.                                   "type": "string",
  24.                                   "name": "post_title",
  25.                                   "multiValueHandling": "SORTED_ARRAY",
  26.                                   "createBitmapIndex": true
  27.                               },
  28.                               {
  29.                                   "type": "long",
  30.                                   "name": "views",
  31.                                   "multiValueHandling": "SORTED_ARRAY",
  32.                                   "createBitmapIndex": false
  33.                               },
  34.                               {
  35.                                   "type": "long",
  36.                                   "name": "upvotes",
  37.                                   "multiValueHandling": "SORTED_ARRAY",
  38.                                   "createBitmapIndex": false
  39.                               },
  40.                               {
  41.                                   "type": "long",
  42.                                   "name": "comments",
  43.                                   "multiValueHandling": "SORTED_ARRAY",
  44.                                   "createBitmapIndex": false
  45.                               },
  46.                               {
  47.                                   "type": "string",
  48.                                   "name": "edited",
  49.                                   "multiValueHandling": "SORTED_ARRAY",
  50.                                   "createBitmapIndex": true
  51.                               }
  52.                           ],
  53.                           "dimensionExclusions": [
  54.                               "__time"
  55.                           ],
  56.                           "includeAllDimensions": false,
  57.                           "useSchemaDiscovery": false
  58.                       },
  59.                       "metricsSpec": [],
  60.                       "granularitySpec": {
  61.                           "type": "uniform",
  62.                           "segmentGranularity": "HOUR",
  63.                           "queryGranularity": {
  64.                               "type": "none"
  65.                           },
  66.                           "rollup": false,
  67.                           "intervals": []
  68.                       },
  69.                       "transformSpec": {
  70.                           "filter": null,
  71.                           "transforms": []
  72.                       }
  73.                   },
  74.                   "ioConfig": {
  75.                       "topic": "social_media",
  76.                       "inputFormat": {
  77.                           "type": "json",
  78.                           "keepNullColumns": false,
  79.                           "assumeNewlineDelimited": false,
  80.                           "useJsonNodeReader": false
  81.                       },
  82.                       "replicas": 1,
  83.                       "taskCount": 1,
  84.                       "taskDuration": "PT3600S",
  85.                       "consumerProperties": {
  86.                           "bootstrap.servers": "localhost:9094"
  87.                       },
  88.                       "autoScalerConfig": null,
  89.                       "pollTimeout": 100,
  90.                       "startDelay": "PT5S",
  91.                       "period": "PT30S",
  92.                       "useEarliestOffset": true,
  93.                       "completionTimeout": "PT1800S",
  94.                       "lateMessageRejectionPeriod": null,
  95.                       "earlyMessageRejectionPeriod": null,
  96.                       "lateMessageRejectionStartDateTime": null,
  97.                       "configOverrides": null,
  98.                       "idleConfig": null,
  99.                       "stream": "social_media",
  100.                       "useEarliestSequenceNumber": true
  101.                   },
  102.                   "tuningConfig": {
  103.                       "type": "kafka",
  104.                       "appendableIndexSpec": {
  105.                           "type": "onheap",
  106.                           "preserveExistingMetrics": false
  107.                       },
  108.                       "maxRowsInMemory": 150000,
  109.                       "maxBytesInMemory": 0,
  110.                       "skipBytesInMemoryOverheadCheck": false,
  111.                       "maxRowsPerSegment": 5000000,
  112.                       "maxTotalRows": null,
  113.                       "intermediatePersistPeriod": "PT10M",
  114.                       "maxPendingPersists": 0,
  115.                       "indexSpec": {
  116.                           "bitmap": {
  117.                               "type": "roaring"
  118.                           },
  119.                           "dimensionCompression": "lz4",
  120.                           "stringDictionaryEncoding": {
  121.                               "type": "utf8"
  122.                           },
  123.                           "metricCompression": "lz4",
  124.                           "longEncoding": "longs"
  125.                       },
  126.                       "indexSpecForIntermediatePersists": {
  127.                           "bitmap": {
  128.                               "type": "roaring"
  129.                           },
  130.                           "dimensionCompression": "lz4",
  131.                           "stringDictionaryEncoding": {
  132.                               "type": "utf8"
  133.                           },
  134.                           "metricCompression": "lz4",
  135.                           "longEncoding": "longs"
  136.                       },
  137.                       "reportParseExceptions": false,
  138.                       "handoffConditionTimeout": 0,
  139.                       "resetOffsetAutomatically": false,
  140.                       "segmentWriteOutMediumFactory": null,
  141.                       "workerThreads": null,
  142.                       "chatRetries": 8,
  143.                       "httpTimeout": "PT10S",
  144.                       "shutdownTimeout": "PT80S",
  145.                       "offsetFetchPeriod": "PT30S",
  146.                       "intermediateHandoffPeriod": "P2147483647D",
  147.                       "logParseExceptions": false,
  148.                       "maxParseExceptions": 2147483647,
  149.                       "maxSavedParseExceptions": 0,
  150.                       "skipSequenceNumberAvailabilityCheck": false,
  151.                       "repartitionTransitionDuration": "PT120S"
  152.                   }
  153.               },
  154.               "dataSchema": {
  155.                   "dataSource": "social_media",
  156.                   "timestampSpec": {
  157.                       "column": "__time",
  158.                       "format": "iso",
  159.                       "missingValue": null
  160.                   },
  161.                   "dimensionsSpec": {
  162.                       "dimensions": [
  163.                           {
  164.                               "type": "string",
  165.                               "name": "username",
  166.                               "multiValueHandling": "SORTED_ARRAY",
  167.                               "createBitmapIndex": true
  168.                           },
  169.                           {
  170.                               "type": "string",
  171.                               "name": "post_title",
  172.                               "multiValueHandling": "SORTED_ARRAY",
  173.                               "createBitmapIndex": true
  174.                           },
  175.                           {
  176.                               "type": "long",
  177.                               "name": "views",
  178.                               "multiValueHandling": "SORTED_ARRAY",
  179.                               "createBitmapIndex": false
  180.                           },
  181.                           {
  182.                               "type": "long",
  183.                               "name": "upvotes",
  184.                               "multiValueHandling": "SORTED_ARRAY",
  185.                               "createBitmapIndex": false
  186.                           },
  187.                           {
  188.                               "type": "long",
  189.                               "name": "comments",
  190.                               "multiValueHandling": "SORTED_ARRAY",
  191.                               "createBitmapIndex": false
  192.                           },
  193.                           {
  194.                               "type": "string",
  195.                               "name": "edited",
  196.                               "multiValueHandling": "SORTED_ARRAY",
  197.                               "createBitmapIndex": true
  198.                           }
  199.                       ],
  200.                       "dimensionExclusions": [
  201.                           "__time"
  202.                       ],
  203.                       "includeAllDimensions": false,
  204.                       "useSchemaDiscovery": false
  205.                   },
  206.                   "metricsSpec": [],
  207.                   "granularitySpec": {
  208.                       "type": "uniform",
  209.                       "segmentGranularity": "HOUR",
  210.                       "queryGranularity": {
  211.                           "type": "none"
  212.                       },
  213.                       "rollup": false,
  214.                       "intervals": []
  215.                   },
  216.                   "transformSpec": {
  217.                       "filter": null,
  218.                       "transforms": []
  219.                   }
  220.               },
  221.               "tuningConfig": {
  222.                   "type": "kafka",
  223.                   "appendableIndexSpec": {
  224.                       "type": "onheap",
  225.                       "preserveExistingMetrics": false
  226.                   },
  227.                   "maxRowsInMemory": 150000,
  228.                   "maxBytesInMemory": 0,
  229.                   "skipBytesInMemoryOverheadCheck": false,
  230.                   "maxRowsPerSegment": 5000000,
  231.                   "maxTotalRows": null,
  232.                   "intermediatePersistPeriod": "PT10M",
  233.                   "maxPendingPersists": 0,
  234.                   "indexSpec": {
  235.                       "bitmap": {
  236.                           "type": "roaring"
  237.                       },
  238.                       "dimensionCompression": "lz4",
  239.                       "stringDictionaryEncoding": {
  240.                           "type": "utf8"
  241.                       },
  242.                       "metricCompression": "lz4",
  243.                       "longEncoding": "longs"
  244.                   },
  245.                   "indexSpecForIntermediatePersists": {
  246.                       "bitmap": {
  247.                           "type": "roaring"
  248.                       },
  249.                       "dimensionCompression": "lz4",
  250.                       "stringDictionaryEncoding": {
  251.                           "type": "utf8"
  252.                       },
  253.                       "metricCompression": "lz4",
  254.                       "longEncoding": "longs"
  255.                   },
  256.                   "reportParseExceptions": false,
  257.                   "handoffConditionTimeout": 0,
  258.                   "resetOffsetAutomatically": false,
  259.                   "segmentWriteOutMediumFactory": null,
  260.                   "workerThreads": null,
  261.                   "chatRetries": 8,
  262.                   "httpTimeout": "PT10S",
  263.                   "shutdownTimeout": "PT80S",
  264.                   "offsetFetchPeriod": "PT30S",
  265.                   "intermediateHandoffPeriod": "P2147483647D",
  266.                   "logParseExceptions": false,
  267.                   "maxParseExceptions": 2147483647,
  268.                   "maxSavedParseExceptions": 0,
  269.                   "skipSequenceNumberAvailabilityCheck": false,
  270.                   "repartitionTransitionDuration": "PT120S"
  271.               },
  272.               "ioConfig": {
  273.                   "topic": "social_media",
  274.                   "inputFormat": {
  275.                       "type": "json",
  276.                       "keepNullColumns": false,
  277.                       "assumeNewlineDelimited": false,
  278.                       "useJsonNodeReader": false
  279.                   },
  280.                   "replicas": 1,
  281.                   "taskCount": 1,
  282.                   "taskDuration": "PT3600S",
  283.                   "consumerProperties": {
  284.                       "bootstrap.servers": "localhost:9094"
  285.                   },
  286.                   "autoScalerConfig": null,
  287.                   "pollTimeout": 100,
  288.                   "startDelay": "PT5S",
  289.                   "period": "PT30S",
  290.                   "useEarliestOffset": true,
  291.                   "completionTimeout": "PT1800S",
  292.                   "lateMessageRejectionPeriod": null,
  293.                   "earlyMessageRejectionPeriod": null,
  294.                   "lateMessageRejectionStartDateTime": null,
  295.                   "configOverrides": null,
  296.                   "idleConfig": null,
  297.                   "stream": "social_media",
  298.                   "useEarliestSequenceNumber": true
  299.               },
  300.               "context": null,
  301.               "suspended": false
  302.           },
  303.           "version": "2023-07-03T18:51:02.970Z"
  304.       }
  305.   ]
  306. }

Get audit history for a specific supervisor

Retrieves an audit history of specs for a single supervisor.

URL

GET /druid/indexer/v1/supervisor/:supervisorId/history

Responses

  • 200 SUCCESS
  • 404 NOT FOUND

Successfully retrieved supervisor audit history

Invalid supervisor ID

Sample request

The following example shows how to retrieve the audit history of a supervisor with the name wikipedia_stream.

  • cURL
  • HTTP
  1. curl "http://ROUTER_IP:ROUTER_PORT/druid/indexer/v1/supervisor/wikipedia_stream/history"
  1. GET /druid/indexer/v1/supervisor/wikipedia_stream/history HTTP/1.1
  2. Host: http://ROUTER_IP:ROUTER_PORT

Sample response

Click to show sample response

  1. [
  2.   {
  3.       "spec": {
  4.           "type": "kafka",
  5.           "spec": {
  6.               "dataSchema": {
  7.                   "dataSource": "wikipedia_stream",
  8.                   "timestampSpec": {
  9.                       "column": "__time",
  10.                       "format": "iso",
  11.                       "missingValue": null
  12.                   },
  13.                   "dimensionsSpec": {
  14.                       "dimensions": [
  15.                           {
  16.                               "type": "string",
  17.                               "name": "username",
  18.                               "multiValueHandling": "SORTED_ARRAY",
  19.                               "createBitmapIndex": true
  20.                           },
  21.                           {
  22.                               "type": "string",
  23.                               "name": "post_title",
  24.                               "multiValueHandling": "SORTED_ARRAY",
  25.                               "createBitmapIndex": true
  26.                           },
  27.                           {
  28.                               "type": "long",
  29.                               "name": "views",
  30.                               "multiValueHandling": "SORTED_ARRAY",
  31.                               "createBitmapIndex": false
  32.                           },
  33.                           {
  34.                               "type": "long",
  35.                               "name": "upvotes",
  36.                               "multiValueHandling": "SORTED_ARRAY",
  37.                               "createBitmapIndex": false
  38.                           },
  39.                           {
  40.                               "type": "long",
  41.                               "name": "comments",
  42.                               "multiValueHandling": "SORTED_ARRAY",
  43.                               "createBitmapIndex": false
  44.                           },
  45.                           {
  46.                               "type": "string",
  47.                               "name": "edited",
  48.                               "multiValueHandling": "SORTED_ARRAY",
  49.                               "createBitmapIndex": true
  50.                           }
  51.                       ],
  52.                       "dimensionExclusions": [
  53.                           "__time"
  54.                       ],
  55.                       "includeAllDimensions": false,
  56.                       "useSchemaDiscovery": false
  57.                   },
  58.                   "metricsSpec": [],
  59.                   "granularitySpec": {
  60.                       "type": "uniform",
  61.                       "segmentGranularity": "HOUR",
  62.                       "queryGranularity": {
  63.                           "type": "none"
  64.                       },
  65.                       "rollup": false,
  66.                       "intervals": []
  67.                   },
  68.                   "transformSpec": {
  69.                       "filter": null,
  70.                       "transforms": []
  71.                   }
  72.               },
  73.               "ioConfig": {
  74.                   "topic": "social_media",
  75.                   "inputFormat": {
  76.                       "type": "json",
  77.                       "keepNullColumns": false,
  78.                       "assumeNewlineDelimited": false,
  79.                       "useJsonNodeReader": false
  80.                   },
  81.                   "replicas": 1,
  82.                   "taskCount": 1,
  83.                   "taskDuration": "PT3600S",
  84.                   "consumerProperties": {
  85.                       "bootstrap.servers": "localhost:9042"
  86.                   },
  87.                   "autoScalerConfig": null,
  88.                   "pollTimeout": 100,
  89.                   "startDelay": "PT5S",
  90.                   "period": "PT30S",
  91.                   "useEarliestOffset": true,
  92.                   "completionTimeout": "PT1800S",
  93.                   "lateMessageRejectionPeriod": null,
  94.                   "earlyMessageRejectionPeriod": null,
  95.                   "lateMessageRejectionStartDateTime": null,
  96.                   "configOverrides": null,
  97.                   "idleConfig": null,
  98.                   "stream": "social_media",
  99.                   "useEarliestSequenceNumber": true
  100.               },
  101.               "tuningConfig": {
  102.                   "type": "kafka",
  103.                   "appendableIndexSpec": {
  104.                       "type": "onheap",
  105.                       "preserveExistingMetrics": false
  106.                   },
  107.                   "maxRowsInMemory": 150000,
  108.                   "maxBytesInMemory": 0,
  109.                   "skipBytesInMemoryOverheadCheck": false,
  110.                   "maxRowsPerSegment": 5000000,
  111.                   "maxTotalRows": null,
  112.                   "intermediatePersistPeriod": "PT10M",
  113.                   "maxPendingPersists": 0,
  114.                   "indexSpec": {
  115.                       "bitmap": {
  116.                           "type": "roaring"
  117.                       },
  118.                       "dimensionCompression": "lz4",
  119.                       "stringDictionaryEncoding": {
  120.                           "type": "utf8"
  121.                       },
  122.                       "metricCompression": "lz4",
  123.                       "longEncoding": "longs"
  124.                   },
  125.                   "indexSpecForIntermediatePersists": {
  126.                       "bitmap": {
  127.                           "type": "roaring"
  128.                       },
  129.                       "dimensionCompression": "lz4",
  130.                       "stringDictionaryEncoding": {
  131.                           "type": "utf8"
  132.                       },
  133.                       "metricCompression": "lz4",
  134.                       "longEncoding": "longs"
  135.                   },
  136.                   "reportParseExceptions": false,
  137.                   "handoffConditionTimeout": 0,
  138.                   "resetOffsetAutomatically": false,
  139.                   "segmentWriteOutMediumFactory": null,
  140.                   "workerThreads": null,
  141.                   "chatRetries": 8,
  142.                   "httpTimeout": "PT10S",
  143.                   "shutdownTimeout": "PT80S",
  144.                   "offsetFetchPeriod": "PT30S",
  145.                   "intermediateHandoffPeriod": "P2147483647D",
  146.                   "logParseExceptions": false,
  147.                   "maxParseExceptions": 2147483647,
  148.                   "maxSavedParseExceptions": 0,
  149.                   "skipSequenceNumberAvailabilityCheck": false,
  150.                   "repartitionTransitionDuration": "PT120S"
  151.               }
  152.           },
  153.           "dataSchema": {
  154.               "dataSource": "wikipedia_stream",
  155.               "timestampSpec": {
  156.                   "column": "__time",
  157.                   "format": "iso",
  158.                   "missingValue": null
  159.               },
  160.               "dimensionsSpec": {
  161.                   "dimensions": [
  162.                       {
  163.                           "type": "string",
  164.                           "name": "username",
  165.                           "multiValueHandling": "SORTED_ARRAY",
  166.                           "createBitmapIndex": true
  167.                       },
  168.                       {
  169.                           "type": "string",
  170.                           "name": "post_title",
  171.                           "multiValueHandling": "SORTED_ARRAY",
  172.                           "createBitmapIndex": true
  173.                       },
  174.                       {
  175.                           "type": "long",
  176.                           "name": "views",
  177.                           "multiValueHandling": "SORTED_ARRAY",
  178.                           "createBitmapIndex": false
  179.                       },
  180.                       {
  181.                           "type": "long",
  182.                           "name": "upvotes",
  183.                           "multiValueHandling": "SORTED_ARRAY",
  184.                           "createBitmapIndex": false
  185.                       },
  186.                       {
  187.                           "type": "long",
  188.                           "name": "comments",
  189.                           "multiValueHandling": "SORTED_ARRAY",
  190.                           "createBitmapIndex": false
  191.                       },
  192.                       {
  193.                           "type": "string",
  194.                           "name": "edited",
  195.                           "multiValueHandling": "SORTED_ARRAY",
  196.                           "createBitmapIndex": true
  197.                       }
  198.                   ],
  199.                   "dimensionExclusions": [
  200.                       "__time"
  201.                   ],
  202.                   "includeAllDimensions": false,
  203.                   "useSchemaDiscovery": false
  204.               },
  205.               "metricsSpec": [],
  206.               "granularitySpec": {
  207.                   "type": "uniform",
  208.                   "segmentGranularity": "HOUR",
  209.                   "queryGranularity": {
  210.                       "type": "none"
  211.                   },
  212.                   "rollup": false,
  213.                   "intervals": []
  214.               },
  215.               "transformSpec": {
  216.                   "filter": null,
  217.                   "transforms": []
  218.               }
  219.           },
  220.           "tuningConfig": {
  221.               "type": "kafka",
  222.               "appendableIndexSpec": {
  223.                   "type": "onheap",
  224.                   "preserveExistingMetrics": false
  225.               },
  226.               "maxRowsInMemory": 150000,
  227.               "maxBytesInMemory": 0,
  228.               "skipBytesInMemoryOverheadCheck": false,
  229.               "maxRowsPerSegment": 5000000,
  230.               "maxTotalRows": null,
  231.               "intermediatePersistPeriod": "PT10M",
  232.               "maxPendingPersists": 0,
  233.               "indexSpec": {
  234.                   "bitmap": {
  235.                       "type": "roaring"
  236.                   },
  237.                   "dimensionCompression": "lz4",
  238.                   "stringDictionaryEncoding": {
  239.                       "type": "utf8"
  240.                   },
  241.                   "metricCompression": "lz4",
  242.                   "longEncoding": "longs"
  243.               },
  244.               "indexSpecForIntermediatePersists": {
  245.                   "bitmap": {
  246.                       "type": "roaring"
  247.                   },
  248.                   "dimensionCompression": "lz4",
  249.                   "stringDictionaryEncoding": {
  250.                       "type": "utf8"
  251.                   },
  252.                   "metricCompression": "lz4",
  253.                   "longEncoding": "longs"
  254.               },
  255.               "reportParseExceptions": false,
  256.               "handoffConditionTimeout": 0,
  257.               "resetOffsetAutomatically": false,
  258.               "segmentWriteOutMediumFactory": null,
  259.               "workerThreads": null,
  260.               "chatRetries": 8,
  261.               "httpTimeout": "PT10S",
  262.               "shutdownTimeout": "PT80S",
  263.               "offsetFetchPeriod": "PT30S",
  264.               "intermediateHandoffPeriod": "P2147483647D",
  265.               "logParseExceptions": false,
  266.               "maxParseExceptions": 2147483647,
  267.               "maxSavedParseExceptions": 0,
  268.               "skipSequenceNumberAvailabilityCheck": false,
  269.               "repartitionTransitionDuration": "PT120S"
  270.           },
  271.           "ioConfig": {
  272.               "topic": "social_media",
  273.               "inputFormat": {
  274.                   "type": "json",
  275.                   "keepNullColumns": false,
  276.                   "assumeNewlineDelimited": false,
  277.                   "useJsonNodeReader": false
  278.               },
  279.               "replicas": 1,
  280.               "taskCount": 1,
  281.               "taskDuration": "PT3600S",
  282.               "consumerProperties": {
  283.                   "bootstrap.servers": "localhost:9042"
  284.               },
  285.               "autoScalerConfig": null,
  286.               "pollTimeout": 100,
  287.               "startDelay": "PT5S",
  288.               "period": "PT30S",
  289.               "useEarliestOffset": true,
  290.               "completionTimeout": "PT1800S",
  291.               "lateMessageRejectionPeriod": null,
  292.               "earlyMessageRejectionPeriod": null,
  293.               "lateMessageRejectionStartDateTime": null,
  294.               "configOverrides": null,
  295.               "idleConfig": null,
  296.               "stream": "social_media",
  297.               "useEarliestSequenceNumber": true
  298.           },
  299.           "context": null,
  300.           "suspended": false
  301.       },
  302.       "version": "2023-07-05T20:59:16.872Z"
  303.   }
  304. ]

Manage supervisors

Create or update a supervisor

Creates a new supervisor or updates an existing one for the same datasource with a new schema and configuration.

You can define a supervisor spec for Apache Kafka or Amazon Kinesis streaming ingestion methods. Once created, the supervisor persists in the metadata database.

When you call this endpoint on an existing supervisor for the same datasource, the running supervisor signals its tasks to stop reading and begin publishing, exiting itself. Druid then uses the provided configuration from the request body to create a new supervisor. Druid submits a new schema while retaining existing publishing tasks and starts new tasks at the previous task offsets.

URL

POST /druid/indexer/v1/supervisor

Responses

  • 200 SUCCESS
  • 415 UNSUPPORTED MEDIA TYPE

Successfully created a new supervisor or updated an existing supervisor

Request body content type is not in JSON format

Sample request

The following example uses JSON input format to create a supervisor spec for Kafka with a social_media datasource and social_media topic.

  • cURL
  • HTTP
  1. curl "http://ROUTER_IP:ROUTER_PORT/druid/indexer/v1/supervisor" \
  2. --header 'Content-Type: application/json' \
  3. --data '{
  4.     "type": "kafka",
  5.     "spec": {
  6.         "ioConfig": {
  7.             "type": "kafka",
  8.             "consumerProperties": {
  9.                 "bootstrap.servers": "localhost:9094"
  10.             },
  11.             "topic": "social_media",
  12.             "inputFormat": {
  13.                 "type": "json"
  14.             },
  15.             "useEarliestOffset": true
  16.         },
  17.         "tuningConfig": {
  18.             "type": "kafka"
  19.         },
  20.         "dataSchema": {
  21.             "dataSource": "social_media",
  22.             "timestampSpec": {
  23.                 "column": "__time",
  24.                 "format": "iso"
  25.             },
  26.             "dimensionsSpec": {
  27.                 "dimensions": [
  28.                     "username",
  29.                     "post_title",
  30.                     {
  31.                         "type": "long",
  32.                         "name": "views"
  33.                     },
  34.                     {
  35.                         "type": "long",
  36.                         "name": "upvotes"
  37.                     },
  38.                     {
  39.                         "type": "long",
  40.                         "name": "comments"
  41.                     },
  42.                     "edited"
  43.                 ]
  44.             },
  45.             "granularitySpec": {
  46.                 "queryGranularity": "none",
  47.                 "rollup": false,
  48.                 "segmentGranularity": "hour"
  49.             }
  50.         }
  51.     }
  52. }'
  1. POST /druid/indexer/v1/supervisor HTTP/1.1
  2. Host: http://ROUTER_IP:ROUTER_PORT
  3. Content-Type: application/json
  4. Content-Length: 1359
  6. {
  7.     "type": "kafka",
  8.     "spec": {
  9.         "ioConfig": {
  10.             "type": "kafka",
  11.             "consumerProperties": {
  12.                 "bootstrap.servers": "localhost:9094"
  13.             },
  14.             "topic": "social_media",
  15.             "inputFormat": {
  16.                 "type": "json"
  17.             },
  18.             "useEarliestOffset": true
  19.         },
  20.         "tuningConfig": {
  21.             "type": "kafka"
  22.         },
  23.         "dataSchema": {
  24.             "dataSource": "social_media",
  25.             "timestampSpec": {
  26.                 "column": "__time",
  27.                 "format": "iso"
  28.             },
  29.             "dimensionsSpec": {
  30.                 "dimensions": [
  31.                     "username",
  32.                     "post_title",
  33.                     {
  34.                         "type": "long",
  35.                         "name": "views"
  36.                     },
  37.                     {
  38.                         "type": "long",
  39.                         "name": "upvotes"
  40.                     },
  41.                     {
  42.                         "type": "long",
  43.                         "name": "comments"
  44.                     },
  45.                     "edited"
  46.                 ]
  47.             },
  48.             "granularitySpec": {
  49.                 "queryGranularity": "none",
  50.                 "rollup": false,
  51.                 "segmentGranularity": "hour"
  52.             }
  53.         }
  54.     }
  55. }

Sample response

Click to show sample response

  1. {
  2.   "id": "social_media"
  3. }

Suspend a running supervisor

Suspends a single running supervisor. Returns the updated supervisor spec, where the suspended property is set to true. The suspended supervisor continues to emit logs and metrics.

URL

POST /druid/indexer/v1/supervisor/:supervisorId/suspend

Responses

  • 200 SUCCESS
  • 400 BAD REQUEST
  • 404 NOT FOUND

Successfully shut down supervisor

Supervisor already suspended

Invalid supervisor ID

Sample request

The following example shows how to suspend a running supervisor with the name social_media.

  • cURL
  • HTTP
  1. curl --request POST "http://ROUTER_IP:ROUTER_PORT/druid/indexer/v1/supervisor/social_media/suspend"
  1. POST /druid/indexer/v1/supervisor/social_media/suspend HTTP/1.1
  2. Host: http://ROUTER_IP:ROUTER_PORT

Sample response

Click to show sample response

  1. {
  2.   "type": "kafka",
  3.   "spec": {
  4.       "dataSchema": {
  5.           "dataSource": "social_media",
  6.           "timestampSpec": {
  7.               "column": "__time",
  8.               "format": "iso",
  9.               "missingValue": null
  10.           },
  11.           "dimensionsSpec": {
  12.               "dimensions": [
  13.                   {
  14.                       "type": "string",
  15.                       "name": "username",
  16.                       "multiValueHandling": "SORTED_ARRAY",
  17.                       "createBitmapIndex": true
  18.                   },
  19.                   {
  20.                       "type": "string",
  21.                       "name": "post_title",
  22.                       "multiValueHandling": "SORTED_ARRAY",
  23.                       "createBitmapIndex": true
  24.                   },
  25.                   {
  26.                       "type": "long",
  27.                       "name": "views",
  28.                       "multiValueHandling": "SORTED_ARRAY",
  29.                       "createBitmapIndex": false
  30.                   },
  31.                   {
  32.                       "type": "long",
  33.                       "name": "upvotes",
  34.                       "multiValueHandling": "SORTED_ARRAY",
  35.                       "createBitmapIndex": false
  36.                   },
  37.                   {
  38.                       "type": "long",
  39.                       "name": "comments",
  40.                       "multiValueHandling": "SORTED_ARRAY",
  41.                       "createBitmapIndex": false
  42.                   },
  43.                   {
  44.                       "type": "string",
  45.                       "name": "edited",
  46.                       "multiValueHandling": "SORTED_ARRAY",
  47.                       "createBitmapIndex": true
  48.                   }
  49.               ],
  50.               "dimensionExclusions": [
  51.                   "__time"
  52.               ],
  53.               "includeAllDimensions": false,
  54.               "useSchemaDiscovery": false
  55.           },
  56.           "metricsSpec": [],
  57.           "granularitySpec": {
  58.               "type": "uniform",
  59.               "segmentGranularity": "HOUR",
  60.               "queryGranularity": {
  61.                   "type": "none"
  62.               },
  63.               "rollup": false,
  64.               "intervals": []
  65.           },
  66.           "transformSpec": {
  67.               "filter": null,
  68.               "transforms": []
  69.           }
  70.       },
  71.       "ioConfig": {
  72.           "topic": "social_media",
  73.           "inputFormat": {
  74.               "type": "json",
  75.               "keepNullColumns": false,
  76.               "assumeNewlineDelimited": false,
  77.               "useJsonNodeReader": false
  78.           },
  79.           "replicas": 1,
  80.           "taskCount": 1,
  81.           "taskDuration": "PT3600S",
  82.           "consumerProperties": {
  83.               "bootstrap.servers": "localhost:9094"
  84.           },
  85.           "autoScalerConfig": null,
  86.           "pollTimeout": 100,
  87.           "startDelay": "PT5S",
  88.           "period": "PT30S",
  89.           "useEarliestOffset": true,
  90.           "completionTimeout": "PT1800S",
  91.           "lateMessageRejectionPeriod": null,
  92.           "earlyMessageRejectionPeriod": null,
  93.           "lateMessageRejectionStartDateTime": null,
  94.           "configOverrides": null,
  95.           "idleConfig": null,
  96.           "stream": "social_media",
  97.           "useEarliestSequenceNumber": true
  98.       },
  99.       "tuningConfig": {
  100.           "type": "kafka",
  101.           "appendableIndexSpec": {
  102.               "type": "onheap",
  103.               "preserveExistingMetrics": false
  104.           },
  105.           "maxRowsInMemory": 150000,
  106.           "maxBytesInMemory": 0,
  107.           "skipBytesInMemoryOverheadCheck": false,
  108.           "maxRowsPerSegment": 5000000,
  109.           "maxTotalRows": null,
  110.           "intermediatePersistPeriod": "PT10M",
  111.           "maxPendingPersists": 0,
  112.           "indexSpec": {
  113.               "bitmap": {
  114.                   "type": "roaring"
  115.               },
  116.               "dimensionCompression": "lz4",
  117.               "stringDictionaryEncoding": {
  118.                   "type": "utf8"
  119.               },
  120.               "metricCompression": "lz4",
  121.               "longEncoding": "longs"
  122.           },
  123.           "indexSpecForIntermediatePersists": {
  124.               "bitmap": {
  125.                   "type": "roaring"
  126.               },
  127.               "dimensionCompression": "lz4",
  128.               "stringDictionaryEncoding": {
  129.                   "type": "utf8"
  130.               },
  131.               "metricCompression": "lz4",
  132.               "longEncoding": "longs"
  133.           },
  134.           "reportParseExceptions": false,
  135.           "handoffConditionTimeout": 0,
  136.           "resetOffsetAutomatically": false,
  137.           "segmentWriteOutMediumFactory": null,
  138.           "workerThreads": null,
  139.           "chatRetries": 8,
  140.           "httpTimeout": "PT10S",
  141.           "shutdownTimeout": "PT80S",
  142.           "offsetFetchPeriod": "PT30S",
  143.           "intermediateHandoffPeriod": "P2147483647D",
  144.           "logParseExceptions": false,
  145.           "maxParseExceptions": 2147483647,
  146.           "maxSavedParseExceptions": 0,
  147.           "skipSequenceNumberAvailabilityCheck": false,
  148.           "repartitionTransitionDuration": "PT120S"
  149.       }
  150.   },
  151.   "dataSchema": {
  152.       "dataSource": "social_media",
  153.       "timestampSpec": {
  154.           "column": "__time",
  155.           "format": "iso",
  156.           "missingValue": null
  157.       },
  158.       "dimensionsSpec": {
  159.           "dimensions": [
  160.               {
  161.                   "type": "string",
  162.                   "name": "username",
  163.                   "multiValueHandling": "SORTED_ARRAY",
  164.                   "createBitmapIndex": true
  165.               },
  166.               {
  167.                   "type": "string",
  168.                   "name": "post_title",
  169.                   "multiValueHandling": "SORTED_ARRAY",
  170.                   "createBitmapIndex": true
  171.               },
  172.               {
  173.                   "type": "long",
  174.                   "name": "views",
  175.                   "multiValueHandling": "SORTED_ARRAY",
  176.                   "createBitmapIndex": false
  177.               },
  178.               {
  179.                   "type": "long",
  180.                   "name": "upvotes",
  181.                   "multiValueHandling": "SORTED_ARRAY",
  182.                   "createBitmapIndex": false
  183.               },
  184.               {
  185.                   "type": "long",
  186.                   "name": "comments",
  187.                   "multiValueHandling": "SORTED_ARRAY",
  188.                   "createBitmapIndex": false
  189.               },
  190.               {
  191.                   "type": "string",
  192.                   "name": "edited",
  193.                   "multiValueHandling": "SORTED_ARRAY",
  194.                   "createBitmapIndex": true
  195.               }
  196.           ],
  197.           "dimensionExclusions": [
  198.               "__time"
  199.           ],
  200.           "includeAllDimensions": false,
  201.           "useSchemaDiscovery": false
  202.       },
  203.       "metricsSpec": [],
  204.       "granularitySpec": {
  205.           "type": "uniform",
  206.           "segmentGranularity": "HOUR",
  207.           "queryGranularity": {
  208.               "type": "none"
  209.           },
  210.           "rollup": false,
  211.           "intervals": []
  212.       },
  213.       "transformSpec": {
  214.           "filter": null,
  215.           "transforms": []
  216.       }
  217.   },
  218.   "tuningConfig": {
  219.       "type": "kafka",
  220.       "appendableIndexSpec": {
  221.           "type": "onheap",
  222.           "preserveExistingMetrics": false
  223.       },
  224.       "maxRowsInMemory": 150000,
  225.       "maxBytesInMemory": 0,
  226.       "skipBytesInMemoryOverheadCheck": false,
  227.       "maxRowsPerSegment": 5000000,
  228.       "maxTotalRows": null,
  229.       "intermediatePersistPeriod": "PT10M",
  230.       "maxPendingPersists": 0,
  231.       "indexSpec": {
  232.           "bitmap": {
  233.               "type": "roaring"
  234.           },
  235.           "dimensionCompression": "lz4",
  236.           "stringDictionaryEncoding": {
  237.               "type": "utf8"
  238.           },
  239.           "metricCompression": "lz4",
  240.           "longEncoding": "longs"
  241.       },
  242.       "indexSpecForIntermediatePersists": {
  243.           "bitmap": {
  244.               "type": "roaring"
  245.           },
  246.           "dimensionCompression": "lz4",
  247.           "stringDictionaryEncoding": {
  248.               "type": "utf8"
  249.           },
  250.           "metricCompression": "lz4",
  251.           "longEncoding": "longs"
  252.       },
  253.       "reportParseExceptions": false,
  254.       "handoffConditionTimeout": 0,
  255.       "resetOffsetAutomatically": false,
  256.       "segmentWriteOutMediumFactory": null,
  257.       "workerThreads": null,
  258.       "chatRetries": 8,
  259.       "httpTimeout": "PT10S",
  260.       "shutdownTimeout": "PT80S",
  261.       "offsetFetchPeriod": "PT30S",
  262.       "intermediateHandoffPeriod": "P2147483647D",
  263.       "logParseExceptions": false,
  264.       "maxParseExceptions": 2147483647,
  265.       "maxSavedParseExceptions": 0,
  266.       "skipSequenceNumberAvailabilityCheck": false,
  267.       "repartitionTransitionDuration": "PT120S"
  268.   },
  269.   "ioConfig": {
  270.       "topic": "social_media",
  271.       "inputFormat": {
  272.           "type": "json",
  273.           "keepNullColumns": false,
  274.           "assumeNewlineDelimited": false,
  275.           "useJsonNodeReader": false
  276.       },
  277.       "replicas": 1,
  278.       "taskCount": 1,
  279.       "taskDuration": "PT3600S",
  280.       "consumerProperties": {
  281.           "bootstrap.servers": "localhost:9094"
  282.       },
  283.       "autoScalerConfig": null,
  284.       "pollTimeout": 100,
  285.       "startDelay": "PT5S",
  286.       "period": "PT30S",
  287.       "useEarliestOffset": true,
  288.       "completionTimeout": "PT1800S",
  289.       "lateMessageRejectionPeriod": null,
  290.       "earlyMessageRejectionPeriod": null,
  291.       "lateMessageRejectionStartDateTime": null,
  292.       "configOverrides": null,
  293.       "idleConfig": null,
  294.       "stream": "social_media",
  295.       "useEarliestSequenceNumber": true
  296.   },
  297.   "context": null,
  298.   "suspended": true
  299. }

Suspend all supervisors

Suspends all supervisors. Note that this endpoint returns an HTTP 200 Success code message even if there are no supervisors or running supervisors to suspend.

URL

POST /druid/indexer/v1/supervisor/suspendAll

Responses

  • 200 SUCCESS

Successfully suspended all supervisors

Sample request

  • cURL
  • HTTP
  1. curl --request POST "http://ROUTER_IP:ROUTER_PORT/druid/indexer/v1/supervisor/suspendAll"
  1. POST /druid/indexer/v1/supervisor/suspendAll HTTP/1.1
  2. Host: http://ROUTER_IP:ROUTER_PORT

Sample response

Click to show sample response

  1. {
  2.   "status": "success"
  3. }

Resume a supervisor

Resumes indexing tasks for a supervisor. Returns an updated supervisor spec with the suspended property set to false.

URL

POST /druid/indexer/v1/supervisor/:supervisorId/resume

Responses

  • 200 SUCCESS
  • 400 BAD REQUEST
  • 404 NOT FOUND

Successfully resumed supervisor

Supervisor already running

Invalid supervisor ID

Sample request

The following example resumes a previously suspended supervisor with name social_media.

  • cURL
  • HTTP
  1. curl --request POST "http://ROUTER_IP:ROUTER_PORT/druid/indexer/v1/supervisor/social_media/resume"
  1. POST /druid/indexer/v1/supervisor/social_media/resume HTTP/1.1
  2. Host: http://ROUTER_IP:ROUTER_PORT

Sample response

Click to show sample response

  1. {
  2.   "type": "kafka",
  3.   "spec": {
  4.       "dataSchema": {
  5.           "dataSource": "social_media",
  6.           "timestampSpec": {
  7.               "column": "__time",
  8.               "format": "iso",
  9.               "missingValue": null
  10.           },
  11.           "dimensionsSpec": {
  12.               "dimensions": [
  13.                   {
  14.                       "type": "string",
  15.                       "name": "username",
  16.                       "multiValueHandling": "SORTED_ARRAY",
  17.                       "createBitmapIndex": true
  18.                   },
  19.                   {
  20.                       "type": "string",
  21.                       "name": "post_title",
  22.                       "multiValueHandling": "SORTED_ARRAY",
  23.                       "createBitmapIndex": true
  24.                   },
  25.                   {
  26.                       "type": "long",
  27.                       "name": "views",
  28.                       "multiValueHandling": "SORTED_ARRAY",
  29.                       "createBitmapIndex": false
  30.                   },
  31.                   {
  32.                       "type": "long",
  33.                       "name": "upvotes",
  34.                       "multiValueHandling": "SORTED_ARRAY",
  35.                       "createBitmapIndex": false
  36.                   },
  37.                   {
  38.                       "type": "long",
  39.                       "name": "comments",
  40.                       "multiValueHandling": "SORTED_ARRAY",
  41.                       "createBitmapIndex": false
  42.                   },
  43.                   {
  44.                       "type": "string",
  45.                       "name": "edited",
  46.                       "multiValueHandling": "SORTED_ARRAY",
  47.                       "createBitmapIndex": true
  48.                   }
  49.               ],
  50.               "dimensionExclusions": [
  51.                   "__time"
  52.               ],
  53.               "includeAllDimensions": false,
  54.               "useSchemaDiscovery": false
  55.           },
  56.           "metricsSpec": [],
  57.           "granularitySpec": {
  58.               "type": "uniform",
  59.               "segmentGranularity": "HOUR",
  60.               "queryGranularity": {
  61.                   "type": "none"
  62.               },
  63.               "rollup": false,
  64.               "intervals": []
  65.           },
  66.           "transformSpec": {
  67.               "filter": null,
  68.               "transforms": []
  69.           }
  70.       },
  71.       "ioConfig": {
  72.           "topic": "social_media",
  73.           "inputFormat": {
  74.               "type": "json",
  75.               "keepNullColumns": false,
  76.               "assumeNewlineDelimited": false,
  77.               "useJsonNodeReader": false
  78.           },
  79.           "replicas": 1,
  80.           "taskCount": 1,
  81.           "taskDuration": "PT3600S",
  82.           "consumerProperties": {
  83.               "bootstrap.servers": "localhost:9094"
  84.           },
  85.           "autoScalerConfig": null,
  86.           "pollTimeout": 100,
  87.           "startDelay": "PT5S",
  88.           "period": "PT30S",
  89.           "useEarliestOffset": true,
  90.           "completionTimeout": "PT1800S",
  91.           "lateMessageRejectionPeriod": null,
  92.           "earlyMessageRejectionPeriod": null,
  93.           "lateMessageRejectionStartDateTime": null,
  94.           "configOverrides": null,
  95.           "idleConfig": null,
  96.           "stream": "social_media",
  97.           "useEarliestSequenceNumber": true
  98.       },
  99.       "tuningConfig": {
  100.           "type": "kafka",
  101.           "appendableIndexSpec": {
  102.               "type": "onheap",
  103.               "preserveExistingMetrics": false
  104.           },
  105.           "maxRowsInMemory": 150000,
  106.           "maxBytesInMemory": 0,
  107.           "skipBytesInMemoryOverheadCheck": false,
  108.           "maxRowsPerSegment": 5000000,
  109.           "maxTotalRows": null,
  110.           "intermediatePersistPeriod": "PT10M",
  111.           "maxPendingPersists": 0,
  112.           "indexSpec": {
  113.               "bitmap": {
  114.                   "type": "roaring"
  115.               },
  116.               "dimensionCompression": "lz4",
  117.               "stringDictionaryEncoding": {
  118.                   "type": "utf8"
  119.               },
  120.               "metricCompression": "lz4",
  121.               "longEncoding": "longs"
  122.           },
  123.           "indexSpecForIntermediatePersists": {
  124.               "bitmap": {
  125.                   "type": "roaring"
  126.               },
  127.               "dimensionCompression": "lz4",
  128.               "stringDictionaryEncoding": {
  129.                   "type": "utf8"
  130.               },
  131.               "metricCompression": "lz4",
  132.               "longEncoding": "longs"
  133.           },
  134.           "reportParseExceptions": false,
  135.           "handoffConditionTimeout": 0,
  136.           "resetOffsetAutomatically": false,
  137.           "segmentWriteOutMediumFactory": null,
  138.           "workerThreads": null,
  139.           "chatRetries": 8,
  140.           "httpTimeout": "PT10S",
  141.           "shutdownTimeout": "PT80S",
  142.           "offsetFetchPeriod": "PT30S",
  143.           "intermediateHandoffPeriod": "P2147483647D",
  144.           "logParseExceptions": false,
  145.           "maxParseExceptions": 2147483647,
  146.           "maxSavedParseExceptions": 0,
  147.           "skipSequenceNumberAvailabilityCheck": false,
  148.           "repartitionTransitionDuration": "PT120S"
  149.       }
  150.   },
  151.   "dataSchema": {
  152.       "dataSource": "social_media",
  153.       "timestampSpec": {
  154.           "column": "__time",
  155.           "format": "iso",
  156.           "missingValue": null
  157.       },
  158.       "dimensionsSpec": {
  159.           "dimensions": [
  160.               {
  161.                   "type": "string",
  162.                   "name": "username",
  163.                   "multiValueHandling": "SORTED_ARRAY",
  164.                   "createBitmapIndex": true
  165.               },
  166.               {
  167.                   "type": "string",
  168.                   "name": "post_title",
  169.                   "multiValueHandling": "SORTED_ARRAY",
  170.                   "createBitmapIndex": true
  171.               },
  172.               {
  173.                   "type": "long",
  174.                   "name": "views",
  175.                   "multiValueHandling": "SORTED_ARRAY",
  176.                   "createBitmapIndex": false
  177.               },
  178.               {
  179.                   "type": "long",
  180.                   "name": "upvotes",
  181.                   "multiValueHandling": "SORTED_ARRAY",
  182.                   "createBitmapIndex": false
  183.               },
  184.               {
  185.                   "type": "long",
  186.                   "name": "comments",
  187.                   "multiValueHandling": "SORTED_ARRAY",
  188.                   "createBitmapIndex": false
  189.               },
  190.               {
  191.                   "type": "string",
  192.                   "name": "edited",
  193.                   "multiValueHandling": "SORTED_ARRAY",
  194.                   "createBitmapIndex": true
  195.               }
  196.           ],
  197.           "dimensionExclusions": [
  198.               "__time"
  199.           ],
  200.           "includeAllDimensions": false,
  201.           "useSchemaDiscovery": false
  202.       },
  203.       "metricsSpec": [],
  204.       "granularitySpec": {
  205.           "type": "uniform",
  206.           "segmentGranularity": "HOUR",
  207.           "queryGranularity": {
  208.               "type": "none"
  209.           },
  210.           "rollup": false,
  211.           "intervals": []
  212.       },
  213.       "transformSpec": {
  214.           "filter": null,
  215.           "transforms": []
  216.       }
  217.   },
  218.   "tuningConfig": {
  219.       "type": "kafka",
  220.       "appendableIndexSpec": {
  221.           "type": "onheap",
  222.           "preserveExistingMetrics": false
  223.       },
  224.       "maxRowsInMemory": 150000,
  225.       "maxBytesInMemory": 0,
  226.       "skipBytesInMemoryOverheadCheck": false,
  227.       "maxRowsPerSegment": 5000000,
  228.       "maxTotalRows": null,
  229.       "intermediatePersistPeriod": "PT10M",
  230.       "maxPendingPersists": 0,
  231.       "indexSpec": {
  232.           "bitmap": {
  233.               "type": "roaring"
  234.           },
  235.           "dimensionCompression": "lz4",
  236.           "stringDictionaryEncoding": {
  237.               "type": "utf8"
  238.           },
  239.           "metricCompression": "lz4",
  240.           "longEncoding": "longs"
  241.       },
  242.       "indexSpecForIntermediatePersists": {
  243.           "bitmap": {
  244.               "type": "roaring"
  245.           },
  246.           "dimensionCompression": "lz4",
  247.           "stringDictionaryEncoding": {
  248.               "type": "utf8"
  249.           },
  250.           "metricCompression": "lz4",
  251.           "longEncoding": "longs"
  252.       },
  253.       "reportParseExceptions": false,
  254.       "handoffConditionTimeout": 0,
  255.       "resetOffsetAutomatically": false,
  256.       "segmentWriteOutMediumFactory": null,
  257.       "workerThreads": null,
  258.       "chatRetries": 8,
  259.       "httpTimeout": "PT10S",
  260.       "shutdownTimeout": "PT80S",
  261.       "offsetFetchPeriod": "PT30S",
  262.       "intermediateHandoffPeriod": "P2147483647D",
  263.       "logParseExceptions": false,
  264.       "maxParseExceptions": 2147483647,
  265.       "maxSavedParseExceptions": 0,
  266.       "skipSequenceNumberAvailabilityCheck": false,
  267.       "repartitionTransitionDuration": "PT120S"
  268.   },
  269.   "ioConfig": {
  270.       "topic": "social_media",
  271.       "inputFormat": {
  272.           "type": "json",
  273.           "keepNullColumns": false,
  274.           "assumeNewlineDelimited": false,
  275.           "useJsonNodeReader": false
  276.       },
  277.       "replicas": 1,
  278.       "taskCount": 1,
  279.       "taskDuration": "PT3600S",
  280.       "consumerProperties": {
  281.           "bootstrap.servers": "localhost:9094"
  282.       },
  283.       "autoScalerConfig": null,
  284.       "pollTimeout": 100,
  285.       "startDelay": "PT5S",
  286.       "period": "PT30S",
  287.       "useEarliestOffset": true,
  288.       "completionTimeout": "PT1800S",
  289.       "lateMessageRejectionPeriod": null,
  290.       "earlyMessageRejectionPeriod": null,
  291.       "lateMessageRejectionStartDateTime": null,
  292.       "configOverrides": null,
  293.       "idleConfig": null,
  294.       "stream": "social_media",
  295.       "useEarliestSequenceNumber": true
  296.   },
  297.   "context": null,
  298.   "suspended": false
  299. }

Resume all supervisors

Resumes all supervisors. Note that this endpoint returns an HTTP 200 Success code even if there are no supervisors or suspended supervisors to resume.

URL

POST /druid/indexer/v1/supervisor/resumeAll

Responses

  • 200 SUCCESS

Successfully resumed all supervisors

Sample request

  • cURL
  • HTTP
  1. curl --request POST "http://ROUTER_IP:ROUTER_PORT/druid/indexer/v1/supervisor/resumeAll"
  1. POST /druid/indexer/v1/supervisor/resumeAll HTTP/1.1
  2. Host: http://ROUTER_IP:ROUTER_PORT

Sample response

Click to show sample response

  1. {
  2.   "status": "success"
  3. }

Reset a supervisor

Resets the specified supervisor. This endpoint clears all stored offsets in Kafka or sequence numbers in Kinesis, prompting the supervisor to resume data reading. The supervisor will start from the earliest or latest available position, depending on the platform (offsets in Kafka or sequence numbers in Kinesis). It kills and recreates active tasks to read from valid positions.

Use this endpoint to recover from a stopped state due to missing offsets in Kafka or sequence numbers in Kinesis. Use this endpoint with caution as it may result in skipped messages and lead to data loss or duplicate data.

URL

POST /druid/indexer/v1/supervisor/:supervisorId/reset

Responses

  • 200 SUCCESS
  • 404 NOT FOUND

Successfully reset supervisor

Invalid supervisor ID

Sample request

The following example shows how to reset a supervisor with the name social_media.

  • cURL
  • HTTP
  1. curl --request POST "http://ROUTER_IP:ROUTER_PORT/druid/indexer/v1/supervisor/social_media/reset"
  1. POST /druid/indexer/v1/supervisor/social_media/reset HTTP/1.1
  2. Host: http://ROUTER_IP:ROUTER_PORT

Sample response

Click to show sample response

  1. {
  2.   "id": "social_media"
  3. }

Reset Offsets for a supervisor

Resets the specified offsets for a supervisor. This endpoint clears only the specified offsets in Kafka or sequence numbers in Kinesis, prompting the supervisor to resume data reading. If there are no stored offsets, the specified offsets will be set in the metadata store. The supervisor will start from the reset offsets for the partitions specified and for the other partitions from the stored offset. It kills and recreates active tasks pertaining to the partitions specified to read from valid offsets.

Use this endpoint to selectively reset offsets for partitions without resetting the entire set.

URL

POST /druid/indexer/v1/supervisor/:supervisorId/resetOffsets

Responses

Successfully reset offsets

Invalid supervisor ID

Reset Offsets Metadata

This section presents the structure and details of the reset offsets metadata payload.

FieldTypeDescriptionRequired
typeStringThe type of reset offsets metadata payload. It must match the supervisor’s type. Possible values: kafka or kinesis.Yes
partitionsObjectAn object representing the reset metadata. See below for details.Yes

Partitions

The following table defines the fields within the partitions object in the reset offsets metadata payload.

FieldTypeDescriptionRequired
typeStringMust be set as end. Indicates the end sequence numbers for the reset offsets.Yes
streamStringThe stream to be reset. It must be a valid stream consumed by the supervisor.Yes
partitionOffsetMapObjectA map of partitions to corresponding offsets for the stream to be reset.Yes

Sample request

The following example shows how to reset offsets for a kafka supervisor with the name social_media. Let’s say the supervisor is reading from a kafka topic ads_media_stream and has the stored offsets: {"0": 0, "1": 10, "2": 20, "3": 40}.

  1. curl --request POST "http://ROUTER_IP:ROUTER_PORT/druid/indexer/v1/supervisor/social_media/resetOffsets"
  2. --header 'Content-Type: application/json'
  3. --data-raw '{"type":"kafka","partitions":{"type":"end","stream":"ads_media_stream","partitionOffsetMap":{"0":100, "2": 650}}}'
  1. POST /druid/indexer/v1/supervisor/social_media/resetOffsets HTTP/1.1
  2. Host: http://ROUTER_IP:ROUTER_PORT
  3. Content-Type: application/json
  5. {
  6.   "type": "kafka",
  7.   "partitions": {
  8.     "type": "end",
  9.     "stream": "ads_media_stream",
  10.     "partitionOffsetMap": {
  11.       "0": 100,
  12.       "2": 650
  13.     }
  14.   }
  15. }

The above operation will reset offsets only for partitions 0 and 2 to 100 and 650 respectively. After a successful reset, when the supervisor’s tasks restart, they will resume reading from {"0": 100, "1": 10, "2": 650, "3": 40}.

Sample response

Click to show sample response

  1. {
  2.   "id": "social_media"
  3. }

Terminate a supervisor

Terminates a supervisor and its associated indexing tasks, triggering the publishing of their segments. When terminated, a tombstone marker is placed in the database to prevent reloading on restart.

The terminated supervisor still exists in the metadata store and its history can be retrieved.

URL

POST /druid/indexer/v1/supervisor/:supervisorId/terminate

Responses

  • 200 SUCCESS
  • 404 NOT FOUND

Successfully terminated a supervisor

Invalid supervisor ID or supervisor not running

Sample request

  • cURL
  • HTTP
  1. curl --request POST "http://ROUTER_IP:ROUTER_PORT/druid/indexer/v1/supervisor/social_media/terminate"
  1. POST /druid/indexer/v1/supervisor/social_media/terminate HTTP/1.1
  2. Host: http://ROUTER_IP:ROUTER_PORT

Sample response

Click to show sample response

  1. {
  2.   "id": "social_media"
  3. }

Terminate all supervisors

Terminates all supervisors. Terminated supervisors still exist in the metadata store and their history can be retrieved. Note that this endpoint returns an HTTP 200 Success code even if there are no supervisors or running supervisors to terminate.

URL

POST /druid/indexer/v1/supervisor/terminateAll

Responses

  • 200 SUCCESS

Successfully terminated all supervisors

Sample request

  • cURL
  • HTTP
  1. curl --request POST "http://ROUTER_IP:ROUTER_PORT/druid/indexer/v1/supervisor/terminateAll"
  1. POST /druid/indexer/v1/supervisor/terminateAll HTTP/1.1
  2. Host: http://ROUTER_IP:ROUTER_PORT

Sample response

Click to show sample response

  1. {
  2.   "status": "success"
  3. }

Shut down a supervisor

Shuts down a supervisor. This endpoint is deprecated and will be removed in future releases. Use the equivalent terminate endpoint instead.

URL

POST /druid/indexer/v1/supervisor/:supervisorId/shutdown