Profile

Introduced 1.0

The Profile API provides timing information about the execution of individual components of a search request. Using the Profile API, you can debug slow requests and understand how to improve their performance. The Profile API does not measure the following:

  • Network latency
  • Time spent in the search fetch phase
  • Amount of time a request spends in queues
  • Idle time while merging shard responses on the coordinating node

The Profile API is a resource-consuming operation that adds overhead to search operations.

Path and HTTP methods

  1. GET /testindex/_search
  2. {
  3. "profile": true,
  4. "query" : {
  5. "match" : { "title" : "wind" }
  6. }
  7. }

Starting in OpenSearch 2.12, concurrent segment search allows each shard-level request to search segments in parallel during the query phase. The Profile API response contains several additional fields with statistics about slices.

A slice is the unit of work that can be executed by a thread. Each query can be partitioned into multiple slices, with each slice containing one or more segments. All the slices can be executed either in parallel or in some order depending on the available threads in the pool.

In general, the max/min/avg slice time captures statistics across all slices for a timing type. For example, when profiling aggregations, the max_slice_time_in_nanos field in the aggregations section shows the maximum time consumed by the aggregation operation and its children across all slices.

Example requests

To use the Profile API, include the profile parameter set to true in the search request sent to the _search endpoint:

  1. GET /testindex/_search
  2. {
  3. "profile": true,
  4. "query" : {
  5. "match" : { "title" : "wind" }
  6. }
  7. }

copy

To turn on human-readable format, include the ?human=true query parameter in the request:

  1. GET /testindex/_search?human=true
  2. {
  3. "profile": true,
  4. "query" : {
  5. "match" : { "title" : "wind" }
  6. }
  7. }

copy

The response contains an additional time field with human-readable units, for example:

  1. "collector": [
  2. {
  3. "name": "SimpleTopScoreDocCollector",
  4. "reason": "search_top_hits",
  5. "time": "113.7micros",
  6. "time_in_nanos": 113711
  7. }
  8. ]

The Profile API response is verbose, so if you’re running the request through the curl command, include the ?pretty query parameter to make the response easier to understand.

Aggregations

To profile aggregations, send an aggregation request and provide the profile parameter set to true.

Global aggregation

  1. GET /opensearch_dashboards_sample_data_ecommerce/_search
  2. {
  3. "profile": "true",
  4. "size": 0,
  5. "query": {
  6. "match": { "manufacturer": "Elitelligence" }
  7. },
  8. "aggs": {
  9. "all_products": {
  10. "global": {},
  11. "aggs": {
  12. "avg_price": { "avg": { "field": "taxful_total_price" } }
  13. }
  14. },
  15. "elitelligence_products": { "avg": { "field": "taxful_total_price" } }
  16. }
  17. }

copy

Non-global aggregation

  1. GET /opensearch_dashboards_sample_data_ecommerce/_search
  2. {
  3. "size": 0,
  4. "aggs": {
  5. "avg_taxful_total_price": {
  6. "avg": {
  7. "field": "taxful_total_price"
  8. }
  9. }
  10. }
  11. }

copy

Example response

The response contains profiling information:

Response

  1. {
  2. "took": 21,
  3. "timed_out": false,
  4. "_shards": {
  5. "total": 1,
  6. "successful": 1,
  7. "skipped": 0,
  8. "failed": 0
  9. },
  10. "hits": {
  11. "total": {
  12. "value": 2,
  13. "relation": "eq"
  14. },
  15. "max_score": 0.19363807,
  16. "hits": [
  17. {
  18. "_index": "testindex",
  19. "_id": "1",
  20. "_score": 0.19363807,
  21. "_source": {
  22. "title": "The wind rises"
  23. }
  24. },
  25. {
  26. "_index": "testindex",
  27. "_id": "2",
  28. "_score": 0.17225474,
  29. "_source": {
  30. "title": "Gone with the wind",
  31. "description": "A 1939 American epic historical film"
  32. }
  33. }
  34. ]
  35. },
  36. "profile": {
  37. "shards": [
  38. {
  39. "id": "[LidyZ1HVS-u93-73Z49dQg][testindex][0]",
  40. "inbound_network_time_in_millis": 0,
  41. "outbound_network_time_in_millis": 0,
  42. "searches": [
  43. {
  44. "query": [
  45. {
  46. "type": "BooleanQuery",
  47. "description": "title:wind title:rise",
  48. "time_in_nanos": 2473919,
  49. "breakdown": {
  50. "set_min_competitive_score_count": 0,
  51. "match_count": 0,
  52. "shallow_advance_count": 0,
  53. "set_min_competitive_score": 0,
  54. "next_doc": 5209,
  55. "match": 0,
  56. "next_doc_count": 2,
  57. "score_count": 2,
  58. "compute_max_score_count": 0,
  59. "compute_max_score": 0,
  60. "advance": 9209,
  61. "advance_count": 2,
  62. "score": 20751,
  63. "build_scorer_count": 4,
  64. "create_weight": 1404458,
  65. "shallow_advance": 0,
  66. "create_weight_count": 1,
  67. "build_scorer": 1034292
  68. },
  69. "children": [
  70. {
  71. "type": "TermQuery",
  72. "description": "title:wind",
  73. "time_in_nanos": 813581,
  74. "breakdown": {
  75. "set_min_competitive_score_count": 0,
  76. "match_count": 0,
  77. "shallow_advance_count": 0,
  78. "set_min_competitive_score": 0,
  79. "next_doc": 3291,
  80. "match": 0,
  81. "next_doc_count": 2,
  82. "score_count": 2,
  83. "compute_max_score_count": 0,
  84. "compute_max_score": 0,
  85. "advance": 7208,
  86. "advance_count": 2,
  87. "score": 18666,
  88. "build_scorer_count": 6,
  89. "create_weight": 616375,
  90. "shallow_advance": 0,
  91. "create_weight_count": 1,
  92. "build_scorer": 168041
  93. }
  94. },
  95. {
  96. "type": "TermQuery",
  97. "description": "title:rise",
  98. "time_in_nanos": 191083,
  99. "breakdown": {
  100. "set_min_competitive_score_count": 0,
  101. "match_count": 0,
  102. "shallow_advance_count": 0,
  103. "set_min_competitive_score": 0,
  104. "next_doc": 0,
  105. "match": 0,
  106. "next_doc_count": 0,
  107. "score_count": 0,
  108. "compute_max_score_count": 0,
  109. "compute_max_score": 0,
  110. "advance": 0,
  111. "advance_count": 0,
  112. "score": 0,
  113. "build_scorer_count": 2,
  114. "create_weight": 188625,
  115. "shallow_advance": 0,
  116. "create_weight_count": 1,
  117. "build_scorer": 2458
  118. }
  119. }
  120. ]
  121. }
  122. ],
  123. "rewrite_time": 192417,
  124. "collector": [
  125. {
  126. "name": "SimpleTopScoreDocCollector",
  127. "reason": "search_top_hits",
  128. "time_in_nanos": 77291
  129. }
  130. ]
  131. }
  132. ],
  133. "aggregations": []
  134. }
  135. ]
  136. }
  137. }

The following is an example response for a concurrent segment search with three segment slices:

Response

  1. {
  2. "took": 10,
  3. "timed_out": false,
  4. "_shards": {
  5. "total": 1,
  6. "successful": 1,
  7. "skipped": 0,
  8. "failed": 0
  9. },
  10. "hits": {
  11. "total": {
  12. "value": 5,
  13. "relation": "eq"
  14. },
  15. "max_score": 1.0,
  16. "hits": [
  17. ...
  18. ]
  19. },
  20. "aggregations": {
  21. ...
  22. },
  23. "profile": {
  24. "shards": [
  25. {
  26. "id": "[9Y7lbpaWRhyr5Y-41Zl48g][idx][0]",
  27. "inbound_network_time_in_millis": 0,
  28. "outbound_network_time_in_millis": 0,
  29. "searches": [
  30. {
  31. "query": [
  32. {
  33. "type": "MatchAllDocsQuery",
  34. "description": "*:*",
  35. "time_in_nanos": 868000,
  36. "max_slice_time_in_nanos": 19376,
  37. "min_slice_time_in_nanos": 12250,
  38. "avg_slice_time_in_nanos": 16847,
  39. "breakdown": {
  40. "max_match": 0,
  41. "set_min_competitive_score_count": 0,
  42. "match_count": 0,
  43. "avg_score_count": 1,
  44. "shallow_advance_count": 0,
  45. "next_doc": 29708,
  46. "min_build_scorer": 3125,
  47. "score_count": 5,
  48. "compute_max_score_count": 0,
  49. "advance": 0,
  50. "min_set_min_competitive_score": 0,
  51. "min_advance": 0,
  52. "score": 29250,
  53. "avg_set_min_competitive_score_count": 0,
  54. "min_match_count": 0,
  55. "avg_score": 333,
  56. "max_next_doc_count": 3,
  57. "max_compute_max_score_count": 0,
  58. "avg_shallow_advance": 0,
  59. "max_shallow_advance_count": 0,
  60. "set_min_competitive_score": 0,
  61. "min_build_scorer_count": 2,
  62. "next_doc_count": 8,
  63. "min_match": 0,
  64. "avg_next_doc": 888,
  65. "compute_max_score": 0,
  66. "min_set_min_competitive_score_count": 0,
  67. "max_build_scorer": 5791,
  68. "avg_match_count": 0,
  69. "avg_advance": 0,
  70. "build_scorer_count": 6,
  71. "avg_build_scorer_count": 2,
  72. "min_next_doc_count": 2,
  73. "min_shallow_advance_count": 0,
  74. "max_score_count": 2,
  75. "avg_match": 0,
  76. "avg_compute_max_score": 0,
  77. "max_advance": 0,
  78. "avg_shallow_advance_count": 0,
  79. "avg_set_min_competitive_score": 0,
  80. "avg_compute_max_score_count": 0,
  81. "avg_build_scorer": 4027,
  82. "max_set_min_competitive_score_count": 0,
  83. "advance_count": 0,
  84. "max_build_scorer_count": 2,
  85. "shallow_advance": 0,
  86. "min_compute_max_score": 0,
  87. "max_match_count": 0,
  88. "create_weight_count": 1,
  89. "build_scorer": 32459,
  90. "max_set_min_competitive_score": 0,
  91. "max_compute_max_score": 0,
  92. "min_shallow_advance": 0,
  93. "match": 0,
  94. "max_shallow_advance": 0,
  95. "avg_advance_count": 0,
  96. "min_next_doc": 708,
  97. "max_advance_count": 0,
  98. "min_score": 291,
  99. "max_next_doc": 999,
  100. "create_weight": 1834,
  101. "avg_next_doc_count": 2,
  102. "max_score": 376,
  103. "min_compute_max_score_count": 0,
  104. "min_score_count": 1,
  105. "min_advance_count": 0
  106. }
  107. }
  108. ],
  109. "rewrite_time": 8126,
  110. "collector": [
  111. {
  112. "name": "QueryCollectorManager",
  113. "reason": "search_multi",
  114. "time_in_nanos": 564708,
  115. "reduce_time_in_nanos": 1251042,
  116. "max_slice_time_in_nanos": 121959,
  117. "min_slice_time_in_nanos": 28958,
  118. "avg_slice_time_in_nanos": 83208,
  119. "slice_count": 3,
  120. "children": [
  121. {
  122. "name": "SimpleTopDocsCollectorManager",
  123. "reason": "search_top_hits",
  124. "time_in_nanos": 500459,
  125. "reduce_time_in_nanos": 840125,
  126. "max_slice_time_in_nanos": 22168,
  127. "min_slice_time_in_nanos": 5792,
  128. "avg_slice_time_in_nanos": 12084,
  129. "slice_count": 3
  130. },
  131. {
  132. "name": "NonGlobalAggCollectorManager: [histo]",
  133. "reason": "aggregation",
  134. "time_in_nanos": 552167,
  135. "reduce_time_in_nanos": 311292,
  136. "max_slice_time_in_nanos": 95333,
  137. "min_slice_time_in_nanos": 18416,
  138. "avg_slice_time_in_nanos": 66249,
  139. "slice_count": 3
  140. }
  141. ]
  142. }
  143. ]
  144. }
  145. ],
  146. "aggregations": [
  147. {
  148. "type": "NumericHistogramAggregator",
  149. "description": "histo",
  150. "time_in_nanos": 2847834,
  151. "max_slice_time_in_nanos": 117374,
  152. "min_slice_time_in_nanos": 20624,
  153. "avg_slice_time_in_nanos": 75597,
  154. "breakdown": {
  155. "min_build_leaf_collector": 9500,
  156. "build_aggregation_count": 3,
  157. "post_collection": 3209,
  158. "max_collect_count": 2,
  159. "initialize_count": 3,
  160. "reduce_count": 0,
  161. "avg_collect": 17055,
  162. "max_build_aggregation": 26000,
  163. "avg_collect_count": 1,
  164. "max_build_leaf_collector": 64833,
  165. "min_build_leaf_collector_count": 1,
  166. "build_aggregation": 41125,
  167. "min_initialize": 583,
  168. "max_reduce": 0,
  169. "build_leaf_collector_count": 3,
  170. "avg_reduce": 0,
  171. "min_collect_count": 1,
  172. "avg_build_leaf_collector_count": 1,
  173. "avg_build_leaf_collector": 45000,
  174. "max_collect": 24625,
  175. "reduce": 0,
  176. "avg_build_aggregation": 12013,
  177. "min_post_collection": 292,
  178. "max_initialize": 1333,
  179. "max_post_collection": 750,
  180. "collect_count": 5,
  181. "avg_post_collection": 541,
  182. "avg_initialize": 986,
  183. "post_collection_count": 3,
  184. "build_leaf_collector": 86833,
  185. "min_collect": 6250,
  186. "min_build_aggregation": 3541,
  187. "initialize": 2786791,
  188. "max_build_leaf_collector_count": 1,
  189. "min_reduce": 0,
  190. "collect": 29834
  191. },
  192. "debug": {
  193. "total_buckets": 1
  194. }
  195. }
  196. ]
  197. }
  198. ]
  199. }
  200. }

Response body fields

The response includes the following fields.

FieldData typeDescription
profileObjectContains profiling information.
profile.shardsArray of objectsA search request can be executed against one or more shards in the index, and a search may involve one or more indexes. Thus, the profile.shards array contains profiling information for each shard that was involved in the search.
profile.shards.idStringThe shard ID of the shard in the [node-ID][index-name][shard-ID] format.
profile.shards.searchesArray of objectsA search represents a query executed against the underlying Lucene index. Most search requests execute a single search against a Lucene index, but some search requests can execute more than one search. For example, including a global aggregation results in a secondary match_all query for the global context. The profile.shards array contains profiling information about each search execution.
profile.shards.searches.queryArray of objectsProfiling information about the query execution.
profile.shards.searches.rewrite_timeIntegerAll Lucene queries are rewritten. A query and its children may be rewritten more than once, until the query stops changing. The rewriting process involves performing optimizations, such as removing redundant clauses or replacing a query path with a more efficient one. After the rewriting process, the original query may change significantly. The rewrite_time field contains the cumulative total rewrite time for the query and all its children, in nanoseconds.
profile.shards.searches.collectorArray of objectsProfiling information about the Lucene collectors that ran the search.
profile.shards.aggregationsArray of objectsProfiling information about the aggregation execution.

The query array

The query array contains objects with the following fields.

FieldData typeDescription
typeStringThe Lucene query type into which the search query was rewritten. Corresponds to the Lucene class name (which often has the same name in OpenSearch).
descriptionStringContains a Lucene explanation of the query. Helps differentiate queries with the same type.
time_in_nanosLongThe total elapsed time for this query, in nanoseconds. For concurrent segment search, time_in_nanos is the total time spent across all the slices (the difference between the last completed slice execution end time and the first slice execution start time).
max_slice_time_in_nanosLongThe maximum amount of time taken by any slice to run a query, in nanoseconds. This field is included only if you enable concurrent segment search.
min_slice_time_in_nanosLongThe minimum amount of time taken by any slice to run a query, in nanoseconds. This field is included only if you enable concurrent segment search.
avg_slice_time_in_nanosLongThe average amount of time taken by any slice to run a query, in nanoseconds. This field is included only if you enable concurrent segment search.
breakdownObjectContains timing statistics about low-level Lucene execution.
childrenArray of objectsIf a query has subqueries (children), this field contains information about the subqueries.

The breakdown object

The breakdown object represents the timing statistics about low-level Lucene execution, broken down by method. Timings are listed in wall-clock nanoseconds and are not normalized. The breakdown timings are inclusive of all child times. The breakdown object comprises the following fields. All fields contain integer values.

FieldDescription
createweightA Query object in Lucene is immutable. Yet, Lucene should be able to reuse Query objects in multiple IndexSearcher objects. Thus, Query objects need to keep temporary state and statistics associated with the index in which the query is executed. To achieve reuse, every Query object generates a Weight object, which keeps the temporary context (state) associated with the <IndexSearcher, Query> tuple. The create_weight field contains the amount of time spent creating the Weight object.
build_scorerA Scorer iterates over matching documents and generates a score for each document. The build_scorer field contains the amount of time spent generating the Scorer object. This does not include the time spent scoring the documents. The Scorer initialization time depends on the optimization and complexity of a particular query. The build_scorer parameter also includes the amount of time associated with caching, if caching is applicable and enabled for the query.
next_docThe next_doc Lucene method returns the document ID of the next document that matches the query. This method is a special type of the advance method and is equivalent to advance(docId() + 1). The next_doc method is more convenient for many Lucene queries. The next_doc field contains the amount of time required to determine the next matching document, which varies depending on the query type.
advanceThe advance method is a lower-level version of the next_doc method in Lucene. It also finds the next matching document but necessitates that the calling query perform additional tasks, such as identifying skips. Some queries, such as conjunctions (must clauses in Boolean queries), cannot use next_doc. For those queries, advance is timed.
matchFor some queries, document matching is performed in two steps. First, the document is matched approximately. Second, those documents that are approximately matched are examined through a more comprehensive process. For example, a phrase query first checks whether a document contains all terms in the phrase. Next, it verifies that the terms are in order (which is a more expensive process). The match field is non-zero only for those queries that use the two-step verification process.
scoreContains the time taken for a Scorer to score a particular document.
shallow_advanceContains the amount of time required to execute the advanceShallow Lucene method.
compute_max_scoreContains the amount of time required to execute the getMaxScore Lucene method.
set_min_competitive_scoreContains the amount of time required to execute the setMinCompetitiveScore Lucene method.
<method>_countContains the number of invocations of a <method>. For example, advance_count contains the number of invocations of the advance method. Different invocations of the same method occur because the method is called on different documents. You can determine the selectivity of a query by comparing counts in different query components.
max<method>The maximum amount of time taken by any slice to run a query method. Breakdown stats for the createweight method do not include profiled max time because the method runs at the query level rather than the slice level. This field is included only if you enable concurrent segment search.
min<method>The minimum amount of time taken by any slice to run a query method. Breakdown stats for the createweight method do not include profiled min time because the method runs at the query level rather than the slice level. This field is included only if you enable concurrent segment search.
avg<method>The average amount of time taken by any slice to run a query method. Breakdown stats for the createweight method do not include profiled avg time because the method runs at the query level rather than the slice level. This field is included only if you enable concurrent segment search.
max<method>countThe maximum number of invocations of a <method> on any slice. Breakdown stats for the create_weight method do not include profiled max count because the method runs at the query level rather than the slice level. This field is included only if you enable concurrent segment search.
min<method>countThe minimum number of invocations of a <method> on any slice. Breakdown stats for the create_weight method do not include profiled min count because the method runs at the query level rather than the slice level. This field is included only if you enable concurrent segment search.
avg<method>_countThe average number of invocations of a <method> on any slice. Breakdown stats for the create_weight method do not include profiled avg count because the method runs at the query level rather than the slice level. This field is included only if you enable concurrent segment search.

The collector array

The collector array contains information about Lucene Collectors. A Collector is responsible for coordinating document traversal and scoring and collecting matching documents. Using Collectors, individual queries can record aggregation results and execute global queries or post-query filters.

FieldDescription
nameThe collector name. In the example response, the collector is a single SimpleTopScoreDocCollector—the default scoring and sorting collector.
reasonContains a description of the collector. For possible field values, see Collector reasons.
time_in_nanosThe total elapsed time for this collector, in nanoseconds. For concurrent segment search, time_in_nanos is the total amount of time across all slices (the difference between the last completed slice execution end time and the first slice execution start time).
childrenIf a collector has subcollectors (children), this field contains information about the subcollectors.
max_slice_time_in_nanosThe maximum amount of time taken by any slice, in nanoseconds. This field is included only if you enable concurrent segment search.
min_slice_time_in_nanosThe minimum amount of time taken by any slice, in nanoseconds. This field is included only if you enable concurrent segment search.
avg_slice_time_in_nanosThe average amount of time taken by any slice, in nanoseconds. This field is included only if you enable concurrent segment search.
slice_countThe total slice count for this query. This field is included only if you enable concurrent segment search.
reduce_time_in_nanosThe amount of time taken to reduce results for all slice collectors, in nanoseconds. This field is included only if you enable concurrent segment search.

Collector times are calculated, combined, and normalized independently, so they are independent of query times.

Collector reasons

The following table describes all available collector reasons.

ReasonDescription
search_sortedA collector that scores and sorts documents. Present in most simple searches.
search_countA collector that counts the number of matching documents but does not fetch the source. Present when size: 0 is specified.
search_terminate_after_countA collector that searches for matching documents and terminates the search when it finds a specified number of documents. Present when the terminate_after_count query parameter is specified.
search_min_scoreA collector that returns matching documents that have a score greater than a minimum score. Present when the min_score parameter is specified.
search_multiA wrapper collector for other collectors. Present when search, aggregations, global aggregations, and post filters are combined in a single search.
search_timeoutA collector that stops running after a specified period of time. Present when a timeout parameter is specified.
aggregationA collector for aggregations that is run against the specified query scope. OpenSearch uses a single aggregation collector to collect documents for all aggregations.
global_aggregationA collector that is run against the global query scope. Global scope is different from a specified query scope, so in order to collect the entire dataset, a match_all query must be run.

Aggregation responses

Response: Global aggregation

The response contains profiling information:

Response

  1. {
  2. "took": 10,
  3. "timed_out": false,
  4. "_shards": {
  5. "total": 1,
  6. "successful": 1,
  7. "skipped": 0,
  8. "failed": 0
  9. },
  10. "hits": {
  11. "total": {
  12. "value": 1370,
  13. "relation": "eq"
  14. },
  15. "max_score": null,
  16. "hits": []
  17. },
  18. "aggregations": {
  19. "all_products": {
  20. "doc_count": 4675,
  21. "avg_price": {
  22. "value": 75.05542864304813
  23. }
  24. },
  25. "elitelligence_products": {
  26. "value": 68.4430200729927
  27. }
  28. },
  29. "profile": {
  30. "shards": [
  31. {
  32. "id": "[LidyZ1HVS-u93-73Z49dQg][opensearch_dashboards_sample_data_ecommerce][0]",
  33. "inbound_network_time_in_millis": 0,
  34. "outbound_network_time_in_millis": 0,
  35. "searches": [
  36. {
  37. "query": [
  38. {
  39. "type": "ConstantScoreQuery",
  40. "description": "ConstantScore(manufacturer:elitelligence)",
  41. "time_in_nanos": 1367487,
  42. "breakdown": {
  43. "set_min_competitive_score_count": 0,
  44. "match_count": 0,
  45. "shallow_advance_count": 0,
  46. "set_min_competitive_score": 0,
  47. "next_doc": 634321,
  48. "match": 0,
  49. "next_doc_count": 1370,
  50. "score_count": 0,
  51. "compute_max_score_count": 0,
  52. "compute_max_score": 0,
  53. "advance": 173250,
  54. "advance_count": 2,
  55. "score": 0,
  56. "build_scorer_count": 4,
  57. "create_weight": 132458,
  58. "shallow_advance": 0,
  59. "create_weight_count": 1,
  60. "build_scorer": 427458
  61. },
  62. "children": [
  63. {
  64. "type": "TermQuery",
  65. "description": "manufacturer:elitelligence",
  66. "time_in_nanos": 1174794,
  67. "breakdown": {
  68. "set_min_competitive_score_count": 0,
  69. "match_count": 0,
  70. "shallow_advance_count": 0,
  71. "set_min_competitive_score": 0,
  72. "next_doc": 470918,
  73. "match": 0,
  74. "next_doc_count": 1370,
  75. "score_count": 0,
  76. "compute_max_score_count": 0,
  77. "compute_max_score": 0,
  78. "advance": 172084,
  79. "advance_count": 2,
  80. "score": 0,
  81. "build_scorer_count": 4,
  82. "create_weight": 114041,
  83. "shallow_advance": 0,
  84. "create_weight_count": 1,
  85. "build_scorer": 417751
  86. }
  87. }
  88. ]
  89. }
  90. ],
  91. "rewrite_time": 42542,
  92. "collector": [
  93. {
  94. "name": "MultiCollector",
  95. "reason": "search_multi",
  96. "time_in_nanos": 778406,
  97. "children": [
  98. {
  99. "name": "EarlyTerminatingCollector",
  100. "reason": "search_count",
  101. "time_in_nanos": 70290
  102. },
  103. {
  104. "name": "ProfilingAggregator: [elitelligence_products]",
  105. "reason": "aggregation",
  106. "time_in_nanos": 502780
  107. }
  108. ]
  109. }
  110. ]
  111. },
  112. {
  113. "query": [
  114. {
  115. "type": "ConstantScoreQuery",
  116. "description": "ConstantScore(*:*)",
  117. "time_in_nanos": 995345,
  118. "breakdown": {
  119. "set_min_competitive_score_count": 0,
  120. "match_count": 0,
  121. "shallow_advance_count": 0,
  122. "set_min_competitive_score": 0,
  123. "next_doc": 930803,
  124. "match": 0,
  125. "next_doc_count": 4675,
  126. "score_count": 0,
  127. "compute_max_score_count": 0,
  128. "compute_max_score": 0,
  129. "advance": 2209,
  130. "advance_count": 2,
  131. "score": 0,
  132. "build_scorer_count": 4,
  133. "create_weight": 23875,
  134. "shallow_advance": 0,
  135. "create_weight_count": 1,
  136. "build_scorer": 38458
  137. },
  138. "children": [
  139. {
  140. "type": "MatchAllDocsQuery",
  141. "description": "*:*",
  142. "time_in_nanos": 431375,
  143. "breakdown": {
  144. "set_min_competitive_score_count": 0,
  145. "match_count": 0,
  146. "shallow_advance_count": 0,
  147. "set_min_competitive_score": 0,
  148. "next_doc": 389875,
  149. "match": 0,
  150. "next_doc_count": 4675,
  151. "score_count": 0,
  152. "compute_max_score_count": 0,
  153. "compute_max_score": 0,
  154. "advance": 1167,
  155. "advance_count": 2,
  156. "score": 0,
  157. "build_scorer_count": 4,
  158. "create_weight": 9458,
  159. "shallow_advance": 0,
  160. "create_weight_count": 1,
  161. "build_scorer": 30875
  162. }
  163. }
  164. ]
  165. }
  166. ],
  167. "rewrite_time": 8792,
  168. "collector": [
  169. {
  170. "name": "ProfilingAggregator: [all_products]",
  171. "reason": "aggregation_global",
  172. "time_in_nanos": 1310536
  173. }
  174. ]
  175. }
  176. ],
  177. "aggregations": [
  178. {
  179. "type": "AvgAggregator",
  180. "description": "elitelligence_products",
  181. "time_in_nanos": 319918,
  182. "breakdown": {
  183. "reduce": 0,
  184. "post_collection_count": 1,
  185. "build_leaf_collector": 130709,
  186. "build_aggregation": 2709,
  187. "build_aggregation_count": 1,
  188. "build_leaf_collector_count": 2,
  189. "post_collection": 584,
  190. "initialize": 4750,
  191. "initialize_count": 1,
  192. "reduce_count": 0,
  193. "collect": 181166,
  194. "collect_count": 1370
  195. }
  196. },
  197. {
  198. "type": "GlobalAggregator",
  199. "description": "all_products",
  200. "time_in_nanos": 1519340,
  201. "breakdown": {
  202. "reduce": 0,
  203. "post_collection_count": 1,
  204. "build_leaf_collector": 134625,
  205. "build_aggregation": 59291,
  206. "build_aggregation_count": 1,
  207. "build_leaf_collector_count": 2,
  208. "post_collection": 5041,
  209. "initialize": 24500,
  210. "initialize_count": 1,
  211. "reduce_count": 0,
  212. "collect": 1295883,
  213. "collect_count": 4675
  214. },
  215. "children": [
  216. {
  217. "type": "AvgAggregator",
  218. "description": "avg_price",
  219. "time_in_nanos": 775967,
  220. "breakdown": {
  221. "reduce": 0,
  222. "post_collection_count": 1,
  223. "build_leaf_collector": 98999,
  224. "build_aggregation": 33083,
  225. "build_aggregation_count": 1,
  226. "build_leaf_collector_count": 2,
  227. "post_collection": 2209,
  228. "initialize": 1708,
  229. "initialize_count": 1,
  230. "reduce_count": 0,
  231. "collect": 639968,
  232. "collect_count": 4675
  233. }
  234. }
  235. ]
  236. }
  237. ]
  238. }
  239. ]
  240. }
  241. }

Response: Non-global aggregation

The response contains profiling information:

Response

  1. {
  2. "took": 13,
  3. "timed_out": false,
  4. "_shards": {
  5. "total": 1,
  6. "successful": 1,
  7. "skipped": 0,
  8. "failed": 0
  9. },
  10. "hits": {
  11. "total": {
  12. "value": 4675,
  13. "relation": "eq"
  14. },
  15. "max_score": null,
  16. "hits": []
  17. },
  18. "aggregations": {
  19. "avg_taxful_total_price": {
  20. "value": 75.05542864304813
  21. }
  22. },
  23. "profile": {
  24. "shards": [
  25. {
  26. "id": "[LidyZ1HVS-u93-73Z49dQg][opensearch_dashboards_sample_data_ecommerce][0]",
  27. "inbound_network_time_in_millis": 0,
  28. "outbound_network_time_in_millis": 0,
  29. "searches": [
  30. {
  31. "query": [
  32. {
  33. "type": "ConstantScoreQuery",
  34. "description": "ConstantScore(*:*)",
  35. "time_in_nanos": 1690820,
  36. "breakdown": {
  37. "set_min_competitive_score_count": 0,
  38. "match_count": 0,
  39. "shallow_advance_count": 0,
  40. "set_min_competitive_score": 0,
  41. "next_doc": 1614112,
  42. "match": 0,
  43. "next_doc_count": 4675,
  44. "score_count": 0,
  45. "compute_max_score_count": 0,
  46. "compute_max_score": 0,
  47. "advance": 2708,
  48. "advance_count": 2,
  49. "score": 0,
  50. "build_scorer_count": 4,
  51. "create_weight": 20250,
  52. "shallow_advance": 0,
  53. "create_weight_count": 1,
  54. "build_scorer": 53750
  55. },
  56. "children": [
  57. {
  58. "type": "MatchAllDocsQuery",
  59. "description": "*:*",
  60. "time_in_nanos": 770902,
  61. "breakdown": {
  62. "set_min_competitive_score_count": 0,
  63. "match_count": 0,
  64. "shallow_advance_count": 0,
  65. "set_min_competitive_score": 0,
  66. "next_doc": 721943,
  67. "match": 0,
  68. "next_doc_count": 4675,
  69. "score_count": 0,
  70. "compute_max_score_count": 0,
  71. "compute_max_score": 0,
  72. "advance": 1042,
  73. "advance_count": 2,
  74. "score": 0,
  75. "build_scorer_count": 4,
  76. "create_weight": 5041,
  77. "shallow_advance": 0,
  78. "create_weight_count": 1,
  79. "build_scorer": 42876
  80. }
  81. }
  82. ]
  83. }
  84. ],
  85. "rewrite_time": 22000,
  86. "collector": [
  87. {
  88. "name": "MultiCollector",
  89. "reason": "search_multi",
  90. "time_in_nanos": 3672676,
  91. "children": [
  92. {
  93. "name": "EarlyTerminatingCollector",
  94. "reason": "search_count",
  95. "time_in_nanos": 78626
  96. },
  97. {
  98. "name": "ProfilingAggregator: [avg_taxful_total_price]",
  99. "reason": "aggregation",
  100. "time_in_nanos": 2834566
  101. }
  102. ]
  103. }
  104. ]
  105. }
  106. ],
  107. "aggregations": [
  108. {
  109. "type": "AvgAggregator",
  110. "description": "avg_taxful_total_price",
  111. "time_in_nanos": 1973702,
  112. "breakdown": {
  113. "reduce": 0,
  114. "post_collection_count": 1,
  115. "build_leaf_collector": 199292,
  116. "build_aggregation": 13584,
  117. "build_aggregation_count": 1,
  118. "build_leaf_collector_count": 2,
  119. "post_collection": 6125,
  120. "initialize": 6916,
  121. "initialize_count": 1,
  122. "reduce_count": 0,
  123. "collect": 1747785,
  124. "collect_count": 4675
  125. }
  126. }
  127. ]
  128. }
  129. ]
  130. }
  131. }

Response body fields

The aggregations array contains aggregation objects with the following fields.

FieldData typeDescription
typeStringThe aggregator type. In the non-global aggregation example response, the aggregator type is AvgAggregator. Global aggregation example response contains a GlobalAggregator with an AvgAggregator child.
descriptionStringContains a Lucene explanation of the aggregation. Helps differentiate aggregations with the same type.
time_in_nanosLongThe total elapsed time for this aggregation, in nanoseconds. For concurrent segment search, time_in_nanos is the total amount of time across all slices (the difference between the last completed slice execution end time and the first slice execution start time).
breakdownObjectContains timing statistics about low-level Lucene execution.
childrenArray of objectsIf an aggregation has subaggregations (children), this field contains information about the subaggregations.
debugObjectSome aggregations return a debug object that describes the details of the underlying execution.
max_slice_time_in_nanosLongThe maximum amount of time taken by any slice to run an aggregation, in nanoseconds. This field is included only if you enable concurrent segment search.
min_slice_time_in_nanosLongThe minimum amount of time taken by any slice to run an aggregation, in nanoseconds. This field is included only if you enable concurrent segment search.
avg_slice_time_in_nanosLongThe average amount of time taken by any slice to run an aggregation, in nanoseconds. This field is included only if you enable concurrent segment search.

The breakdown object

The breakdown object represents the timing statistics about low-level Lucene execution, broken down by method. Each field in the breakdown object represents an internal Lucene method executed within the aggregation. Timings are listed in wall-clock nanoseconds and are not normalized. The breakdown timings are inclusive of all child times. The breakdown object is comprised of the following fields. All fields contain integer values.

FieldDescription
initializeContains the amount of time taken to execute the preCollection() callback method during AggregationCollectorManager creation. For concurrent segment search, the initialize method contains the total elapsed time across all slices (the difference between the last completed slice execution end time and the first slice execution start time).
buildleaf_collectorContains the time spent running the aggregation’s getLeafCollector() method, which creates a new collector to collect the given context. For concurrent segment search, the build_leaf_collector method contains the total elapsed time across all slices (the difference between the last completed slice execution end time and the first slice execution start time).
collectContains the time spent collecting the documents into buckets. For concurrent segment search, the collect method contains the total elapsed time across all slices (the difference between the last completed slice execution end time and the first slice execution start time).
post_collectionContains the time spent running the aggregation’s postCollection() callback method. For concurrent segment search, the post_collection method contains the total elapsed time across all slices (the difference between the last completed slice execution end time and the first slice execution start time).
build_aggregationContains the time spent running the aggregation’s buildAggregations() method, which builds the results of this aggregation. For concurrent segment search, the build_aggregation method contains the total elapsed time across all slices (the difference between the last completed slice execution end time and the first slice execution start time).
reduceContains the time spent in the reduce phase. For concurrent segment search, the reduce method contains the total elapsed time across all slices (the difference between the last completed slice execution end time and the first slice execution start time).
<method>_countContains the number of invocations of a <method>. For example, build_leaf_collector_count contains the number of invocations of the build_leaf_collector method.
max<method>The maximum amount of time taken by any slice to run an aggregation method. This field is included only if you enable concurrent segment search.
min<method>The minimum amount of time taken by any slice to run an aggregation method. This field is included only if you enable concurrent segment search.
avg<method>The average amount of time taken by any slice to run an aggregation method. This field is included only if you enable concurrent segment search.
<method>countThe total method count across all slices. For example, for the collect method, it is the total number of invocations of this method needed to collect documents into buckets across all slices.
max<method>countThe maximum number of invocations of a <method> on any slice. This field is included only if you enable concurrent segment search.
min<method>countThe minimum number of invocations of a <method> on any slice. This field is included only if you enable concurrent segment search.
avg<method>_countThe average number of invocations of a <method> on any slice. This field is included only if you enable concurrent segment search.