Normalize Aggregation

A parent pipeline aggregation which calculates the specific normalized/rescaled value for a specific bucket value. Values that cannot be normalized, will be skipped using the skip gap policy.

Syntax

A normalize aggregation looks like this in isolation:

  1. {
  2.   "normalize": {
  3.     "buckets_path": "normalized",
  4.     "method": "percent_of_sum"
  5.   }
  6. }

Table 67. normalize_pipeline Parameters

Parameter NameDescriptionRequiredDefault Value

buckets_path

The path to the buckets we wish to normalize (see buckets_path syntax for more details)

Required

method

The specific method to apply

Required

format

format to apply to the output value of this aggregation

Optional

null

Methods

The Normalize Aggregation supports multiple methods to transform the bucket values. Each method definition will use the following original set of bucket values as examples: [5, 5, 10, 50, 10, 20].

rescale_0_1

This method rescales the data such that the minimum number is zero, and the maximum number is 1, with the rest normalized linearly in-between.

  1. x' = (x - min_x) / (max_x - min_x)
  1. [0, 0, .1111, 1, .1111, .3333]

rescale_0_100

This method rescales the data such that the minimum number is zero, and the maximum number is 1, with the rest normalized linearly in-between.

  1. x' = 100 * (x - min_x) / (max_x - min_x)
  1. [0, 0, 11.11, 100, 11.11, 33.33]

percent_of_sum

This method normalizes each value so that it represents a percentage of the total sum it attributes to.

  1. x' = x / sum_x
  1. [5%, 5%, 10%, 50%, 10%, 20%]

mean

This method normalizes such that each value is normalized by how much it differs from the average.

  1. x' = (x - mean_x) / (max_x - min_x)
  1. [4.63, 4.63, 9.63, 49.63, 9.63, 9.63, 19.63]

zscore

This method normalizes such that each value represents how far it is from the mean relative to the standard deviation

  1. x' = (x - mean_x) / stdev_x
  1. [-0.68, -0.68, -0.39, 1.94, -0.39, 0.19]

softmax

This method normalizes such that each value is exponentiated and relative to the sum of the exponents of the original values.

  1. x' = e^x / sum_e_x
  1. [2.862E-20, 2.862E-20, 4.248E-18, 0.999, 9.357E-14, 4.248E-18]

Example

The following snippet calculates the percent of total sales for each month:

  1. POST /sales/_search
  2. {
  3.   "size": 0,
  4.   "aggs": {
  5.     "sales_per_month": {
  6.       "date_histogram": {
  7.         "field": "date",
  8.         "calendar_interval": "month"
  9.       },
  10.       "aggs": {
  11.         "sales": {
  12.           "sum": {
  13.             "field": "price"
  14.           }
  15.         },
  16.         "percent_of_total_sales": {
  17.           "normalize": {
  18.             "buckets_path": "sales",          
  19.             "method": "percent_of_sum",       
  20.             "format": "00.00%"                
  21.           }
  22.         }
  23.       }
  24.     }
  25.   }
  26. }

buckets_path instructs this normalize aggregation to use the output of the sales aggregation for rescaling

method sets which rescaling to apply. In this case, percent_of_sum will calculate the sales value as a percent of all sales in the parent bucket

format influences how to format the metric as a string using Java’s DecimalFormat pattern. In this case, multiplying by 100 and adding a %

And the following may be the response:

  1. {
  2.    "took": 11,
  3.    "timed_out": false,
  4.    "_shards": ...,
  5.    "hits": ...,
  6.    "aggregations": {
  7.       "sales_per_month": {
  8.          "buckets": [
  9.             {
  10.                "key_as_string": "2015/01/01 00:00:00",
  11.                "key": 1420070400000,
  12.                "doc_count": 3,
  13.                "sales": {
  14.                   "value": 550.0
  15.                },
  16.                "percent_of_total_sales": {
  17.                   "value": 0.5583756345177665,
  18.                   "value_as_string": "55.84%"
  19.                }
  20.             },
  21.             {
  22.                "key_as_string": "2015/02/01 00:00:00",
  23.                "key": 1422748800000,
  24.                "doc_count": 2,
  25.                "sales": {
  26.                   "value": 60.0
  27.                },
  28.                "percent_of_total_sales": {
  29.                   "value": 0.06091370558375635,
  30.                   "value_as_string": "06.09%"
  31.                }
  32.             },
  33.             {
  34.                "key_as_string": "2015/03/01 00:00:00",
  35.                "key": 1425168000000,
  36.                "doc_count": 2,
  37.                "sales": {
  38.                   "value": 375.0
  39.                },
  40.                "percent_of_total_sales": {
  41.                   "value": 0.38071065989847713,
  42.                   "value_as_string": "38.07%"
  43.                }
  44.             }
  45.          ]
  46.       }
  47.    }
  48. }