Run downsampling manually

Run downsampling manually

The recommended way to downsample a time-series data stream (TSDS) is through index lifecycle management (ILM). However, if you’re not using ILM, you can downsample a TSDS manually. This guide shows you how, using typical Kubernetes cluster monitoring data.

To test out manual downsampling, follow these steps:

  1. Check the prerequisites.
  2. Create a time series data stream.
  3. Ingest time series data.
  4. Downsample the TSDS.
  5. View the results.

Prerequisites

  • Refer to the TSDS prerequisites.
  • It is not possible to downsample a data stream directly, nor multiple indices at once. It’s only possible to downsample one time series index (TSDS backing index).
  • In order to downsample an index, it needs to be read-only. For a TSDS write index, this means it needs to be rolled over and made read-only first.
  • Downsampling uses UTC timestamps.
  • Downsampling needs at least one metric field to exist in the time series index.

Create a time series data stream

First, you’ll create a TSDS. For simplicity, in the time series mapping all time_series_metric parameters are set to type gauge, but other values such as counter and histogram may also be used. The time_series_metric values determine the kind of statistical representations that are used during downsampling.

The index template includes a set of static time series dimensions: host, namespace, node, and pod. The time series dimensions are not changed by the downsampling process.

  1. resp = client.indices.put_index_template(
  2.     name="my-data-stream-template",
  3.     index_patterns=[
  4.         "my-data-stream*"
  5.     ],
  6.     data_stream={},
  7.     template={
  8.         "settings": {
  9.             "index": {
  10.                 "mode": "time_series",
  11.                 "routing_path": [
  12.                     "kubernetes.namespace",
  13.                     "kubernetes.host",
  14.                     "kubernetes.node",
  15.                     "kubernetes.pod"
  16.                 ],
  17.                 "number_of_replicas": 0,
  18.                 "number_of_shards": 2
  19.             }
  20.         },
  21.         "mappings": {
  22.             "properties": {
  23.                 "@timestamp": {
  24.                     "type": "date"
  25.                 },
  26.                 "kubernetes": {
  27.                     "properties": {
  28.                         "container": {
  29.                             "properties": {
  30.                                 "cpu": {
  31.                                     "properties": {
  32.                                         "usage": {
  33.                                             "properties": {
  34.                                                 "core": {
  35.                                                     "properties": {
  36.                                                         "ns": {
  37.                                                             "type": "long"
  38.                                                         }
  39.                                                     }
  40.                                                 },
  41.                                                 "limit": {
  42.                                                     "properties": {
  43.                                                         "pct": {
  44.                                                             "type": "float"
  45.                                                         }
  46.                                                     }
  47.                                                 },
  48.                                                 "nanocores": {
  49.                                                     "type": "long",
  50.                                                     "time_series_metric": "gauge"
  51.                                                 },
  52.                                                 "node": {
  53.                                                     "properties": {
  54.                                                         "pct": {
  55.                                                             "type": "float"
  56.                                                         }
  57.                                                     }
  58.                                                 }
  59.                                             }
  60.                                         }
  61.                                     }
  62.                                 },
  63.                                 "memory": {
  64.                                     "properties": {
  65.                                         "available": {
  66.                                             "properties": {
  67.                                                 "bytes": {
  68.                                                     "type": "long",
  69.                                                     "time_series_metric": "gauge"
  70.                                                 }
  71.                                             }
  72.                                         },
  73.                                         "majorpagefaults": {
  74.                                             "type": "long"
  75.                                         },
  76.                                         "pagefaults": {
  77.                                             "type": "long",
  78.                                             "time_series_metric": "gauge"
  79.                                         },
  80.                                         "rss": {
  81.                                             "properties": {
  82.                                                 "bytes": {
  83.                                                     "type": "long",
  84.                                                     "time_series_metric": "gauge"
  85.                                                 }
  86.                                             }
  87.                                         },
  88.                                         "usage": {
  89.                                             "properties": {
  90.                                                 "bytes": {
  91.                                                     "type": "long",
  92.                                                     "time_series_metric": "gauge"
  93.                                                 },
  94.                                                 "limit": {
  95.                                                     "properties": {
  96.                                                         "pct": {
  97.                                                             "type": "float"
  98.                                                         }
  99.                                                     }
  100.                                                 },
  101.                                                 "node": {
  102.                                                     "properties": {
  103.                                                         "pct": {
  104.                                                             "type": "float"
  105.                                                         }
  106.                                                     }
  107.                                                 }
  108.                                             }
  109.                                         },
  110.                                         "workingset": {
  111.                                             "properties": {
  112.                                                 "bytes": {
  113.                                                     "type": "long",
  114.                                                     "time_series_metric": "gauge"
  115.                                                 }
  116.                                             }
  117.                                         }
  118.                                     }
  119.                                 },
  120.                                 "name": {
  121.                                     "type": "keyword"
  122.                                 },
  123.                                 "start_time": {
  124.                                     "type": "date"
  125.                                 }
  126.                             }
  127.                         },
  128.                         "host": {
  129.                             "type": "keyword",
  130.                             "time_series_dimension": True
  131.                         },
  132.                         "namespace": {
  133.                             "type": "keyword",
  134.                             "time_series_dimension": True
  135.                         },
  136.                         "node": {
  137.                             "type": "keyword",
  138.                             "time_series_dimension": True
  139.                         },
  140.                         "pod": {
  141.                             "type": "keyword",
  142.                             "time_series_dimension": True
  143.                         }
  144.                     }
  145.                 }
  146.             }
  147.         }
  148.     },
  149. )
  150. print(resp)
  1. response = client.indices.put_index_template(
  2.   name: 'my-data-stream-template',
  3.   body: {
  4.     index_patterns: [
  5.       'my-data-stream*'
  6.     ],
  7.     data_stream: {},
  8.     template: {
  9.       settings: {
  10.         index: {
  11.           mode: 'time_series',
  12.           routing_path: [
  13.             'kubernetes.namespace',
  14.             'kubernetes.host',
  15.             'kubernetes.node',
  16.             'kubernetes.pod'
  17.           ],
  18.           number_of_replicas: 0,
  19.           number_of_shards: 2
  20.         }
  21.       },
  22.       mappings: {
  23.         properties: {
  24.           "@timestamp": {
  25.             type: 'date'
  26.           },
  27.           kubernetes: {
  28.             properties: {
  29.               container: {
  30.                 properties: {
  31.                   cpu: {
  32.                     properties: {
  33.                       usage: {
  34.                         properties: {
  35.                           core: {
  36.                             properties: {
  37.                               ns: {
  38.                                 type: 'long'
  39.                               }
  40.                             }
  41.                           },
  42.                           limit: {
  43.                             properties: {
  44.                               pct: {
  45.                                 type: 'float'
  46.                               }
  47.                             }
  48.                           },
  49.                           nanocores: {
  50.                             type: 'long',
  51.                             time_series_metric: 'gauge'
  52.                           },
  53.                           node: {
  54.                             properties: {
  55.                               pct: {
  56.                                 type: 'float'
  57.                               }
  58.                             }
  59.                           }
  60.                         }
  61.                       }
  62.                     }
  63.                   },
  64.                   memory: {
  65.                     properties: {
  66.                       available: {
  67.                         properties: {
  68.                           bytes: {
  69.                             type: 'long',
  70.                             time_series_metric: 'gauge'
  71.                           }
  72.                         }
  73.                       },
  74.                       majorpagefaults: {
  75.                         type: 'long'
  76.                       },
  77.                       pagefaults: {
  78.                         type: 'long',
  79.                         time_series_metric: 'gauge'
  80.                       },
  81.                       rss: {
  82.                         properties: {
  83.                           bytes: {
  84.                             type: 'long',
  85.                             time_series_metric: 'gauge'
  86.                           }
  87.                         }
  88.                       },
  89.                       usage: {
  90.                         properties: {
  91.                           bytes: {
  92.                             type: 'long',
  93.                             time_series_metric: 'gauge'
  94.                           },
  95.                           limit: {
  96.                             properties: {
  97.                               pct: {
  98.                                 type: 'float'
  99.                               }
  100.                             }
  101.                           },
  102.                           node: {
  103.                             properties: {
  104.                               pct: {
  105.                                 type: 'float'
  106.                               }
  107.                             }
  108.                           }
  109.                         }
  110.                       },
  111.                       workingset: {
  112.                         properties: {
  113.                           bytes: {
  114.                             type: 'long',
  115.                             time_series_metric: 'gauge'
  116.                           }
  117.                         }
  118.                       }
  119.                     }
  120.                   },
  121.                   name: {
  122.                     type: 'keyword'
  123.                   },
  124.                   start_time: {
  125.                     type: 'date'
  126.                   }
  127.                 }
  128.               },
  129.               host: {
  130.                 type: 'keyword',
  131.                 time_series_dimension: true
  132.               },
  133.               namespace: {
  134.                 type: 'keyword',
  135.                 time_series_dimension: true
  136.               },
  137.               node: {
  138.                 type: 'keyword',
  139.                 time_series_dimension: true
  140.               },
  141.               pod: {
  142.                 type: 'keyword',
  143.                 time_series_dimension: true
  144.               }
  145.             }
  146.           }
  147.         }
  148.       }
  149.     }
  150.   }
  151. )
  152. puts response
  1. const response = await client.indices.putIndexTemplate({
  2.   name: "my-data-stream-template",
  3.   index_patterns: ["my-data-stream*"],
  4.   data_stream: {},
  5.   template: {
  6.     settings: {
  7.       index: {
  8.         mode: "time_series",
  9.         routing_path: [
  10.           "kubernetes.namespace",
  11.           "kubernetes.host",
  12.           "kubernetes.node",
  13.           "kubernetes.pod",
  14.         ],
  15.         number_of_replicas: 0,
  16.         number_of_shards: 2,
  17.       },
  18.     },
  19.     mappings: {
  20.       properties: {
  21.         "@timestamp": {
  22.           type: "date",
  23.         },
  24.         kubernetes: {
  25.           properties: {
  26.             container: {
  27.               properties: {
  28.                 cpu: {
  29.                   properties: {
  30.                     usage: {
  31.                       properties: {
  32.                         core: {
  33.                           properties: {
  34.                             ns: {
  35.                               type: "long",
  36.                             },
  37.                           },
  38.                         },
  39.                         limit: {
  40.                           properties: {
  41.                             pct: {
  42.                               type: "float",
  43.                             },
  44.                           },
  45.                         },
  46.                         nanocores: {
  47.                           type: "long",
  48.                           time_series_metric: "gauge",
  49.                         },
  50.                         node: {
  51.                           properties: {
  52.                             pct: {
  53.                               type: "float",
  54.                             },
  55.                           },
  56.                         },
  57.                       },
  58.                     },
  59.                   },
  60.                 },
  61.                 memory: {
  62.                   properties: {
  63.                     available: {
  64.                       properties: {
  65.                         bytes: {
  66.                           type: "long",
  67.                           time_series_metric: "gauge",
  68.                         },
  69.                       },
  70.                     },
  71.                     majorpagefaults: {
  72.                       type: "long",
  73.                     },
  74.                     pagefaults: {
  75.                       type: "long",
  76.                       time_series_metric: "gauge",
  77.                     },
  78.                     rss: {
  79.                       properties: {
  80.                         bytes: {
  81.                           type: "long",
  82.                           time_series_metric: "gauge",
  83.                         },
  84.                       },
  85.                     },
  86.                     usage: {
  87.                       properties: {
  88.                         bytes: {
  89.                           type: "long",
  90.                           time_series_metric: "gauge",
  91.                         },
  92.                         limit: {
  93.                           properties: {
  94.                             pct: {
  95.                               type: "float",
  96.                             },
  97.                           },
  98.                         },
  99.                         node: {
  100.                           properties: {
  101.                             pct: {
  102.                               type: "float",
  103.                             },
  104.                           },
  105.                         },
  106.                       },
  107.                     },
  108.                     workingset: {
  109.                       properties: {
  110.                         bytes: {
  111.                           type: "long",
  112.                           time_series_metric: "gauge",
  113.                         },
  114.                       },
  115.                     },
  116.                   },
  117.                 },
  118.                 name: {
  119.                   type: "keyword",
  120.                 },
  121.                 start_time: {
  122.                   type: "date",
  123.                 },
  124.               },
  125.             },
  126.             host: {
  127.               type: "keyword",
  128.               time_series_dimension: true,
  129.             },
  130.             namespace: {
  131.               type: "keyword",
  132.               time_series_dimension: true,
  133.             },
  134.             node: {
  135.               type: "keyword",
  136.               time_series_dimension: true,
  137.             },
  138.             pod: {
  139.               type: "keyword",
  140.               time_series_dimension: true,
  141.             },
  142.           },
  143.         },
  144.       },
  145.     },
  146.   },
  147. });
  148. console.log(response);
  1. PUT _index_template/my-data-stream-template
  2. {
  3.   "index_patterns": [
  4.     "my-data-stream*"
  5.   ],
  6.   "data_stream": {},
  7.   "template": {
  8.     "settings": {
  9.       "index": {
  10.         "mode": "time_series",
  11.         "routing_path": [
  12.           "kubernetes.namespace",
  13.           "kubernetes.host",
  14.           "kubernetes.node",
  15.           "kubernetes.pod"
  16.         ],
  17.         "number_of_replicas": 0,
  18.         "number_of_shards": 2
  19.       }
  20.     },
  21.     "mappings": {
  22.       "properties": {
  23.         "@timestamp": {
  24.           "type": "date"
  25.         },
  26.         "kubernetes": {
  27.           "properties": {
  28.             "container": {
  29.               "properties": {
  30.                 "cpu": {
  31.                   "properties": {
  32.                     "usage": {
  33.                       "properties": {
  34.                         "core": {
  35.                           "properties": {
  36.                             "ns": {
  37.                               "type": "long"
  38.                             }
  39.                           }
  40.                         },
  41.                         "limit": {
  42.                           "properties": {
  43.                             "pct": {
  44.                               "type": "float"
  45.                             }
  46.                           }
  47.                         },
  48.                         "nanocores": {
  49.                           "type": "long",
  50.                           "time_series_metric": "gauge"
  51.                         },
  52.                         "node": {
  53.                           "properties": {
  54.                             "pct": {
  55.                               "type": "float"
  56.                             }
  57.                           }
  58.                         }
  59.                       }
  60.                     }
  61.                   }
  62.                 },
  63.                 "memory": {
  64.                   "properties": {
  65.                     "available": {
  66.                       "properties": {
  67.                         "bytes": {
  68.                           "type": "long",
  69.                           "time_series_metric": "gauge"
  70.                         }
  71.                       }
  72.                     },
  73.                     "majorpagefaults": {
  74.                       "type": "long"
  75.                     },
  76.                     "pagefaults": {
  77.                       "type": "long",
  78.                       "time_series_metric": "gauge"
  79.                     },
  80.                     "rss": {
  81.                       "properties": {
  82.                         "bytes": {
  83.                           "type": "long",
  84.                           "time_series_metric": "gauge"
  85.                         }
  86.                       }
  87.                     },
  88.                     "usage": {
  89.                       "properties": {
  90.                         "bytes": {
  91.                           "type": "long",
  92.                           "time_series_metric": "gauge"
  93.                         },
  94.                         "limit": {
  95.                           "properties": {
  96.                             "pct": {
  97.                               "type": "float"
  98.                             }
  99.                           }
  100.                         },
  101.                         "node": {
  102.                           "properties": {
  103.                             "pct": {
  104.                               "type": "float"
  105.                             }
  106.                           }
  107.                         }
  108.                       }
  109.                     },
  110.                     "workingset": {
  111.                       "properties": {
  112.                         "bytes": {
  113.                           "type": "long",
  114.                           "time_series_metric": "gauge"
  115.                         }
  116.                       }
  117.                     }
  118.                   }
  119.                 },
  120.                 "name": {
  121.                   "type": "keyword"
  122.                 },
  123.                 "start_time": {
  124.                   "type": "date"
  125.                 }
  126.               }
  127.             },
  128.             "host": {
  129.               "type": "keyword",
  130.               "time_series_dimension": true
  131.             },
  132.             "namespace": {
  133.               "type": "keyword",
  134.               "time_series_dimension": true
  135.             },
  136.             "node": {
  137.               "type": "keyword",
  138.               "time_series_dimension": true
  139.             },
  140.             "pod": {
  141.               "type": "keyword",
  142.               "time_series_dimension": true
  143.             }
  144.           }
  145.         }
  146.       }
  147.     }
  148.   }
  149. }

Ingest time series data

Because time series data streams have been designed to only accept recent data, in this example, you’ll use an ingest pipeline to time-shift the data as it gets indexed. As a result, the indexed data will have an @timestamp from the last 15 minutes.

Create the pipeline with this request:

  1. resp = client.ingest.put_pipeline(
  2.     id="my-timestamp-pipeline",
  3.     description="Shifts the @timestamp to the last 15 minutes",
  4.     processors=[
  5.         {
  6.             "set": {
  7.                 "field": "ingest_time",
  8.                 "value": "{{_ingest.timestamp}}"
  9.             }
  10.         },
  11.         {
  12.             "script": {
  13.                 "lang": "painless",
  14.                 "source": "\n          def delta = ChronoUnit.SECONDS.between(\n            ZonedDateTime.parse(\"2022-06-21T15:49:00Z\"),\n            ZonedDateTime.parse(ctx[\"ingest_time\"])\n          );\n          ctx[\"@timestamp\"] = ZonedDateTime.parse(ctx[\"@timestamp\"]).plus(delta,ChronoUnit.SECONDS).toString();\n        "
  15.             }
  16.         }
  17.     ],
  18. )
  19. print(resp)
  1. response = client.ingest.put_pipeline(
  2.   id: 'my-timestamp-pipeline',
  3.   body: {
  4.     description: 'Shifts the @timestamp to the last 15 minutes',
  5.     processors: [
  6.       {
  7.         set: {
  8.           field: 'ingest_time',
  9.           value: '{{_ingest.timestamp}}'
  10.         }
  11.       },
  12.       {
  13.         script: {
  14.           lang: 'painless',
  15.           source: "\n          def delta = ChronoUnit.SECONDS.between(\n            ZonedDateTime.parse(\"2022-06-21T15:49:00Z\"),\n            ZonedDateTime.parse(ctx[\"ingest_time\"])\n          );\n          ctx[\"@timestamp\"] = ZonedDateTime.parse(ctx[\"@timestamp\"]).plus(delta,ChronoUnit.SECONDS).toString();\n        "
  16.         }
  17.       }
  18.     ]
  19.   }
  20. )
  21. puts response
  1. const response = await client.ingest.putPipeline({
  2.   id: "my-timestamp-pipeline",
  3.   description: "Shifts the @timestamp to the last 15 minutes",
  4.   processors: [
  5.     {
  6.       set: {
  7.         field: "ingest_time",
  8.         value: "{{_ingest.timestamp}}",
  9.       },
  10.     },
  11.     {
  12.       script: {
  13.         lang: "painless",
  14.         source:
  15.           '\n          def delta = ChronoUnit.SECONDS.between(\n            ZonedDateTime.parse("2022-06-21T15:49:00Z"),\n            ZonedDateTime.parse(ctx["ingest_time"])\n          );\n          ctx["@timestamp"] = ZonedDateTime.parse(ctx["@timestamp"]).plus(delta,ChronoUnit.SECONDS).toString();\n        ',
  16.       },
  17.     },
  18.   ],
  19. });
  20. console.log(response);
  1. PUT _ingest/pipeline/my-timestamp-pipeline
  2. {
  3.   "description": "Shifts the @timestamp to the last 15 minutes",
  4.   "processors": [
  5.     {
  6.       "set": {
  7.         "field": "ingest_time",
  8.         "value": "{{_ingest.timestamp}}"
  9.       }
  10.     },
  11.     {
  12.       "script": {
  13.         "lang": "painless",
  14.         "source": """
  15.           def delta = ChronoUnit.SECONDS.between(
  16.             ZonedDateTime.parse("2022-06-21T15:49:00Z"),
  17.             ZonedDateTime.parse(ctx["ingest_time"])
  18.           );
  19.           ctx["@timestamp"] = ZonedDateTime.parse(ctx["@timestamp"]).plus(delta,ChronoUnit.SECONDS).toString();
  20.         """
  21.       }
  22.     }
  23.   ]
  24. }

Next, use a bulk API request to automatically create your TSDS and index a set of ten documents:

  1. resp = client.bulk(
  2.     index="my-data-stream",
  3.     refresh=True,
  4.     pipeline="my-timestamp-pipeline",
  5.     operations=[
  6.         {
  7.             "create": {}
  8.         },
  9.         {
  10.             "@timestamp": "2022-06-21T15:49:00Z",
  11.             "kubernetes": {
  12.                 "host": "gke-apps-0",
  13.                 "node": "gke-apps-0-0",
  14.                 "pod": "gke-apps-0-0-0",
  15.                 "container": {
  16.                     "cpu": {
  17.                         "usage": {
  18.                             "nanocores": 91153,
  19.                             "core": {
  20.                                 "ns": 12828317850
  21.                             },
  22.                             "node": {
  23.                                 "pct": 0.0000277905
  24.                             },
  25.                             "limit": {
  26.                                 "pct": 0.0000277905
  27.                             }
  28.                         }
  29.                     },
  30.                     "memory": {
  31.                         "available": {
  32.                             "bytes": 463314616
  33.                         },
  34.                         "usage": {
  35.                             "bytes": 307007078,
  36.                             "node": {
  37.                                 "pct": 0.01770037710617187
  38.                             },
  39.                             "limit": {
  40.                                 "pct": 0.00009923134671484496
  41.                             }
  42.                         },
  43.                         "workingset": {
  44.                             "bytes": 585236
  45.                         },
  46.                         "rss": {
  47.                             "bytes": 102728
  48.                         },
  49.                         "pagefaults": 120901,
  50.                         "majorpagefaults": 0
  51.                     },
  52.                     "start_time": "2021-03-30T07:59:06Z",
  53.                     "name": "container-name-44"
  54.                 },
  55.                 "namespace": "namespace26"
  56.             }
  57.         },
  58.         {
  59.             "create": {}
  60.         },
  61.         {
  62.             "@timestamp": "2022-06-21T15:45:50Z",
  63.             "kubernetes": {
  64.                 "host": "gke-apps-0",
  65.                 "node": "gke-apps-0-0",
  66.                 "pod": "gke-apps-0-0-0",
  67.                 "container": {
  68.                     "cpu": {
  69.                         "usage": {
  70.                             "nanocores": 124501,
  71.                             "core": {
  72.                                 "ns": 12828317850
  73.                             },
  74.                             "node": {
  75.                                 "pct": 0.0000277905
  76.                             },
  77.                             "limit": {
  78.                                 "pct": 0.0000277905
  79.                             }
  80.                         }
  81.                     },
  82.                     "memory": {
  83.                         "available": {
  84.                             "bytes": 982546514
  85.                         },
  86.                         "usage": {
  87.                             "bytes": 360035574,
  88.                             "node": {
  89.                                 "pct": 0.01770037710617187
  90.                             },
  91.                             "limit": {
  92.                                 "pct": 0.00009923134671484496
  93.                             }
  94.                         },
  95.                         "workingset": {
  96.                             "bytes": 1339884
  97.                         },
  98.                         "rss": {
  99.                             "bytes": 381174
  100.                         },
  101.                         "pagefaults": 178473,
  102.                         "majorpagefaults": 0
  103.                     },
  104.                     "start_time": "2021-03-30T07:59:06Z",
  105.                     "name": "container-name-44"
  106.                 },
  107.                 "namespace": "namespace26"
  108.             }
  109.         },
  110.         {
  111.             "create": {}
  112.         },
  113.         {
  114.             "@timestamp": "2022-06-21T15:44:50Z",
  115.             "kubernetes": {
  116.                 "host": "gke-apps-0",
  117.                 "node": "gke-apps-0-0",
  118.                 "pod": "gke-apps-0-0-0",
  119.                 "container": {
  120.                     "cpu": {
  121.                         "usage": {
  122.                             "nanocores": 38907,
  123.                             "core": {
  124.                                 "ns": 12828317850
  125.                             },
  126.                             "node": {
  127.                                 "pct": 0.0000277905
  128.                             },
  129.                             "limit": {
  130.                                 "pct": 0.0000277905
  131.                             }
  132.                         }
  133.                     },
  134.                     "memory": {
  135.                         "available": {
  136.                             "bytes": 862723768
  137.                         },
  138.                         "usage": {
  139.                             "bytes": 379572388,
  140.                             "node": {
  141.                                 "pct": 0.01770037710617187
  142.                             },
  143.                             "limit": {
  144.                                 "pct": 0.00009923134671484496
  145.                             }
  146.                         },
  147.                         "workingset": {
  148.                             "bytes": 431227
  149.                         },
  150.                         "rss": {
  151.                             "bytes": 386580
  152.                         },
  153.                         "pagefaults": 233166,
  154.                         "majorpagefaults": 0
  155.                     },
  156.                     "start_time": "2021-03-30T07:59:06Z",
  157.                     "name": "container-name-44"
  158.                 },
  159.                 "namespace": "namespace26"
  160.             }
  161.         },
  162.         {
  163.             "create": {}
  164.         },
  165.         {
  166.             "@timestamp": "2022-06-21T15:44:40Z",
  167.             "kubernetes": {
  168.                 "host": "gke-apps-0",
  169.                 "node": "gke-apps-0-0",
  170.                 "pod": "gke-apps-0-0-0",
  171.                 "container": {
  172.                     "cpu": {
  173.                         "usage": {
  174.                             "nanocores": 86706,
  175.                             "core": {
  176.                                 "ns": 12828317850
  177.                             },
  178.                             "node": {
  179.                                 "pct": 0.0000277905
  180.                             },
  181.                             "limit": {
  182.                                 "pct": 0.0000277905
  183.                             }
  184.                         }
  185.                     },
  186.                     "memory": {
  187.                         "available": {
  188.                             "bytes": 567160996
  189.                         },
  190.                         "usage": {
  191.                             "bytes": 103266017,
  192.                             "node": {
  193.                                 "pct": 0.01770037710617187
  194.                             },
  195.                             "limit": {
  196.                                 "pct": 0.00009923134671484496
  197.                             }
  198.                         },
  199.                         "workingset": {
  200.                             "bytes": 1724908
  201.                         },
  202.                         "rss": {
  203.                             "bytes": 105431
  204.                         },
  205.                         "pagefaults": 233166,
  206.                         "majorpagefaults": 0
  207.                     },
  208.                     "start_time": "2021-03-30T07:59:06Z",
  209.                     "name": "container-name-44"
  210.                 },
  211.                 "namespace": "namespace26"
  212.             }
  213.         },
  214.         {
  215.             "create": {}
  216.         },
  217.         {
  218.             "@timestamp": "2022-06-21T15:44:00Z",
  219.             "kubernetes": {
  220.                 "host": "gke-apps-0",
  221.                 "node": "gke-apps-0-0",
  222.                 "pod": "gke-apps-0-0-0",
  223.                 "container": {
  224.                     "cpu": {
  225.                         "usage": {
  226.                             "nanocores": 150069,
  227.                             "core": {
  228.                                 "ns": 12828317850
  229.                             },
  230.                             "node": {
  231.                                 "pct": 0.0000277905
  232.                             },
  233.                             "limit": {
  234.                                 "pct": 0.0000277905
  235.                             }
  236.                         }
  237.                     },
  238.                     "memory": {
  239.                         "available": {
  240.                             "bytes": 639054643
  241.                         },
  242.                         "usage": {
  243.                             "bytes": 265142477,
  244.                             "node": {
  245.                                 "pct": 0.01770037710617187
  246.                             },
  247.                             "limit": {
  248.                                 "pct": 0.00009923134671484496
  249.                             }
  250.                         },
  251.                         "workingset": {
  252.                             "bytes": 1786511
  253.                         },
  254.                         "rss": {
  255.                             "bytes": 189235
  256.                         },
  257.                         "pagefaults": 138172,
  258.                         "majorpagefaults": 0
  259.                     },
  260.                     "start_time": "2021-03-30T07:59:06Z",
  261.                     "name": "container-name-44"
  262.                 },
  263.                 "namespace": "namespace26"
  264.             }
  265.         },
  266.         {
  267.             "create": {}
  268.         },
  269.         {
  270.             "@timestamp": "2022-06-21T15:42:40Z",
  271.             "kubernetes": {
  272.                 "host": "gke-apps-0",
  273.                 "node": "gke-apps-0-0",
  274.                 "pod": "gke-apps-0-0-0",
  275.                 "container": {
  276.                     "cpu": {
  277.                         "usage": {
  278.                             "nanocores": 82260,
  279.                             "core": {
  280.                                 "ns": 12828317850
  281.                             },
  282.                             "node": {
  283.                                 "pct": 0.0000277905
  284.                             },
  285.                             "limit": {
  286.                                 "pct": 0.0000277905
  287.                             }
  288.                         }
  289.                     },
  290.                     "memory": {
  291.                         "available": {
  292.                             "bytes": 854735585
  293.                         },
  294.                         "usage": {
  295.                             "bytes": 309798052,
  296.                             "node": {
  297.                                 "pct": 0.01770037710617187
  298.                             },
  299.                             "limit": {
  300.                                 "pct": 0.00009923134671484496
  301.                             }
  302.                         },
  303.                         "workingset": {
  304.                             "bytes": 924058
  305.                         },
  306.                         "rss": {
  307.                             "bytes": 110838
  308.                         },
  309.                         "pagefaults": 259073,
  310.                         "majorpagefaults": 0
  311.                     },
  312.                     "start_time": "2021-03-30T07:59:06Z",
  313.                     "name": "container-name-44"
  314.                 },
  315.                 "namespace": "namespace26"
  316.             }
  317.         },
  318.         {
  319.             "create": {}
  320.         },
  321.         {
  322.             "@timestamp": "2022-06-21T15:42:10Z",
  323.             "kubernetes": {
  324.                 "host": "gke-apps-0",
  325.                 "node": "gke-apps-0-0",
  326.                 "pod": "gke-apps-0-0-0",
  327.                 "container": {
  328.                     "cpu": {
  329.                         "usage": {
  330.                             "nanocores": 153404,
  331.                             "core": {
  332.                                 "ns": 12828317850
  333.                             },
  334.                             "node": {
  335.                                 "pct": 0.0000277905
  336.                             },
  337.                             "limit": {
  338.                                 "pct": 0.0000277905
  339.                             }
  340.                         }
  341.                     },
  342.                     "memory": {
  343.                         "available": {
  344.                             "bytes": 279586406
  345.                         },
  346.                         "usage": {
  347.                             "bytes": 214904955,
  348.                             "node": {
  349.                                 "pct": 0.01770037710617187
  350.                             },
  351.                             "limit": {
  352.                                 "pct": 0.00009923134671484496
  353.                             }
  354.                         },
  355.                         "workingset": {
  356.                             "bytes": 1047265
  357.                         },
  358.                         "rss": {
  359.                             "bytes": 91914
  360.                         },
  361.                         "pagefaults": 302252,
  362.                         "majorpagefaults": 0
  363.                     },
  364.                     "start_time": "2021-03-30T07:59:06Z",
  365.                     "name": "container-name-44"
  366.                 },
  367.                 "namespace": "namespace26"
  368.             }
  369.         },
  370.         {
  371.             "create": {}
  372.         },
  373.         {
  374.             "@timestamp": "2022-06-21T15:40:20Z",
  375.             "kubernetes": {
  376.                 "host": "gke-apps-0",
  377.                 "node": "gke-apps-0-0",
  378.                 "pod": "gke-apps-0-0-0",
  379.                 "container": {
  380.                     "cpu": {
  381.                         "usage": {
  382.                             "nanocores": 125613,
  383.                             "core": {
  384.                                 "ns": 12828317850
  385.                             },
  386.                             "node": {
  387.                                 "pct": 0.0000277905
  388.                             },
  389.                             "limit": {
  390.                                 "pct": 0.0000277905
  391.                             }
  392.                         }
  393.                     },
  394.                     "memory": {
  395.                         "available": {
  396.                             "bytes": 822782853
  397.                         },
  398.                         "usage": {
  399.                             "bytes": 100475044,
  400.                             "node": {
  401.                                 "pct": 0.01770037710617187
  402.                             },
  403.                             "limit": {
  404.                                 "pct": 0.00009923134671484496
  405.                             }
  406.                         },
  407.                         "workingset": {
  408.                             "bytes": 2109932
  409.                         },
  410.                         "rss": {
  411.                             "bytes": 278446
  412.                         },
  413.                         "pagefaults": 74843,
  414.                         "majorpagefaults": 0
  415.                     },
  416.                     "start_time": "2021-03-30T07:59:06Z",
  417.                     "name": "container-name-44"
  418.                 },
  419.                 "namespace": "namespace26"
  420.             }
  421.         },
  422.         {
  423.             "create": {}
  424.         },
  425.         {
  426.             "@timestamp": "2022-06-21T15:40:10Z",
  427.             "kubernetes": {
  428.                 "host": "gke-apps-0",
  429.                 "node": "gke-apps-0-0",
  430.                 "pod": "gke-apps-0-0-0",
  431.                 "container": {
  432.                     "cpu": {
  433.                         "usage": {
  434.                             "nanocores": 100046,
  435.                             "core": {
  436.                                 "ns": 12828317850
  437.                             },
  438.                             "node": {
  439.                                 "pct": 0.0000277905
  440.                             },
  441.                             "limit": {
  442.                                 "pct": 0.0000277905
  443.                             }
  444.                         }
  445.                     },
  446.                     "memory": {
  447.                         "available": {
  448.                             "bytes": 567160996
  449.                         },
  450.                         "usage": {
  451.                             "bytes": 362826547,
  452.                             "node": {
  453.                                 "pct": 0.01770037710617187
  454.                             },
  455.                             "limit": {
  456.                                 "pct": 0.00009923134671484496
  457.                             }
  458.                         },
  459.                         "workingset": {
  460.                             "bytes": 1986724
  461.                         },
  462.                         "rss": {
  463.                             "bytes": 402801
  464.                         },
  465.                         "pagefaults": 296495,
  466.                         "majorpagefaults": 0
  467.                     },
  468.                     "start_time": "2021-03-30T07:59:06Z",
  469.                     "name": "container-name-44"
  470.                 },
  471.                 "namespace": "namespace26"
  472.             }
  473.         },
  474.         {
  475.             "create": {}
  476.         },
  477.         {
  478.             "@timestamp": "2022-06-21T15:38:30Z",
  479.             "kubernetes": {
  480.                 "host": "gke-apps-0",
  481.                 "node": "gke-apps-0-0",
  482.                 "pod": "gke-apps-0-0-0",
  483.                 "container": {
  484.                     "cpu": {
  485.                         "usage": {
  486.                             "nanocores": 40018,
  487.                             "core": {
  488.                                 "ns": 12828317850
  489.                             },
  490.                             "node": {
  491.                                 "pct": 0.0000277905
  492.                             },
  493.                             "limit": {
  494.                                 "pct": 0.0000277905
  495.                             }
  496.                         }
  497.                     },
  498.                     "memory": {
  499.                         "available": {
  500.                             "bytes": 1062428344
  501.                         },
  502.                         "usage": {
  503.                             "bytes": 265142477,
  504.                             "node": {
  505.                                 "pct": 0.01770037710617187
  506.                             },
  507.                             "limit": {
  508.                                 "pct": 0.00009923134671484496
  509.                             }
  510.                         },
  511.                         "workingset": {
  512.                             "bytes": 2294743
  513.                         },
  514.                         "rss": {
  515.                             "bytes": 340623
  516.                         },
  517.                         "pagefaults": 224530,
  518.                         "majorpagefaults": 0
  519.                     },
  520.                     "start_time": "2021-03-30T07:59:06Z",
  521.                     "name": "container-name-44"
  522.                 },
  523.                 "namespace": "namespace26"
  524.             }
  525.         }
  526.     ],
  527. )
  528. print(resp)
  1. response = client.bulk(
  2.   index: 'my-data-stream',
  3.   refresh: true,
  4.   pipeline: 'my-timestamp-pipeline',
  5.   body: [
  6.     {
  7.       create: {}
  8.     },
  9.     {
  10.       "@timestamp": '2022-06-21T15:49:00Z',
  11.       kubernetes: {
  12.         host: 'gke-apps-0',
  13.         node: 'gke-apps-0-0',
  14.         pod: 'gke-apps-0-0-0',
  15.         container: {
  16.           cpu: {
  17.             usage: {
  18.               nanocores: 91_153,
  19.               core: {
  20.                 ns: 12_828_317_850
  21.               },
  22.               node: {
  23.                 pct: 2.77905e-05
  24.               },
  25.               limit: {
  26.                 pct: 2.77905e-05
  27.               }
  28.             }
  29.           },
  30.           memory: {
  31.             available: {
  32.               bytes: 463_314_616
  33.             },
  34.             usage: {
  35.               bytes: 307_007_078,
  36.               node: {
  37.                 pct: 0.01770037710617187
  38.               },
  39.               limit: {
  40.                 pct: 9.923134671484496e-05
  41.               }
  42.             },
  43.             workingset: {
  44.               bytes: 585_236
  45.             },
  46.             rss: {
  47.               bytes: 102_728
  48.             },
  49.             pagefaults: 120_901,
  50.             majorpagefaults: 0
  51.           },
  52.           start_time: '2021-03-30T07:59:06Z',
  53.           name: 'container-name-44'
  54.         },
  55.         namespace: 'namespace26'
  56.       }
  57.     },
  58.     {
  59.       create: {}
  60.     },
  61.     {
  62.       "@timestamp": '2022-06-21T15:45:50Z',
  63.       kubernetes: {
  64.         host: 'gke-apps-0',
  65.         node: 'gke-apps-0-0',
  66.         pod: 'gke-apps-0-0-0',
  67.         container: {
  68.           cpu: {
  69.             usage: {
  70.               nanocores: 124_501,
  71.               core: {
  72.                 ns: 12_828_317_850
  73.               },
  74.               node: {
  75.                 pct: 2.77905e-05
  76.               },
  77.               limit: {
  78.                 pct: 2.77905e-05
  79.               }
  80.             }
  81.           },
  82.           memory: {
  83.             available: {
  84.               bytes: 982_546_514
  85.             },
  86.             usage: {
  87.               bytes: 360_035_574,
  88.               node: {
  89.                 pct: 0.01770037710617187
  90.               },
  91.               limit: {
  92.                 pct: 9.923134671484496e-05
  93.               }
  94.             },
  95.             workingset: {
  96.               bytes: 1_339_884
  97.             },
  98.             rss: {
  99.               bytes: 381_174
  100.             },
  101.             pagefaults: 178_473,
  102.             majorpagefaults: 0
  103.           },
  104.           start_time: '2021-03-30T07:59:06Z',
  105.           name: 'container-name-44'
  106.         },
  107.         namespace: 'namespace26'
  108.       }
  109.     },
  110.     {
  111.       create: {}
  112.     },
  113.     {
  114.       "@timestamp": '2022-06-21T15:44:50Z',
  115.       kubernetes: {
  116.         host: 'gke-apps-0',
  117.         node: 'gke-apps-0-0',
  118.         pod: 'gke-apps-0-0-0',
  119.         container: {
  120.           cpu: {
  121.             usage: {
  122.               nanocores: 38_907,
  123.               core: {
  124.                 ns: 12_828_317_850
  125.               },
  126.               node: {
  127.                 pct: 2.77905e-05
  128.               },
  129.               limit: {
  130.                 pct: 2.77905e-05
  131.               }
  132.             }
  133.           },
  134.           memory: {
  135.             available: {
  136.               bytes: 862_723_768
  137.             },
  138.             usage: {
  139.               bytes: 379_572_388,
  140.               node: {
  141.                 pct: 0.01770037710617187
  142.               },
  143.               limit: {
  144.                 pct: 9.923134671484496e-05
  145.               }
  146.             },
  147.             workingset: {
  148.               bytes: 431_227
  149.             },
  150.             rss: {
  151.               bytes: 386_580
  152.             },
  153.             pagefaults: 233_166,
  154.             majorpagefaults: 0
  155.           },
  156.           start_time: '2021-03-30T07:59:06Z',
  157.           name: 'container-name-44'
  158.         },
  159.         namespace: 'namespace26'
  160.       }
  161.     },
  162.     {
  163.       create: {}
  164.     },
  165.     {
  166.       "@timestamp": '2022-06-21T15:44:40Z',
  167.       kubernetes: {
  168.         host: 'gke-apps-0',
  169.         node: 'gke-apps-0-0',
  170.         pod: 'gke-apps-0-0-0',
  171.         container: {
  172.           cpu: {
  173.             usage: {
  174.               nanocores: 86_706,
  175.               core: {
  176.                 ns: 12_828_317_850
  177.               },
  178.               node: {
  179.                 pct: 2.77905e-05
  180.               },
  181.               limit: {
  182.                 pct: 2.77905e-05
  183.               }
  184.             }
  185.           },
  186.           memory: {
  187.             available: {
  188.               bytes: 567_160_996
  189.             },
  190.             usage: {
  191.               bytes: 103_266_017,
  192.               node: {
  193.                 pct: 0.01770037710617187
  194.               },
  195.               limit: {
  196.                 pct: 9.923134671484496e-05
  197.               }
  198.             },
  199.             workingset: {
  200.               bytes: 1_724_908
  201.             },
  202.             rss: {
  203.               bytes: 105_431
  204.             },
  205.             pagefaults: 233_166,
  206.             majorpagefaults: 0
  207.           },
  208.           start_time: '2021-03-30T07:59:06Z',
  209.           name: 'container-name-44'
  210.         },
  211.         namespace: 'namespace26'
  212.       }
  213.     },
  214.     {
  215.       create: {}
  216.     },
  217.     {
  218.       "@timestamp": '2022-06-21T15:44:00Z',
  219.       kubernetes: {
  220.         host: 'gke-apps-0',
  221.         node: 'gke-apps-0-0',
  222.         pod: 'gke-apps-0-0-0',
  223.         container: {
  224.           cpu: {
  225.             usage: {
  226.               nanocores: 150_069,
  227.               core: {
  228.                 ns: 12_828_317_850
  229.               },
  230.               node: {
  231.                 pct: 2.77905e-05
  232.               },
  233.               limit: {
  234.                 pct: 2.77905e-05
  235.               }
  236.             }
  237.           },
  238.           memory: {
  239.             available: {
  240.               bytes: 639_054_643
  241.             },
  242.             usage: {
  243.               bytes: 265_142_477,
  244.               node: {
  245.                 pct: 0.01770037710617187
  246.               },
  247.               limit: {
  248.                 pct: 9.923134671484496e-05
  249.               }
  250.             },
  251.             workingset: {
  252.               bytes: 1_786_511
  253.             },
  254.             rss: {
  255.               bytes: 189_235
  256.             },
  257.             pagefaults: 138_172,
  258.             majorpagefaults: 0
  259.           },
  260.           start_time: '2021-03-30T07:59:06Z',
  261.           name: 'container-name-44'
  262.         },
  263.         namespace: 'namespace26'
  264.       }
  265.     },
  266.     {
  267.       create: {}
  268.     },
  269.     {
  270.       "@timestamp": '2022-06-21T15:42:40Z',
  271.       kubernetes: {
  272.         host: 'gke-apps-0',
  273.         node: 'gke-apps-0-0',
  274.         pod: 'gke-apps-0-0-0',
  275.         container: {
  276.           cpu: {
  277.             usage: {
  278.               nanocores: 82_260,
  279.               core: {
  280.                 ns: 12_828_317_850
  281.               },
  282.               node: {
  283.                 pct: 2.77905e-05
  284.               },
  285.               limit: {
  286.                 pct: 2.77905e-05
  287.               }
  288.             }
  289.           },
  290.           memory: {
  291.             available: {
  292.               bytes: 854_735_585
  293.             },
  294.             usage: {
  295.               bytes: 309_798_052,
  296.               node: {
  297.                 pct: 0.01770037710617187
  298.               },
  299.               limit: {
  300.                 pct: 9.923134671484496e-05
  301.               }
  302.             },
  303.             workingset: {
  304.               bytes: 924_058
  305.             },
  306.             rss: {
  307.               bytes: 110_838
  308.             },
  309.             pagefaults: 259_073,
  310.             majorpagefaults: 0
  311.           },
  312.           start_time: '2021-03-30T07:59:06Z',
  313.           name: 'container-name-44'
  314.         },
  315.         namespace: 'namespace26'
  316.       }
  317.     },
  318.     {
  319.       create: {}
  320.     },
  321.     {
  322.       "@timestamp": '2022-06-21T15:42:10Z',
  323.       kubernetes: {
  324.         host: 'gke-apps-0',
  325.         node: 'gke-apps-0-0',
  326.         pod: 'gke-apps-0-0-0',
  327.         container: {
  328.           cpu: {
  329.             usage: {
  330.               nanocores: 153_404,
  331.               core: {
  332.                 ns: 12_828_317_850
  333.               },
  334.               node: {
  335.                 pct: 2.77905e-05
  336.               },
  337.               limit: {
  338.                 pct: 2.77905e-05
  339.               }
  340.             }
  341.           },
  342.           memory: {
  343.             available: {
  344.               bytes: 279_586_406
  345.             },
  346.             usage: {
  347.               bytes: 214_904_955,
  348.               node: {
  349.                 pct: 0.01770037710617187
  350.               },
  351.               limit: {
  352.                 pct: 9.923134671484496e-05
  353.               }
  354.             },
  355.             workingset: {
  356.               bytes: 1_047_265
  357.             },
  358.             rss: {
  359.               bytes: 91_914
  360.             },
  361.             pagefaults: 302_252,
  362.             majorpagefaults: 0
  363.           },
  364.           start_time: '2021-03-30T07:59:06Z',
  365.           name: 'container-name-44'
  366.         },
  367.         namespace: 'namespace26'
  368.       }
  369.     },
  370.     {
  371.       create: {}
  372.     },
  373.     {
  374.       "@timestamp": '2022-06-21T15:40:20Z',
  375.       kubernetes: {
  376.         host: 'gke-apps-0',
  377.         node: 'gke-apps-0-0',
  378.         pod: 'gke-apps-0-0-0',
  379.         container: {
  380.           cpu: {
  381.             usage: {
  382.               nanocores: 125_613,
  383.               core: {
  384.                 ns: 12_828_317_850
  385.               },
  386.               node: {
  387.                 pct: 2.77905e-05
  388.               },
  389.               limit: {
  390.                 pct: 2.77905e-05
  391.               }
  392.             }
  393.           },
  394.           memory: {
  395.             available: {
  396.               bytes: 822_782_853
  397.             },
  398.             usage: {
  399.               bytes: 100_475_044,
  400.               node: {
  401.                 pct: 0.01770037710617187
  402.               },
  403.               limit: {
  404.                 pct: 9.923134671484496e-05
  405.               }
  406.             },
  407.             workingset: {
  408.               bytes: 2_109_932
  409.             },
  410.             rss: {
  411.               bytes: 278_446
  412.             },
  413.             pagefaults: 74_843,
  414.             majorpagefaults: 0
  415.           },
  416.           start_time: '2021-03-30T07:59:06Z',
  417.           name: 'container-name-44'
  418.         },
  419.         namespace: 'namespace26'
  420.       }
  421.     },
  422.     {
  423.       create: {}
  424.     },
  425.     {
  426.       "@timestamp": '2022-06-21T15:40:10Z',
  427.       kubernetes: {
  428.         host: 'gke-apps-0',
  429.         node: 'gke-apps-0-0',
  430.         pod: 'gke-apps-0-0-0',
  431.         container: {
  432.           cpu: {
  433.             usage: {
  434.               nanocores: 100_046,
  435.               core: {
  436.                 ns: 12_828_317_850
  437.               },
  438.               node: {
  439.                 pct: 2.77905e-05
  440.               },
  441.               limit: {
  442.                 pct: 2.77905e-05
  443.               }
  444.             }
  445.           },
  446.           memory: {
  447.             available: {
  448.               bytes: 567_160_996
  449.             },
  450.             usage: {
  451.               bytes: 362_826_547,
  452.               node: {
  453.                 pct: 0.01770037710617187
  454.               },
  455.               limit: {
  456.                 pct: 9.923134671484496e-05
  457.               }
  458.             },
  459.             workingset: {
  460.               bytes: 1_986_724
  461.             },
  462.             rss: {
  463.               bytes: 402_801
  464.             },
  465.             pagefaults: 296_495,
  466.             majorpagefaults: 0
  467.           },
  468.           start_time: '2021-03-30T07:59:06Z',
  469.           name: 'container-name-44'
  470.         },
  471.         namespace: 'namespace26'
  472.       }
  473.     },
  474.     {
  475.       create: {}
  476.     },
  477.     {
  478.       "@timestamp": '2022-06-21T15:38:30Z',
  479.       kubernetes: {
  480.         host: 'gke-apps-0',
  481.         node: 'gke-apps-0-0',
  482.         pod: 'gke-apps-0-0-0',
  483.         container: {
  484.           cpu: {
  485.             usage: {
  486.               nanocores: 40_018,
  487.               core: {
  488.                 ns: 12_828_317_850
  489.               },
  490.               node: {
  491.                 pct: 2.77905e-05
  492.               },
  493.               limit: {
  494.                 pct: 2.77905e-05
  495.               }
  496.             }
  497.           },
  498.           memory: {
  499.             available: {
  500.               bytes: 1_062_428_344
  501.             },
  502.             usage: {
  503.               bytes: 265_142_477,
  504.               node: {
  505.                 pct: 0.01770037710617187
  506.               },
  507.               limit: {
  508.                 pct: 9.923134671484496e-05
  509.               }
  510.             },
  511.             workingset: {
  512.               bytes: 2_294_743
  513.             },
  514.             rss: {
  515.               bytes: 340_623
  516.             },
  517.             pagefaults: 224_530,
  518.             majorpagefaults: 0
  519.           },
  520.           start_time: '2021-03-30T07:59:06Z',
  521.           name: 'container-name-44'
  522.         },
  523.         namespace: 'namespace26'
  524.       }
  525.     }
  526.   ]
  527. )
  528. puts response
  1. const response = await client.bulk({
  2.   index: "my-data-stream",
  3.   refresh: "true",
  4.   pipeline: "my-timestamp-pipeline",
  5.   operations: [
  6.     {
  7.       create: {},
  8.     },
  9.     {
  10.       "@timestamp": "2022-06-21T15:49:00Z",
  11.       kubernetes: {
  12.         host: "gke-apps-0",
  13.         node: "gke-apps-0-0",
  14.         pod: "gke-apps-0-0-0",
  15.         container: {
  16.           cpu: {
  17.             usage: {
  18.               nanocores: 91153,
  19.               core: {
  20.                 ns: 12828317850,
  21.               },
  22.               node: {
  23.                 pct: 0.0000277905,
  24.               },
  25.               limit: {
  26.                 pct: 0.0000277905,
  27.               },
  28.             },
  29.           },
  30.           memory: {
  31.             available: {
  32.               bytes: 463314616,
  33.             },
  34.             usage: {
  35.               bytes: 307007078,
  36.               node: {
  37.                 pct: 0.01770037710617187,
  38.               },
  39.               limit: {
  40.                 pct: 0.00009923134671484496,
  41.               },
  42.             },
  43.             workingset: {
  44.               bytes: 585236,
  45.             },
  46.             rss: {
  47.               bytes: 102728,
  48.             },
  49.             pagefaults: 120901,
  50.             majorpagefaults: 0,
  51.           },
  52.           start_time: "2021-03-30T07:59:06Z",
  53.           name: "container-name-44",
  54.         },
  55.         namespace: "namespace26",
  56.       },
  57.     },
  58.     {
  59.       create: {},
  60.     },
  61.     {
  62.       "@timestamp": "2022-06-21T15:45:50Z",
  63.       kubernetes: {
  64.         host: "gke-apps-0",
  65.         node: "gke-apps-0-0",
  66.         pod: "gke-apps-0-0-0",
  67.         container: {
  68.           cpu: {
  69.             usage: {
  70.               nanocores: 124501,
  71.               core: {
  72.                 ns: 12828317850,
  73.               },
  74.               node: {
  75.                 pct: 0.0000277905,
  76.               },
  77.               limit: {
  78.                 pct: 0.0000277905,
  79.               },
  80.             },
  81.           },
  82.           memory: {
  83.             available: {
  84.               bytes: 982546514,
  85.             },
  86.             usage: {
  87.               bytes: 360035574,
  88.               node: {
  89.                 pct: 0.01770037710617187,
  90.               },
  91.               limit: {
  92.                 pct: 0.00009923134671484496,
  93.               },
  94.             },
  95.             workingset: {
  96.               bytes: 1339884,
  97.             },
  98.             rss: {
  99.               bytes: 381174,
  100.             },
  101.             pagefaults: 178473,
  102.             majorpagefaults: 0,
  103.           },
  104.           start_time: "2021-03-30T07:59:06Z",
  105.           name: "container-name-44",
  106.         },
  107.         namespace: "namespace26",
  108.       },
  109.     },
  110.     {
  111.       create: {},
  112.     },
  113.     {
  114.       "@timestamp": "2022-06-21T15:44:50Z",
  115.       kubernetes: {
  116.         host: "gke-apps-0",
  117.         node: "gke-apps-0-0",
  118.         pod: "gke-apps-0-0-0",
  119.         container: {
  120.           cpu: {
  121.             usage: {
  122.               nanocores: 38907,
  123.               core: {
  124.                 ns: 12828317850,
  125.               },
  126.               node: {
  127.                 pct: 0.0000277905,
  128.               },
  129.               limit: {
  130.                 pct: 0.0000277905,
  131.               },
  132.             },
  133.           },
  134.           memory: {
  135.             available: {
  136.               bytes: 862723768,
  137.             },
  138.             usage: {
  139.               bytes: 379572388,
  140.               node: {
  141.                 pct: 0.01770037710617187,
  142.               },
  143.               limit: {
  144.                 pct: 0.00009923134671484496,
  145.               },
  146.             },
  147.             workingset: {
  148.               bytes: 431227,
  149.             },
  150.             rss: {
  151.               bytes: 386580,
  152.             },
  153.             pagefaults: 233166,
  154.             majorpagefaults: 0,
  155.           },
  156.           start_time: "2021-03-30T07:59:06Z",
  157.           name: "container-name-44",
  158.         },
  159.         namespace: "namespace26",
  160.       },
  161.     },
  162.     {
  163.       create: {},
  164.     },
  165.     {
  166.       "@timestamp": "2022-06-21T15:44:40Z",
  167.       kubernetes: {
  168.         host: "gke-apps-0",
  169.         node: "gke-apps-0-0",
  170.         pod: "gke-apps-0-0-0",
  171.         container: {
  172.           cpu: {
  173.             usage: {
  174.               nanocores: 86706,
  175.               core: {
  176.                 ns: 12828317850,
  177.               },
  178.               node: {
  179.                 pct: 0.0000277905,
  180.               },
  181.               limit: {
  182.                 pct: 0.0000277905,
  183.               },
  184.             },
  185.           },
  186.           memory: {
  187.             available: {
  188.               bytes: 567160996,
  189.             },
  190.             usage: {
  191.               bytes: 103266017,
  192.               node: {
  193.                 pct: 0.01770037710617187,
  194.               },
  195.               limit: {
  196.                 pct: 0.00009923134671484496,
  197.               },
  198.             },
  199.             workingset: {
  200.               bytes: 1724908,
  201.             },
  202.             rss: {
  203.               bytes: 105431,
  204.             },
  205.             pagefaults: 233166,
  206.             majorpagefaults: 0,
  207.           },
  208.           start_time: "2021-03-30T07:59:06Z",
  209.           name: "container-name-44",
  210.         },
  211.         namespace: "namespace26",
  212.       },
  213.     },
  214.     {
  215.       create: {},
  216.     },
  217.     {
  218.       "@timestamp": "2022-06-21T15:44:00Z",
  219.       kubernetes: {
  220.         host: "gke-apps-0",
  221.         node: "gke-apps-0-0",
  222.         pod: "gke-apps-0-0-0",
  223.         container: {
  224.           cpu: {
  225.             usage: {
  226.               nanocores: 150069,
  227.               core: {
  228.                 ns: 12828317850,
  229.               },
  230.               node: {
  231.                 pct: 0.0000277905,
  232.               },
  233.               limit: {
  234.                 pct: 0.0000277905,
  235.               },
  236.             },
  237.           },
  238.           memory: {
  239.             available: {
  240.               bytes: 639054643,
  241.             },
  242.             usage: {
  243.               bytes: 265142477,
  244.               node: {
  245.                 pct: 0.01770037710617187,
  246.               },
  247.               limit: {
  248.                 pct: 0.00009923134671484496,
  249.               },
  250.             },
  251.             workingset: {
  252.               bytes: 1786511,
  253.             },
  254.             rss: {
  255.               bytes: 189235,
  256.             },
  257.             pagefaults: 138172,
  258.             majorpagefaults: 0,
  259.           },
  260.           start_time: "2021-03-30T07:59:06Z",
  261.           name: "container-name-44",
  262.         },
  263.         namespace: "namespace26",
  264.       },
  265.     },
  266.     {
  267.       create: {},
  268.     },
  269.     {
  270.       "@timestamp": "2022-06-21T15:42:40Z",
  271.       kubernetes: {
  272.         host: "gke-apps-0",
  273.         node: "gke-apps-0-0",
  274.         pod: "gke-apps-0-0-0",
  275.         container: {
  276.           cpu: {
  277.             usage: {
  278.               nanocores: 82260,
  279.               core: {
  280.                 ns: 12828317850,
  281.               },
  282.               node: {
  283.                 pct: 0.0000277905,
  284.               },
  285.               limit: {
  286.                 pct: 0.0000277905,
  287.               },
  288.             },
  289.           },
  290.           memory: {
  291.             available: {
  292.               bytes: 854735585,
  293.             },
  294.             usage: {
  295.               bytes: 309798052,
  296.               node: {
  297.                 pct: 0.01770037710617187,
  298.               },
  299.               limit: {
  300.                 pct: 0.00009923134671484496,
  301.               },
  302.             },
  303.             workingset: {
  304.               bytes: 924058,
  305.             },
  306.             rss: {
  307.               bytes: 110838,
  308.             },
  309.             pagefaults: 259073,
  310.             majorpagefaults: 0,
  311.           },
  312.           start_time: "2021-03-30T07:59:06Z",
  313.           name: "container-name-44",
  314.         },
  315.         namespace: "namespace26",
  316.       },
  317.     },
  318.     {
  319.       create: {},
  320.     },
  321.     {
  322.       "@timestamp": "2022-06-21T15:42:10Z",
  323.       kubernetes: {
  324.         host: "gke-apps-0",
  325.         node: "gke-apps-0-0",
  326.         pod: "gke-apps-0-0-0",
  327.         container: {
  328.           cpu: {
  329.             usage: {
  330.               nanocores: 153404,
  331.               core: {
  332.                 ns: 12828317850,
  333.               },
  334.               node: {
  335.                 pct: 0.0000277905,
  336.               },
  337.               limit: {
  338.                 pct: 0.0000277905,
  339.               },
  340.             },
  341.           },
  342.           memory: {
  343.             available: {
  344.               bytes: 279586406,
  345.             },
  346.             usage: {
  347.               bytes: 214904955,
  348.               node: {
  349.                 pct: 0.01770037710617187,
  350.               },
  351.               limit: {
  352.                 pct: 0.00009923134671484496,
  353.               },
  354.             },
  355.             workingset: {
  356.               bytes: 1047265,
  357.             },
  358.             rss: {
  359.               bytes: 91914,
  360.             },
  361.             pagefaults: 302252,
  362.             majorpagefaults: 0,
  363.           },
  364.           start_time: "2021-03-30T07:59:06Z",
  365.           name: "container-name-44",
  366.         },
  367.         namespace: "namespace26",
  368.       },
  369.     },
  370.     {
  371.       create: {},
  372.     },
  373.     {
  374.       "@timestamp": "2022-06-21T15:40:20Z",
  375.       kubernetes: {
  376.         host: "gke-apps-0",
  377.         node: "gke-apps-0-0",
  378.         pod: "gke-apps-0-0-0",
  379.         container: {
  380.           cpu: {
  381.             usage: {
  382.               nanocores: 125613,
  383.               core: {
  384.                 ns: 12828317850,
  385.               },
  386.               node: {
  387.                 pct: 0.0000277905,
  388.               },
  389.               limit: {
  390.                 pct: 0.0000277905,
  391.               },
  392.             },
  393.           },
  394.           memory: {
  395.             available: {
  396.               bytes: 822782853,
  397.             },
  398.             usage: {
  399.               bytes: 100475044,
  400.               node: {
  401.                 pct: 0.01770037710617187,
  402.               },
  403.               limit: {
  404.                 pct: 0.00009923134671484496,
  405.               },
  406.             },
  407.             workingset: {
  408.               bytes: 2109932,
  409.             },
  410.             rss: {
  411.               bytes: 278446,
  412.             },
  413.             pagefaults: 74843,
  414.             majorpagefaults: 0,
  415.           },
  416.           start_time: "2021-03-30T07:59:06Z",
  417.           name: "container-name-44",
  418.         },
  419.         namespace: "namespace26",
  420.       },
  421.     },
  422.     {
  423.       create: {},
  424.     },
  425.     {
  426.       "@timestamp": "2022-06-21T15:40:10Z",
  427.       kubernetes: {
  428.         host: "gke-apps-0",
  429.         node: "gke-apps-0-0",
  430.         pod: "gke-apps-0-0-0",
  431.         container: {
  432.           cpu: {
  433.             usage: {
  434.               nanocores: 100046,
  435.               core: {
  436.                 ns: 12828317850,
  437.               },
  438.               node: {
  439.                 pct: 0.0000277905,
  440.               },
  441.               limit: {
  442.                 pct: 0.0000277905,
  443.               },
  444.             },
  445.           },
  446.           memory: {
  447.             available: {
  448.               bytes: 567160996,
  449.             },
  450.             usage: {
  451.               bytes: 362826547,
  452.               node: {
  453.                 pct: 0.01770037710617187,
  454.               },
  455.               limit: {
  456.                 pct: 0.00009923134671484496,
  457.               },
  458.             },
  459.             workingset: {
  460.               bytes: 1986724,
  461.             },
  462.             rss: {
  463.               bytes: 402801,
  464.             },
  465.             pagefaults: 296495,
  466.             majorpagefaults: 0,
  467.           },
  468.           start_time: "2021-03-30T07:59:06Z",
  469.           name: "container-name-44",
  470.         },
  471.         namespace: "namespace26",
  472.       },
  473.     },
  474.     {
  475.       create: {},
  476.     },
  477.     {
  478.       "@timestamp": "2022-06-21T15:38:30Z",
  479.       kubernetes: {
  480.         host: "gke-apps-0",
  481.         node: "gke-apps-0-0",
  482.         pod: "gke-apps-0-0-0",
  483.         container: {
  484.           cpu: {
  485.             usage: {
  486.               nanocores: 40018,
  487.               core: {
  488.                 ns: 12828317850,
  489.               },
  490.               node: {
  491.                 pct: 0.0000277905,
  492.               },
  493.               limit: {
  494.                 pct: 0.0000277905,
  495.               },
  496.             },
  497.           },
  498.           memory: {
  499.             available: {
  500.               bytes: 1062428344,
  501.             },
  502.             usage: {
  503.               bytes: 265142477,
  504.               node: {
  505.                 pct: 0.01770037710617187,
  506.               },
  507.               limit: {
  508.                 pct: 0.00009923134671484496,
  509.               },
  510.             },
  511.             workingset: {
  512.               bytes: 2294743,
  513.             },
  514.             rss: {
  515.               bytes: 340623,
  516.             },
  517.             pagefaults: 224530,
  518.             majorpagefaults: 0,
  519.           },
  520.           start_time: "2021-03-30T07:59:06Z",
  521.           name: "container-name-44",
  522.         },
  523.         namespace: "namespace26",
  524.       },
  525.     },
  526.   ],
  527. });
  528. console.log(response);
  1. PUT /my-data-stream/_bulk?refresh&pipeline=my-timestamp-pipeline
  2. {"create": {}}
  3. {"@timestamp":"2022-06-21T15:49:00Z","kubernetes":{"host":"gke-apps-0","node":"gke-apps-0-0","pod":"gke-apps-0-0-0","container":{"cpu":{"usage":{"nanocores":91153,"core":{"ns":12828317850},"node":{"pct":2.77905e-05},"limit":{"pct":2.77905e-05}}},"memory":{"available":{"bytes":463314616},"usage":{"bytes":307007078,"node":{"pct":0.01770037710617187},"limit":{"pct":9.923134671484496e-05}},"workingset":{"bytes":585236},"rss":{"bytes":102728},"pagefaults":120901,"majorpagefaults":0},"start_time":"2021-03-30T07:59:06Z","name":"container-name-44"},"namespace":"namespace26"}}
  4. {"create": {}}
  5. {"@timestamp":"2022-06-21T15:45:50Z","kubernetes":{"host":"gke-apps-0","node":"gke-apps-0-0","pod":"gke-apps-0-0-0","container":{"cpu":{"usage":{"nanocores":124501,"core":{"ns":12828317850},"node":{"pct":2.77905e-05},"limit":{"pct":2.77905e-05}}},"memory":{"available":{"bytes":982546514},"usage":{"bytes":360035574,"node":{"pct":0.01770037710617187},"limit":{"pct":9.923134671484496e-05}},"workingset":{"bytes":1339884},"rss":{"bytes":381174},"pagefaults":178473,"majorpagefaults":0},"start_time":"2021-03-30T07:59:06Z","name":"container-name-44"},"namespace":"namespace26"}}
  6. {"create": {}}
  7. {"@timestamp":"2022-06-21T15:44:50Z","kubernetes":{"host":"gke-apps-0","node":"gke-apps-0-0","pod":"gke-apps-0-0-0","container":{"cpu":{"usage":{"nanocores":38907,"core":{"ns":12828317850},"node":{"pct":2.77905e-05},"limit":{"pct":2.77905e-05}}},"memory":{"available":{"bytes":862723768},"usage":{"bytes":379572388,"node":{"pct":0.01770037710617187},"limit":{"pct":9.923134671484496e-05}},"workingset":{"bytes":431227},"rss":{"bytes":386580},"pagefaults":233166,"majorpagefaults":0},"start_time":"2021-03-30T07:59:06Z","name":"container-name-44"},"namespace":"namespace26"}}
  8. {"create": {}}
  9. {"@timestamp":"2022-06-21T15:44:40Z","kubernetes":{"host":"gke-apps-0","node":"gke-apps-0-0","pod":"gke-apps-0-0-0","container":{"cpu":{"usage":{"nanocores":86706,"core":{"ns":12828317850},"node":{"pct":2.77905e-05},"limit":{"pct":2.77905e-05}}},"memory":{"available":{"bytes":567160996},"usage":{"bytes":103266017,"node":{"pct":0.01770037710617187},"limit":{"pct":9.923134671484496e-05}},"workingset":{"bytes":1724908},"rss":{"bytes":105431},"pagefaults":233166,"majorpagefaults":0},"start_time":"2021-03-30T07:59:06Z","name":"container-name-44"},"namespace":"namespace26"}}
  10. {"create": {}}
  11. {"@timestamp":"2022-06-21T15:44:00Z","kubernetes":{"host":"gke-apps-0","node":"gke-apps-0-0","pod":"gke-apps-0-0-0","container":{"cpu":{"usage":{"nanocores":150069,"core":{"ns":12828317850},"node":{"pct":2.77905e-05},"limit":{"pct":2.77905e-05}}},"memory":{"available":{"bytes":639054643},"usage":{"bytes":265142477,"node":{"pct":0.01770037710617187},"limit":{"pct":9.923134671484496e-05}},"workingset":{"bytes":1786511},"rss":{"bytes":189235},"pagefaults":138172,"majorpagefaults":0},"start_time":"2021-03-30T07:59:06Z","name":"container-name-44"},"namespace":"namespace26"}}
  12. {"create": {}}
  13. {"@timestamp":"2022-06-21T15:42:40Z","kubernetes":{"host":"gke-apps-0","node":"gke-apps-0-0","pod":"gke-apps-0-0-0","container":{"cpu":{"usage":{"nanocores":82260,"core":{"ns":12828317850},"node":{"pct":2.77905e-05},"limit":{"pct":2.77905e-05}}},"memory":{"available":{"bytes":854735585},"usage":{"bytes":309798052,"node":{"pct":0.01770037710617187},"limit":{"pct":9.923134671484496e-05}},"workingset":{"bytes":924058},"rss":{"bytes":110838},"pagefaults":259073,"majorpagefaults":0},"start_time":"2021-03-30T07:59:06Z","name":"container-name-44"},"namespace":"namespace26"}}
  14. {"create": {}}
  15. {"@timestamp":"2022-06-21T15:42:10Z","kubernetes":{"host":"gke-apps-0","node":"gke-apps-0-0","pod":"gke-apps-0-0-0","container":{"cpu":{"usage":{"nanocores":153404,"core":{"ns":12828317850},"node":{"pct":2.77905e-05},"limit":{"pct":2.77905e-05}}},"memory":{"available":{"bytes":279586406},"usage":{"bytes":214904955,"node":{"pct":0.01770037710617187},"limit":{"pct":9.923134671484496e-05}},"workingset":{"bytes":1047265},"rss":{"bytes":91914},"pagefaults":302252,"majorpagefaults":0},"start_time":"2021-03-30T07:59:06Z","name":"container-name-44"},"namespace":"namespace26"}}
  16. {"create": {}}
  17. {"@timestamp":"2022-06-21T15:40:20Z","kubernetes":{"host":"gke-apps-0","node":"gke-apps-0-0","pod":"gke-apps-0-0-0","container":{"cpu":{"usage":{"nanocores":125613,"core":{"ns":12828317850},"node":{"pct":2.77905e-05},"limit":{"pct":2.77905e-05}}},"memory":{"available":{"bytes":822782853},"usage":{"bytes":100475044,"node":{"pct":0.01770037710617187},"limit":{"pct":9.923134671484496e-05}},"workingset":{"bytes":2109932},"rss":{"bytes":278446},"pagefaults":74843,"majorpagefaults":0},"start_time":"2021-03-30T07:59:06Z","name":"container-name-44"},"namespace":"namespace26"}}
  18. {"create": {}}
  19. {"@timestamp":"2022-06-21T15:40:10Z","kubernetes":{"host":"gke-apps-0","node":"gke-apps-0-0","pod":"gke-apps-0-0-0","container":{"cpu":{"usage":{"nanocores":100046,"core":{"ns":12828317850},"node":{"pct":2.77905e-05},"limit":{"pct":2.77905e-05}}},"memory":{"available":{"bytes":567160996},"usage":{"bytes":362826547,"node":{"pct":0.01770037710617187},"limit":{"pct":9.923134671484496e-05}},"workingset":{"bytes":1986724},"rss":{"bytes":402801},"pagefaults":296495,"majorpagefaults":0},"start_time":"2021-03-30T07:59:06Z","name":"container-name-44"},"namespace":"namespace26"}}
  20. {"create": {}}
  21. {"@timestamp":"2022-06-21T15:38:30Z","kubernetes":{"host":"gke-apps-0","node":"gke-apps-0-0","pod":"gke-apps-0-0-0","container":{"cpu":{"usage":{"nanocores":40018,"core":{"ns":12828317850},"node":{"pct":2.77905e-05},"limit":{"pct":2.77905e-05}}},"memory":{"available":{"bytes":1062428344},"usage":{"bytes":265142477,"node":{"pct":0.01770037710617187},"limit":{"pct":9.923134671484496e-05}},"workingset":{"bytes":2294743},"rss":{"bytes":340623},"pagefaults":224530,"majorpagefaults":0},"start_time":"2021-03-30T07:59:06Z","name":"container-name-44"},"namespace":"namespace26"}}

You can use the search API to check if the documents have been indexed correctly:

  1. resp = client.search(
  2.     index="my-data-stream",
  3. )
  4. print(resp)
  1. response = client.search(
  2.   index: 'my-data-stream'
  3. )
  4. puts response
  1. const response = await client.search({
  2.   index: "my-data-stream",
  3. });
  4. console.log(response);
  1. GET /my-data-stream/_search

Run the following aggregation on the data to calculate some interesting statistics:

  1. resp = client.search(
  2.     index="my-data-stream",
  3.     size=0,
  4.     aggs={
  5.         "tsid": {
  6.             "terms": {
  7.                 "field": "_tsid"
  8.             },
  9.             "aggs": {
  10.                 "over_time": {
  11.                     "date_histogram": {
  12.                         "field": "@timestamp",
  13.                         "fixed_interval": "1d"
  14.                     },
  15.                     "aggs": {
  16.                         "min": {
  17.                             "min": {
  18.                                 "field": "kubernetes.container.memory.usage.bytes"
  19.                             }
  20.                         },
  21.                         "max": {
  22.                             "max": {
  23.                                 "field": "kubernetes.container.memory.usage.bytes"
  24.                             }
  25.                         },
  26.                         "avg": {
  27.                             "avg": {
  28.                                 "field": "kubernetes.container.memory.usage.bytes"
  29.                             }
  30.                         }
  31.                     }
  32.                 }
  33.             }
  34.         }
  35.     },
  36. )
  37. print(resp)
  1. response = client.search(
  2.   index: 'my-data-stream',
  3.   body: {
  4.     size: 0,
  5.     aggregations: {
  6.       tsid: {
  7.         terms: {
  8.           field: '_tsid'
  9.         },
  10.         aggregations: {
  11.           over_time: {
  12.             date_histogram: {
  13.               field: '@timestamp',
  14.               fixed_interval: '1d'
  15.             },
  16.             aggregations: {
  17.               min: {
  18.                 min: {
  19.                   field: 'kubernetes.container.memory.usage.bytes'
  20.                 }
  21.               },
  22.               max: {
  23.                 max: {
  24.                   field: 'kubernetes.container.memory.usage.bytes'
  25.                 }
  26.               },
  27.               avg: {
  28.                 avg: {
  29.                   field: 'kubernetes.container.memory.usage.bytes'
  30.                 }
  31.               }
  32.             }
  33.           }
  34.         }
  35.       }
  36.     }
  37.   }
  38. )
  39. puts response
  1. const response = await client.search({
  2.   index: "my-data-stream",
  3.   size: 0,
  4.   aggs: {
  5.     tsid: {
  6.       terms: {
  7.         field: "_tsid",
  8.       },
  9.       aggs: {
  10.         over_time: {
  11.           date_histogram: {
  12.             field: "@timestamp",
  13.             fixed_interval: "1d",
  14.           },
  15.           aggs: {
  16.             min: {
  17.               min: {
  18.                 field: "kubernetes.container.memory.usage.bytes",
  19.               },
  20.             },
  21.             max: {
  22.               max: {
  23.                 field: "kubernetes.container.memory.usage.bytes",
  24.               },
  25.             },
  26.             avg: {
  27.               avg: {
  28.                 field: "kubernetes.container.memory.usage.bytes",
  29.               },
  30.             },
  31.           },
  32.         },
  33.       },
  34.     },
  35.   },
  36. });
  37. console.log(response);
  1. GET /my-data-stream/_search
  2. {
  3.     "size": 0,
  4.     "aggs": {
  5.         "tsid": {
  6.             "terms": {
  7.                 "field": "_tsid"
  8.             },
  9.             "aggs": {
  10.                 "over_time": {
  11.                     "date_histogram": {
  12.                         "field": "@timestamp",
  13.                         "fixed_interval": "1d"
  14.                     },
  15.                     "aggs": {
  16.                         "min": {
  17.                             "min": {
  18.                                 "field": "kubernetes.container.memory.usage.bytes"
  19.                             }
  20.                         },
  21.                         "max": {
  22.                             "max": {
  23.                                 "field": "kubernetes.container.memory.usage.bytes"
  24.                             }
  25.                         },
  26.                         "avg": {
  27.                             "avg": {
  28.                                 "field": "kubernetes.container.memory.usage.bytes"
  29.                             }
  30.                         }
  31.                     }
  32.                 }
  33.             }
  34.         }
  35.     }
  36. }

Downsample the TSDS

A TSDS can’t be downsampled directly. You need to downsample its backing indices instead. You can see the backing index for your data stream by running:

  1. resp = client.indices.get_data_stream(
  2.     name="my-data-stream",
  3. )
  4. print(resp)
  1. response = client.indices.get_data_stream(
  2.   name: 'my-data-stream'
  3. )
  4. puts response
  1. const response = await client.indices.getDataStream({
  2.   name: "my-data-stream",
  3. });
  4. console.log(response);
  1. GET /_data_stream/my-data-stream

This returns:

  1. {
  2.   "data_streams": [
  3.     {
  4.       "name": "my-data-stream",
  5.       "timestamp_field": {
  6.         "name": "@timestamp"
  7.       },
  8.       "indices": [
  9.         {
  10.           "index_name": ".ds-my-data-stream-2023.07.26-000001", 
  11.           "index_uuid": "ltOJGmqgTVm4T-Buoe7Acg",
  12.           "prefer_ilm": true,
  13.           "managed_by": "Unmanaged"
  14.         }
  15.       ],
  16.       "generation": 1,
  17.       "status": "GREEN",
  18.       "next_generation_managed_by": "Unmanaged",
  19.       "prefer_ilm": true,
  20.       "template": "my-data-stream-template",
  21.       "hidden": false,
  22.       "system": false,
  23.       "allow_custom_routing": false,
  24.       "replicated": false,
  25.       "rollover_on_write": false,
  26.       "time_series": {
  27.         "temporal_ranges": [
  28.           {
  29.             "start": "2023-07-26T09:26:42.000Z",
  30.             "end": "2023-07-26T13:26:42.000Z"
  31.           }
  32.         ]
  33.       }
  34.     }
  35.   ]
  36. }

The backing index for this data stream.

Before a backing index can be downsampled, the TSDS needs to be rolled over and the old index needs to be made read-only.

Roll over the TSDS using the rollover API:

  1. resp = client.indices.rollover(
  2.     alias="my-data-stream",
  3. )
  4. print(resp)
  1. response = client.indices.rollover(
  2.   alias: 'my-data-stream'
  3. )
  4. puts response
  1. const response = await client.indices.rollover({
  2.   alias: "my-data-stream",
  3. });
  4. console.log(response);
  1. POST /my-data-stream/_rollover/

Copy the name of the old_index from the response. In the following steps, replace the index name with that of your old_index.

The old index needs to be set to read-only mode. Run the following request:

  1. resp = client.indices.add_block(
  2.     index=".ds-my-data-stream-2023.07.26-000001",
  3.     block="write",
  4. )
  5. print(resp)
  1. response = client.indices.add_block(
  2.   index: '.ds-my-data-stream-2023.07.26-000001',
  3.   block: 'write'
  4. )
  5. puts response
  1. const response = await client.indices.addBlock({
  2.   index: ".ds-my-data-stream-2023.07.26-000001",
  3.   block: "write",
  4. });
  5. console.log(response);
  1. PUT /.ds-my-data-stream-2023.07.26-000001/_block/write

Next, use the downsample API to downsample the index, setting the time series interval to one hour:

  1. resp = client.indices.downsample(
  2.     index=".ds-my-data-stream-2023.07.26-000001",
  3.     target_index=".ds-my-data-stream-2023.07.26-000001-downsample",
  4.     config={
  5.         "fixed_interval": "1h"
  6.     },
  7. )
  8. print(resp)
  1. response = client.indices.downsample(
  2.   index: '.ds-my-data-stream-2023.07.26-000001',
  3.   target_index: '.ds-my-data-stream-2023.07.26-000001-downsample',
  4.   body: {
  5.     fixed_interval: '1h'
  6.   }
  7. )
  8. puts response
  1. const response = await client.indices.downsample({
  2.   index: ".ds-my-data-stream-2023.07.26-000001",
  3.   target_index: ".ds-my-data-stream-2023.07.26-000001-downsample",
  4.   config: {
  5.     fixed_interval: "1h",
  6.   },
  7. });
  8. console.log(response);
  1. POST /.ds-my-data-stream-2023.07.26-000001/_downsample/.ds-my-data-stream-2023.07.26-000001-downsample
  2. {
  3.   "fixed_interval": "1h"
  4. }

Now you can modify the data stream, and replace the original index with the downsampled one:

  1. resp = client.indices.modify_data_stream(
  2.     actions=[
  3.         {
  4.             "remove_backing_index": {
  5.                 "data_stream": "my-data-stream",
  6.                 "index": ".ds-my-data-stream-2023.07.26-000001"
  7.             }
  8.         },
  9.         {
  10.             "add_backing_index": {
  11.                 "data_stream": "my-data-stream",
  12.                 "index": ".ds-my-data-stream-2023.07.26-000001-downsample"
  13.             }
  14.         }
  15.     ],
  16. )
  17. print(resp)
  1. const response = await client.indices.modifyDataStream({
  2.   actions: [
  3.     {
  4.       remove_backing_index: {
  5.         data_stream: "my-data-stream",
  6.         index: ".ds-my-data-stream-2023.07.26-000001",
  7.       },
  8.     },
  9.     {
  10.       add_backing_index: {
  11.         data_stream: "my-data-stream",
  12.         index: ".ds-my-data-stream-2023.07.26-000001-downsample",
  13.       },
  14.     },
  15.   ],
  16. });
  17. console.log(response);
  1. POST _data_stream/_modify
  2. {
  3.   "actions": [
  4.     {
  5.       "remove_backing_index": {
  6.         "data_stream": "my-data-stream",
  7.         "index": ".ds-my-data-stream-2023.07.26-000001"
  8.       }
  9.     },
  10.     {
  11.       "add_backing_index": {
  12.         "data_stream": "my-data-stream",
  13.         "index": ".ds-my-data-stream-2023.07.26-000001-downsample"
  14.       }
  15.     }
  16.   ]
  17. }

You can now delete the old backing index. But be aware this will delete the original data. Don’t delete the index if you may need the original data in the future.

View the results

Re-run the earlier search query (note that when querying downsampled indices there are a few nuances to be aware of):

  1. resp = client.search(
  2.     index="my-data-stream",
  3. )
  4. print(resp)
  1. response = client.search(
  2.   index: 'my-data-stream'
  3. )
  4. puts response
  1. const response = await client.search({
  2.   index: "my-data-stream",
  3. });
  4. console.log(response);
  1. GET /my-data-stream/_search

The TSDS with the new downsampled backing index contains just one document. For counters, this document would only have the last value. For gauges, the field type is now aggregate_metric_double. You see the min, max, sum, and value_count statistics based off of the original sampled metrics:

  1. {
  2.   "took": 2,
  3.   "timed_out": false,
  4.   "_shards": {
  5.     "total": 4,
  6.     "successful": 4,
  7.     "skipped": 0,
  8.     "failed": 0
  9.   },
  10.   "hits": {
  11.     "total": {
  12.       "value": 1,
  13.       "relation": "eq"
  14.     },
  15.     "max_score": 1,
  16.     "hits": [
  17.       {
  18.         "_index": ".ds-my-data-stream-2023.07.26-000001-downsample",
  19.         "_id": "0eL0wC_4-45SnTNFAAABiZHbD4A",
  20.         "_score": 1,
  21.         "_source": {
  22.           "@timestamp": "2023-07-26T11:00:00.000Z",
  23.           "_doc_count": 10,
  24.           "ingest_time": "2023-07-26T11:26:42.715Z",
  25.           "kubernetes": {
  26.             "container": {
  27.               "cpu": {
  28.                 "usage": {
  29.                   "core": {
  30.                     "ns": 12828317850
  31.                   },
  32.                   "limit": {
  33.                     "pct": 0.0000277905
  34.                   },
  35.                   "nanocores": {
  36.                     "min": 38907,
  37.                     "max": 153404,
  38.                     "sum": 992677,
  39.                     "value_count": 10
  40.                   },
  41.                   "node": {
  42.                     "pct": 0.0000277905
  43.                   }
  44.                 }
  45.               },
  46.               "memory": {
  47.                 "available": {
  48.                   "bytes": {
  49.                     "min": 279586406,
  50.                     "max": 1062428344,
  51.                     "sum": 7101494721,
  52.                     "value_count": 10
  53.                   }
  54.                 },
  55.                 "majorpagefaults": 0,
  56.                 "pagefaults": {
  57.                   "min": 74843,
  58.                   "max": 302252,
  59.                   "sum": 2061071,
  60.                   "value_count": 10
  61.                 },
  62.                 "rss": {
  63.                   "bytes": {
  64.                     "min": 91914,
  65.                     "max": 402801,
  66.                     "sum": 2389770,
  67.                     "value_count": 10
  68.                   }
  69.                 },
  70.                 "usage": {
  71.                   "bytes": {
  72.                     "min": 100475044,
  73.                     "max": 379572388,
  74.                     "sum": 2668170609,
  75.                     "value_count": 10
  76.                   },
  77.                   "limit": {
  78.                     "pct": 0.00009923134
  79.                   },
  80.                   "node": {
  81.                     "pct": 0.017700378
  82.                   }
  83.                 },
  84.                 "workingset": {
  85.                   "bytes": {
  86.                     "min": 431227,
  87.                     "max": 2294743,
  88.                     "sum": 14230488,
  89.                     "value_count": 10
  90.                   }
  91.                 }
  92.               },
  93.               "name": "container-name-44",
  94.               "start_time": "2021-03-30T07:59:06.000Z"
  95.             },
  96.             "host": "gke-apps-0",
  97.             "namespace": "namespace26",
  98.             "node": "gke-apps-0-0",
  99.             "pod": "gke-apps-0-0-0"
  100.           }
  101.         }
  102.       }
  103.     ]
  104.   }
  105. }

Re-run the earlier aggregation. Even though the aggregation runs on the downsampled TSDS that only contains 1 document, it returns the same results as the earlier aggregation on the original TSDS.

  1. resp = client.search(
  2.     index="my-data-stream",
  3.     size=0,
  4.     aggs={
  5.         "tsid": {
  6.             "terms": {
  7.                 "field": "_tsid"
  8.             },
  9.             "aggs": {
  10.                 "over_time": {
  11.                     "date_histogram": {
  12.                         "field": "@timestamp",
  13.                         "fixed_interval": "1d"
  14.                     },
  15.                     "aggs": {
  16.                         "min": {
  17.                             "min": {
  18.                                 "field": "kubernetes.container.memory.usage.bytes"
  19.                             }
  20.                         },
  21.                         "max": {
  22.                             "max": {
  23.                                 "field": "kubernetes.container.memory.usage.bytes"
  24.                             }
  25.                         },
  26.                         "avg": {
  27.                             "avg": {
  28.                                 "field": "kubernetes.container.memory.usage.bytes"
  29.                             }
  30.                         }
  31.                     }
  32.                 }
  33.             }
  34.         }
  35.     },
  36. )
  37. print(resp)
  1. response = client.search(
  2.   index: 'my-data-stream',
  3.   body: {
  4.     size: 0,
  5.     aggregations: {
  6.       tsid: {
  7.         terms: {
  8.           field: '_tsid'
  9.         },
  10.         aggregations: {
  11.           over_time: {
  12.             date_histogram: {
  13.               field: '@timestamp',
  14.               fixed_interval: '1d'
  15.             },
  16.             aggregations: {
  17.               min: {
  18.                 min: {
  19.                   field: 'kubernetes.container.memory.usage.bytes'
  20.                 }
  21.               },
  22.               max: {
  23.                 max: {
  24.                   field: 'kubernetes.container.memory.usage.bytes'
  25.                 }
  26.               },
  27.               avg: {
  28.                 avg: {
  29.                   field: 'kubernetes.container.memory.usage.bytes'
  30.                 }
  31.               }
  32.             }
  33.           }
  34.         }
  35.       }
  36.     }
  37.   }
  38. )
  39. puts response
  1. const response = await client.search({
  2.   index: "my-data-stream",
  3.   size: 0,
  4.   aggs: {
  5.     tsid: {
  6.       terms: {
  7.         field: "_tsid",
  8.       },
  9.       aggs: {
  10.         over_time: {
  11.           date_histogram: {
  12.             field: "@timestamp",
  13.             fixed_interval: "1d",
  14.           },
  15.           aggs: {
  16.             min: {
  17.               min: {
  18.                 field: "kubernetes.container.memory.usage.bytes",
  19.               },
  20.             },
  21.             max: {
  22.               max: {
  23.                 field: "kubernetes.container.memory.usage.bytes",
  24.               },
  25.             },
  26.             avg: {
  27.               avg: {
  28.                 field: "kubernetes.container.memory.usage.bytes",
  29.               },
  30.             },
  31.           },
  32.         },
  33.       },
  34.     },
  35.   },
  36. });
  37. console.log(response);
  1. GET /my-data-stream/_search
  2. {
  3.     "size": 0,
  4.     "aggs": {
  5.         "tsid": {
  6.             "terms": {
  7.                 "field": "_tsid"
  8.             },
  9.             "aggs": {
  10.                 "over_time": {
  11.                     "date_histogram": {
  12.                         "field": "@timestamp",
  13.                         "fixed_interval": "1d"
  14.                     },
  15.                     "aggs": {
  16.                         "min": {
  17.                             "min": {
  18.                                 "field": "kubernetes.container.memory.usage.bytes"
  19.                             }
  20.                         },
  21.                         "max": {
  22.                             "max": {
  23.                                 "field": "kubernetes.container.memory.usage.bytes"
  24.                             }
  25.                         },
  26.                         "avg": {
  27.                             "avg": {
  28.                                 "field": "kubernetes.container.memory.usage.bytes"
  29.                             }
  30.                         }
  31.                     }
  32.                 }
  33.             }
  34.         }
  35.     }
  36. }

This example demonstrates how downsampling can dramatically reduce the number of documents stored for time series data, within whatever time boundaries you choose. It’s also possible to perform downsampling on already downsampled data, to further reduce storage and associated costs, as the time series data ages and the data resolution becomes less critical.

The recommended way to downsample a TSDS is with ILM. To learn more, try the Run downsampling with ILM example.