Explain data stream lifecycle
- Explain data stream lifecycle

Explain data stream lifecycle

New API reference

For the most up-to-date API details, refer to Data stream APIs.

Retrieves the current data stream lifecycle status for one or more data stream backing indices.

Prerequisites

If the Elasticsearch security features are enabled, you must have at least the manage_data_stream_lifecycle index privilege or view_index_metadata index privilege to use this API. For more information, see Security privileges.

Request

GET <target>/_lifecycle/explain

Description

Retrieves information about the index or data stream’s current data stream lifecycle state, such as time since index creation, time since rollover, the lifecycle configuration managing the index, or any error that Elasticsearch might’ve encountered during the lifecycle execution.

Path parameters

<target>

(Required, string) Comma-separated list of indices or data streams.

Query parameters

include_defaults

(Optional, Boolean) Includes default configurations related to the lifecycle of the target. Defaults to false.

master_timeout

(Optional, time units) Period to wait for the master node. If the master node is not available before the timeout expires, the request fails and returns an error. Defaults to 30s. Can also be set to -1 to indicate that the request should never timeout.

Examples

If you want to retrieve the lifecycle state of all the backing indices of a data stream, you can use the data stream name. For simplicity, the following example retrieves the lifecycle state of one backing index .ds-metrics-2023.03.22-000001:

resp = client.indices.explain_data_lifecycle(
    index=".ds-metrics-2023.03.22-000001",
)
print(resp)

response = client.indices.explain_data_lifecycle(
  index: '.ds-metrics-2023.03.22-000001'
)
puts response

const response = await client.indices.explainDataLifecycle({
  index: ".ds-metrics-2023.03.22-000001",
});
console.log(response);

GET .ds-metrics-2023.03.22-000001/_lifecycle/explain

If the index is managed by a data stream lifecycle explain will show the managed_by_lifecycle field set to true and the rest of the response will contain information about the lifecycle execution status for this index:

{
  "indices": {
    ".ds-metrics-2023.03.22-000001": {
      "index" : ".ds-metrics-2023.03.22-000001",
      "managed_by_lifecycle" : true,                        
      "index_creation_date_millis" : 1679475563571,   
      "time_since_index_creation" : "843ms",          
      "rollover_date_millis" : 1679475564293,         
      "time_since_rollover" : "121ms",                
      "lifecycle" : { },                              
      "generation_time" : "121ms"                     
  }
}

	Shows if the index is being managed by data stream lifecycle. If the index is not managed by a data stream lifecycle the other fields will not be shown
	When the index was created, this timestamp is used to determine when to rollover
	The time since the index creation (used for calculating when to rollover the index via the `max_age`)
	When the index was rolled over. If the index was not rolled over this will not be shown.
	The time since rollover. If the index was not rolled over this will not be shown.
	The lifecycle configuration that applies to this index (which is configured on the parent data stream)
	The generation time of the index represents the time since the index started progressing towards the user configurable / business specific parts of the lifecycle (e.g. retention). The `generation_time` is calculated from the origination date if it exists, or from the rollover date if it exists, or from the creation date if neither of the other two exist. If the index is the write index the `generation_time` will not be reported because it is not eligible for retention or other parts of the lifecycle.

The explain will also report any errors related to the lifecycle execution for the target index:

{
  "indices": {
    ".ds-metrics-2023.03.22-000001": {
      "index" : ".ds-metrics-2023.03.22-000001",
      "managed_by_lifecycle" : true,
      "index_creation_date_millis" : 1679475563571,
      "time_since_index_creation" : "843ms",
      "lifecycle" : {
        "enabled": true
      },
      "error": "{\"type\":\"validation_exception\",\"reason\":\"Validation Failed: 1: this action would add [2] shards, but this cluster
currently has [4]/[3] maximum normal shards open;\"}"        
  }
}

The target index could not be rolled over due to a limitation in the number of shards allowed in the cluster.

Explain Data Stream Lifecycle

Explain data stream lifecycle