Index recovery API

Index recovery API

New API reference

For the most up-to-date API details, refer to Index APIs.

Returns information about ongoing and completed shard recoveries for one or more indices. For data streams, the API returns information for the stream’s backing indices.

  1. resp = client.indices.recovery(
  2.     index="my-index-000001",
  3. )
  4. print(resp)
  1. response = client.indices.recovery(
  2.   index: 'my-index-000001'
  3. )
  4. puts response
  1. const response = await client.indices.recovery({
  2.   index: "my-index-000001",
  3. });
  4. console.log(response);
  1. GET /my-index-000001/_recovery

Request

GET /<target>/_recovery

GET /_recovery

Prerequisites

  • If the Elasticsearch security features are enabled, you must have the monitor or manage index privilege for the target data stream, index, or alias.

Description

Use the index recovery API to get information about ongoing and completed shard recoveries.

Shard recovery is the process of initializing a shard copy, such as restoring a primary shard from a snapshot or creating a replica shard from a primary shard. When a shard recovery completes, the recovered shard is available for search and indexing.

Recovery automatically occurs during the following processes:

  • When creating an index for the first time.
  • When a node rejoins the cluster and starts up any missing primary shard copies using the data that it holds in its data path.
  • Creation of new replica shard copies from the primary.
  • Relocation of a shard copy to a different node in the same cluster.
  • A snapshot restore operation.
  • A clone, shrink, or split operation.

You can determine the cause of a shard recovery using the recovery or cat recovery APIs.

The index recovery API reports information about completed recoveries only for shard copies that currently exist in the cluster. It only reports the last recovery for each shard copy and does not report historical information about earlier recoveries, nor does it report information about the recoveries of shard copies that no longer exist. This means that if a shard copy completes a recovery and then Elasticsearch relocates it onto a different node then the information about the original recovery will not be shown in the recovery API.

Path parameters

<target>

(Optional, string) Comma-separated list of data streams, indices, and aliases used to limit the request. Supports wildcards (*). To target all data streams and indices, omit this parameter or use * or _all.

Query parameters

active_only

(Optional, Boolean) If true, the response only includes ongoing shard recoveries. Defaults to false.

detailed

(Optional, Boolean) If true, the response includes detailed information about shard recoveries. Defaults to false.

index

(Optional, string) Comma-separated list or wildcard expression of index names used to limit the request.

Response body

id

(Integer) ID of the shard.

type

(String) Recovery source for the shard. Returned values include:

  • EMPTY_STORE

    An empty store. Indicates a new primary shard or the forced allocation of an empty primary shard using the cluster reroute API.

    EXISTING_STORE

    The store of an existing primary shard. Indicates recovery is related to node startup or the allocation of an existing primary shard.

    LOCAL_SHARDS

    Shards of another index on the same node. Indicates recovery is related to a clone, shrink, or split operation.

    PEER

    A primary shard on another node. Indicates recovery is related to shard replication.

    SNAPSHOT

    A snapshot. Indicates recovery is related to a snapshot restore operation.

STAGE

(String) Recovery stage. Returned values can include:

  • INIT

    Recovery has not started.

    INDEX

    Reading index metadata and copying bytes from source to destination.

    VERIFY_INDEX

    Verifying the integrity of the index.

    TRANSLOG

    Replaying transaction log.

    FINALIZE

    Cleanup.

    DONE

    Complete.

primary

(Boolean) If true, the shard is a primary shard.

start_time

(String) Timestamp of recovery start.

stop_time

(String) Timestamp of recovery finish.

total_time_in_millis

(String) Total time to recover shard in milliseconds.

source

(Object) Recovery source. This can include:

  • A repository description if recovery is from a snapshot
  • A description of source node

target

(Object) Destination node.

index

(Object) Statistics about physical index recovery.

translog

(Object) Statistics about translog recovery.

start

(Object) Statistics about time to open and start the index.

Examples

Get recovery information for several data streams and indices

  1. resp = client.indices.recovery(
  2.     index="index1,index2",
  3.     human=True,
  4. )
  5. print(resp)
  1. response = client.indices.recovery(
  2.   index: 'index1,index2',
  3.   human: true
  4. )
  5. puts response
  1. const response = await client.indices.recovery({
  2.   index: "index1,index2",
  3.   human: "true",
  4. });
  5. console.log(response);
  1. GET index1,index2/_recovery?human

Get segment information for all data streams and indices in a cluster

  1. resp = client.indices.recovery(
  2.     human=True,
  3. )
  4. print(resp)
  1. response = client.indices.recovery(
  2.   human: true
  3. )
  4. puts response
  1. const response = await client.indices.recovery({
  2.   human: "true",
  3. });
  4. console.log(response);
  1. GET /_recovery?human

The API returns the following response:

  1. {
  2.   "index1" : {
  3.     "shards" : [ {
  4.       "id" : 0,
  5.       "type" : "SNAPSHOT",
  6.       "stage" : "INDEX",
  7.       "primary" : true,
  8.       "start_time" : "2014-02-24T12:15:59.716",
  9.       "start_time_in_millis": 1393244159716,
  10.       "stop_time" : "0s",
  11.       "stop_time_in_millis" : 0,
  12.       "total_time" : "2.9m",
  13.       "total_time_in_millis" : 175576,
  14.       "source" : {
  15.         "repository" : "my_repository",
  16.         "snapshot" : "my_snapshot",
  17.         "index" : "index1",
  18.         "version" : "{version}",
  19.         "restoreUUID": "PDh1ZAOaRbiGIVtCvZOMww"
  20.       },
  21.       "target" : {
  22.         "id" : "ryqJ5lO5S4-lSFbGntkEkg",
  23.         "host" : "my.fqdn",
  24.         "transport_address" : "my.fqdn",
  25.         "ip" : "10.0.1.7",
  26.         "name" : "my_es_node"
  27.       },
  28.       "index" : {
  29.         "size" : {
  30.           "total" : "75.4mb",
  31.           "total_in_bytes" : 79063092,
  32.           "reused" : "0b",
  33.           "reused_in_bytes" : 0,
  34.           "recovered" : "65.7mb",
  35.           "recovered_in_bytes" : 68891939,
  36.           "recovered_from_snapshot" : "0b",
  37.           "recovered_from_snapshot_in_bytes" : 0,
  38.           "percent" : "87.1%"
  39.         },
  40.         "files" : {
  41.           "total" : 73,
  42.           "reused" : 0,
  43.           "recovered" : 69,
  44.           "percent" : "94.5%"
  45.         },
  46.         "total_time" : "0s",
  47.         "total_time_in_millis" : 0,
  48.         "source_throttle_time" : "0s",
  49.         "source_throttle_time_in_millis" : 0,
  50.         "target_throttle_time" : "0s",
  51.         "target_throttle_time_in_millis" : 0
  52.       },
  53.       "translog" : {
  54.         "recovered" : 0,
  55.         "total" : 0,
  56.         "percent" : "100.0%",
  57.         "total_on_start" : 0,
  58.         "total_time" : "0s",
  59.         "total_time_in_millis" : 0
  60.       },
  61.       "verify_index" : {
  62.         "check_index_time" : "0s",
  63.         "check_index_time_in_millis" : 0,
  64.         "total_time" : "0s",
  65.         "total_time_in_millis" : 0
  66.       }
  67.     } ]
  68.   }
  69. }

This response includes information about a single index recovering a single shard. The source of the recovery is a snapshot repository and the target of the recovery is the my_es_node node.

The response also includes the number and percentage of files and bytes recovered.

Get detailed recovery information

To get a list of physical files in recovery, set the detailed query parameter to true.

  1. resp = client.indices.recovery(
  2.     human=True,
  3.     detailed=True,
  4. )
  5. print(resp)
  1. response = client.indices.recovery(
  2.   human: true,
  3.   detailed: true
  4. )
  5. puts response
  1. const response = await client.indices.recovery({
  2.   human: "true",
  3.   detailed: "true",
  4. });
  5. console.log(response);
  1. GET _recovery?human&detailed=true

The API returns the following response:

  1. {
  2.   "index1" : {
  3.     "shards" : [ {
  4.       "id" : 0,
  5.       "type" : "EXISTING_STORE",
  6.       "stage" : "DONE",
  7.       "primary" : true,
  8.       "start_time" : "2014-02-24T12:38:06.349",
  9.       "start_time_in_millis" : "1393245486349",
  10.       "stop_time" : "2014-02-24T12:38:08.464",
  11.       "stop_time_in_millis" : "1393245488464",
  12.       "total_time" : "2.1s",
  13.       "total_time_in_millis" : 2115,
  14.       "source" : {
  15.         "id" : "RGMdRc-yQWWKIBM4DGvwqQ",
  16.         "host" : "my.fqdn",
  17.         "transport_address" : "my.fqdn",
  18.         "ip" : "10.0.1.7",
  19.         "name" : "my_es_node"
  20.       },
  21.       "target" : {
  22.         "id" : "RGMdRc-yQWWKIBM4DGvwqQ",
  23.         "host" : "my.fqdn",
  24.         "transport_address" : "my.fqdn",
  25.         "ip" : "10.0.1.7",
  26.         "name" : "my_es_node"
  27.       },
  28.       "index" : {
  29.         "size" : {
  30.           "total" : "24.7mb",
  31.           "total_in_bytes" : 26001617,
  32.           "reused" : "24.7mb",
  33.           "reused_in_bytes" : 26001617,
  34.           "recovered" : "0b",
  35.           "recovered_in_bytes" : 0,
  36.           "recovered_from_snapshot" : "0b",
  37.           "recovered_from_snapshot_in_bytes" : 0,
  38.           "percent" : "100.0%"
  39.         },
  40.         "files" : {
  41.           "total" : 26,
  42.           "reused" : 26,
  43.           "recovered" : 0,
  44.           "percent" : "100.0%",
  45.           "details" : [ {
  46.             "name" : "segments.gen",
  47.             "length" : 20,
  48.             "recovered" : 20
  49.           }, {
  50.             "name" : "_0.cfs",
  51.             "length" : 135306,
  52.             "recovered" : 135306,
  53.             "recovered_from_snapshot": 0
  54.           }, {
  55.             "name" : "segments_2",
  56.             "length" : 251,
  57.             "recovered" : 251,
  58.             "recovered_from_snapshot": 0
  59.           }
  60.           ]
  61.         },
  62.         "total_time" : "2ms",
  63.         "total_time_in_millis" : 2,
  64.         "source_throttle_time" : "0s",
  65.         "source_throttle_time_in_millis" : 0,
  66.         "target_throttle_time" : "0s",
  67.         "target_throttle_time_in_millis" : 0
  68.       },
  69.       "translog" : {
  70.         "recovered" : 71,
  71.         "total" : 0,
  72.         "percent" : "100.0%",
  73.         "total_on_start" : 0,
  74.         "total_time" : "2.0s",
  75.         "total_time_in_millis" : 2025
  76.       },
  77.       "verify_index" : {
  78.         "check_index_time" : 0,
  79.         "check_index_time_in_millis" : 0,
  80.         "total_time" : "88ms",
  81.         "total_time_in_millis" : 88
  82.       }
  83.     } ]
  84.   }
  85. }

The response includes a listing of any physical files recovered and their sizes.

The response also includes timings in milliseconds of the various stages of recovery:

  • Index retrieval
  • Translog replay
  • Index start time

This response indicates the recovery is done. All recoveries, whether ongoing or complete, are kept in the cluster state and may be reported on at any time.

To only return information about ongoing recoveries, set the active_only query parameter to true.