Snapshot Management API

Use the snapshot management (SM) API to automate taking snapshots.

Create or update a policy

Introduced 2.1

Creates or updates an SM policy.

Request

Create:

  1. POST _plugins/_sm/policies/<policy_name>

Update:

  1. PUT _plugins/_sm/policies/<policy_name>?if_seq_no=0&if_primary_term=1

You must provide the seq_no and primary_term parameters for an update request.

Example

  1. POST _plugins/_sm/policies/daily-policy
  2. {
  3.   "description": "Daily snapshot policy",
  4.   "creation": {
  5.     "schedule": {
  6.       "cron": {
  7.         "expression": "0 8 * * *",
  8.         "timezone": "UTC"
  9.       }
  10.     },
  11.     "time_limit": "1h"
  12.   },
  13.   "deletion": {
  14.     "schedule": {
  15.       "cron": {
  16.         "expression": "0 1 * * *",
  17.         "timezone": "America/Los_Angeles"
  18.       }
  19.     },
  20.     "condition": {
  21.       "max_age": "7d",
  22.       "max_count": 21,
  23.       "min_count": 7
  24.     },
  25.     "time_limit": "1h"
  26.   },
  27.   "snapshot_config": {
  28.     "date_format": "yyyy-MM-dd-HH:mm",
  29.     "timezone": "America/Los_Angeles",
  30.     "indices": "*",
  31.     "repository": "s3-repo",
  32.     "ignore_unavailable": "true",
  33.     "include_global_state": "false",
  34.     "partial": "true",
  35.     "metadata": {
  36.       "any_key": "any_value"
  37.     }
  38.   },
  39.   "notification": {
  40.     "channel": {
  41.       "id": "NC3OpoEBzEoHMX183R3f"
  42.     },
  43.     "conditions": {
  44.       "creation": true,
  45.       "deletion": false,
  46.       "failure": false,
  47.       "time_limit_exceeded": false
  48.     }
  49.   }
  50. }

Response

  1. {
  2.   "_id" : "daily-policy-sm-policy",
  3.   "_version" : 5,
  4.   "_seq_no" : 54983,
  5.   "_primary_term" : 21,
  6.   "sm_policy" : {
  7.     "name" : "daily-policy",
  8.     "description" : "Daily snapshot policy",
  9.     "schema_version" : 15,
  10.     "creation" : {
  11.       "schedule" : {
  12.         "cron" : {
  13.           "expression" : "0 8 * * *",
  14.           "timezone" : "UTC"
  15.         }
  16.       },
  17.       "time_limit" : "1h"
  18.     },
  19.     "deletion" : {
  20.       "schedule" : {
  21.         "cron" : {
  22.           "expression" : "0 1 * * *",
  23.           "timezone" : "America/Los_Angeles"
  24.         }
  25.       },
  26.       "condition" : {
  27.         "max_age" : "7d",
  28.         "min_count" : 7,
  29.         "max_count" : 21
  30.       },
  31.       "time_limit" : "1h"
  32.     },
  33.     "snapshot_config" : {
  34.       "indices" : "*",
  35.       "metadata" : {
  36.         "any_key" : "any_value"
  37.       },
  38.       "ignore_unavailable" : "true",
  39.       "timezone" : "America/Los_Angeles",
  40.       "include_global_state" : "false",
  41.       "date_format" : "yyyy-MM-dd-HH:mm",
  42.       "repository" : "s3-repo",
  43.       "partial" : "true"
  44.     },
  45.     "schedule" : {
  46.       "interval" : {
  47.         "start_time" : 1656425122909,
  48.         "period" : 1,
  49.         "unit" : "Minutes"
  50.       }
  51.     },
  52.     "enabled" : true,
  53.     "last_updated_time" : 1656425122909,
  54.     "enabled_time" : 1656425122909,
  55.     "notification" : {
  56.       "channel" : {
  57.         "id" : "NC3OpoEBzEoHMX183R3f"
  58.       },
  59.       "conditions" : {
  60.         "creation" : true,
  61.         "deletion" : false,
  62.         "failure" : false,
  63.         "time_limit_exceeded" : false
  64.       }
  65.     }
  66.   }
  67. }

Parameters

You can specify the following parameters to create/update an SM policy.

ParameterTypeDescription
descriptionStringThe description of the SM policy. Optional.
enabledBooleanShould this SM policy be enabled at creation? Optional.
snapshot_configObjectThe configuration options for snapshot creation. Required.
snapshot_config.date_formatStringSnapshot names have the format <policy_name>-<date>-<random number>. date_format specifies the format for the date in the snapshot name. Supports all date formats supported by OpenSearch. Optional. Default is “yyyy-MM-dd’T’HH:mm:ss”.
snapshot_config.date_format_timezoneStringSnapshot names have the format <policy_name>-<date>-<random number>. date_format_timezone specifies the time zone for the date in the snapshot name. Optional. Default is UTC.
snapshot_config.indicesStringThe names of the indexes in the snapshot. Multiple index names are separated by ,. Supports wildcards (). Optional. Default is (all indexes).
snapshot_config.repositoryStringThe repository in which to store snapshots. Required.
snapshot_config.ignore_unavailableBooleanDo you want to ignore unavailable indexes? Optional. Default is false.
snapshot_config.include_global_stateBooleanDo you want to include cluster state? Optional. Default is true because of Security plugin considerations.
snapshot_config.partialBooleanDo you want to allow partial snapshots? Optional. Default is false.
snapshot_config.metadataObjectMetadata in the form of key/value pairs. Optional.
creationObjectConfiguration for snapshot creation. Required.
creation.scheduleStringThe cron schedule used to create snapshots. Required.
creation.time_limitStringSets the maximum time to wait for snapshot creation to finish. If time_limit is longer than the scheduled time interval for taking snapshots, no scheduled snapshots are taken until time_limit elapses. For example, if time_limit is set to 35 minutes and snapshots are taken every 30 minutes starting at midnight, the snapshots at 00:00 and 01:00 are taken, but the snapshot at 00:30 is skipped. Optional.
deletionObjectConfiguration for snapshot deletion. Optional. Default is to retain all snapshots.
deletion.scheduleStringThe cron schedule used to delete snapshots. Optional. Default is to use creation.schedule, which is required.
deletion.time_limitStringSets the maximum time to wait for snapshot deletion to finish. Optional.
deletion.delete_conditionObjectConditions for snapshot deletion. Optional.
deletion.delete_condition.max_countIntegerThe maximum number of snapshots to be retained. Optional.
deletion.delete_condition.max_ageStringThe maximum time a snapshot is retained. Optional.
deletion.delete_condition.min_countIntegerThe minimum number of snapshots to be retained. Optional. Default is 1.
notificationObjectDefines notifications for SM events. Optional.
notification.channelObjectDefines a channel for notifications. You must create and configure a notification channel before setting up SM notifications. Required.
notification.channel.idStringThe channel ID of the channel used for notifications. To get the channel IDs of all created channels, use GET _plugins/_notifications/configs. Required.
notification.conditionsObjectSM events you want to be notified about. Set the ones you are interested in to true.
notification.conditions.creationBooleanDo you want notifications about snapshot creation? Optional. Default is true.
notification.conditions.deletionBooleanDo you want notifications about snapshot deletion? Optional. Default is false.
notification.conditions.failureBooleanDo you want notifications about creation or deletion failure? Optional. Default is false.
notification.conditions.time_limit_exceededBooleanDo you want notifications when snapshot operations take longer than time_limit? Optional. Default is false.

Get policies

Introduced 2.1

Gets SM policies.

Request

Get all SM policies:

  1. GET _plugins/_sm/policies

You can use a query string and specify pagination, the field to be sorted by, and sort order:

  1. GET _plugins/_sm/policies?from=0&size=20&sortField=sm_policy.name&sortOrder=desc&queryString=*

Get a specific SM policy:

  1. GET _plugins/_sm/policies/<policy_name>

Example

  1. GET _plugins/_sm/policies/daily-policy

Response

  1. {
  2.   "_id" : "daily-policy-sm-policy",
  3.   "_version" : 6,
  4.   "_seq_no" : 44696,
  5.   "_primary_term" : 19,
  6.   "sm_policy" : {
  7.     "name" : "daily-policy",
  8.     "description" : "Daily snapshot policy",
  9.     "schema_version" : 15,
  10.     "creation" : {
  11.       "schedule" : {
  12.         "cron" : {
  13.           "expression" : "0 8 * * *",
  14.           "timezone" : "UTC"
  15.         }
  16.       },
  17.       "time_limit" : "1h"
  18.     },
  19.     "deletion" : {
  20.       "schedule" : {
  21.         "cron" : {
  22.           "expression" : "0 1 * * *",
  23.           "timezone" : "America/Los_Angeles"
  24.         }
  25.       },
  26.       "condition" : {
  27.         "max_age" : "7d",
  28.         "min_count" : 7,
  29.         "max_count" : 21
  30.       },
  31.       "time_limit" : "1h"
  32.     },
  33.     "snapshot_config" : {
  34.       "metadata" : {
  35.         "any_key" : "any_value"
  36.       },
  37.       "ignore_unavailable" : "true",
  38.       "include_global_state" : "false",
  39.       "date_format" : "yyyy-MM-dd-HH:mm",
  40.       "repository" : "s3-repo",
  41.       "partial" : "true"
  42.     },
  43.     "schedule" : {
  44.       "interval" : {
  45.         "start_time" : 1656341042874,
  46.         "period" : 1,
  47.         "unit" : "Minutes"
  48.       }
  49.     },
  50.     "enabled" : true,
  51.     "last_updated_time" : 1656341042874,
  52.     "enabled_time" : 1656341042874
  53.   }
  54. }

Explain

Introduced 2.1

Provides the enabled/disabled status and the metadata for all policies specified. Multiple policy names are separated with ,. You can also specify desired policies with a wildcard pattern.

SM State Machine

SM uses a state machine for snapshot creation and deletion. The image on the left shows one execution period of the creation workflow, from the CREATION_START state to the CREATION_FINISHED state. Deletion workflow follows the same pattern as creation workflow.

The creation workflow starts in the CREATION_START state and continuously checks if the conditions in the creation cron schedule are met. After the conditions are met, the creation workflow switches to the CREATION_CONDITION_MET state and continues to the CREATING state. The CREATING state calls the create snapshot API asynchronously and then waits for snapshot creation to end in the CREATION_FINISHED state. Once snapshot creation ends, the creation workflow goes back to the CREATION_START state, and the cycle continues. The current_state field of metadata.creation and metadata.deletion returns the current state of the state machine.

Request

  1. GET _plugins/_sm/policies/<policy_names>/_explain

Example

  1. GET _plugins/_sm/policies/daily*/_explain

Response

  1. {
  2.   "policies" : [
  3.     {
  4.       "name" : "daily-policy",
  5.       "creation" : {
  6.         "current_state" : "CREATION_START",
  7.         "trigger" : {
  8.           "time" : 1656403200000
  9.         }
  10.       },
  11.       "deletion" : {
  12.         "current_state" : "DELETION_START",
  13.         "trigger" : {
  14.           "time" : 1656403200000
  15.         }
  16.       },
  17.       "policy_seq_no" : 44696,
  18.       "policy_primary_term" : 19,
  19.       "enabled" : true
  20.     }
  21.   ]
  22. }

The following table lists all fields for each policy in the response.

FieldDescription
nameThe name of the SM policy.
creationInformation about the latest creation operation. See subfields below.
deletionInformation about the latest deletion operation. See subfields below.
policy_seq_no
policy_primary_term		The version of the SM policy.
enabledIs the policy running?

The following table lists all fields in the creation and deletion objects of each policy.

FieldDescription
current_stateThe current state of the state machine that runs snapshot creation/deletion as described above.
trigger.timeThe next creation/deletion execution time in milliseconds since the epoch.
latest_executionDescribes the latest creation/deletion execution.
latest_execution.statusThe execution status of the latest creation/deletion. Possible values are:
IN_PROGRESS: Snapshot creation/deletion has started.
SUCCESS: Snapshot creation/deletion has finished successfully.
RETRYING: The creation/deletion attempt has failed. It will be retried three times.
FAILED: The creation/deletion attempt failed after three retries. End the current execution period and go to the next execution period.
TIME_LIMIT_EXCEEDED: The creation/deletion time exceeded the time_limit set in the policy. End the current execution period and go to the next execution period.
latest_execution.start_timeThe start time of the latest execution in milliseconds since the epoch.
latest_execution.end_timeThe end time of the latest execution in milliseconds since the epoch.
latest_execution.info.messageA user-friendly message describing the status of the latest execution.
latest_execution.info.causeContains the failure reason if the latest execution fails.
retry.countThe number of remaining execution retry attempts.

Start a policy

Introduced 2.1

Starts the policy by setting its enabled flag to true.

Request

  1. POST  _plugins/_sm/policies/<policy_name>/_start

Example

  1. POST  _plugins/_sm/policies/daily-policy/_start

Response

  1. {
  2.   "acknowledged" : true
  3. }

Stop a policy

Introduced 2.1

Sets the enabled flag to false for an SM policy. The policy will not run until you start it.

Request

  1. POST  _plugins/_sm/policies/<policy_name>/_stop

Example

  1. POST  _plugins/_sm/policies/daily-policy/_stop

Response

  1. {
  2.   "acknowledged" : true
  3. }

Delete a policy

Introduced 2.1

Deletes the specified SM policy.

Request

  1. DELETE  _plugins/_sm/policies/<policy_name>

Example

  1. DELETE _plugins/_sm/policies/daily-policy

Response

  1. {
  2.   "_index" : ".opendistro-ism-config",
  3.   "_id" : "daily-policy-sm-policy",
  4.   "_version" : 8,
  5.   "result" : "deleted",
  6.   "forced_refresh" : true,
  7.   "_shards" : {
  8.     "total" : 2,
  9.     "successful" : 2,
  10.     "failed" : 0
  11.   },
  12.   "_seq_no" : 45366,
  13.   "_primary_term" : 20
  14. }