Rollover index API

Creates a new index for a rollover target when the target’s existing index meets a condition you provide. A rollover target can be either an index alias or a data stream. When targeting an alias, the alias is updated to point to the new index. When targeting a data stream, the new index becomes the data stream’s write index and its generation is incremented.

  1. POST /alias1/_rollover/my-index-000002
  2. {
  3.   "conditions": {
  4.     "max_age":   "7d",
  5.     "max_docs":  1000,
  6.     "max_size": "5gb"
  7.   }
  8. }

Request

POST /<rollover-target>/_rollover/<target-index>

POST /<rollover-target>/_rollover/

Description

The rollover index API rolls a rollover target to a new index when the existing index meets a condition you provide. You can use this API to retire an index that becomes too large or too old.

To roll over an index, a condition must be met when you call the API. Elasticsearch does not monitor the index after you receive an API response. To automatically roll over indices when a condition is met, you can use Elasticsearch’s index lifecycle management (ILM) policies.

The rollover index API accepts a rollover target name and a list of conditions.

If the specified rollover target is an alias pointing to a single index, the rollover request:

  1. Creates a new index
  2. Adds the alias to the new index
  3. Removes the alias from the original index

If the specified rollover target is an alias pointing to multiple indices, one of these indices must have is_write_index set to true. In this case, the rollover request:

  1. Creates a new index
  2. Sets is_write_index to true for the new index
  3. Sets is_write_index to false for the original index

If the specified rollover target is a data stream, the rollover request:

  1. Creates a new index
  2. Adds the new index as a backing index and the write index on the data stream
  3. Increments the generation attribute of the data stream

Wait for active shards

Because the rollover operation creates a new index to rollover to, the wait_for_active_shards setting on index creation applies to the rollover action.

Path parameters

<rollover-target>

(Required*, string) Name of the existing index alias or data stream on which to to assign to the target index. perform the rollover.

<target-index>

(Optional*, string) Name of the target index to create and assign the index alias.

Index names must meet the following criteria:

  • Lowercase only
  • Cannot include \, /, *, ?, ", <, >, |, ` ` (space character), ,, #
  • Indices prior to 7.0 could contain a colon (:), but that’s been deprecated and won’t be supported in 7.0+
  • Cannot start with -, _, +
  • Cannot be . or ..
  • Cannot be longer than 255 bytes (note it is bytes, so multi-byte characters will count towards the 255 limit faster)
  • Names starting with . are deprecated, except for hidden indices and internal indices managed by plugins

*This parameter is not permitted if rollover-target is a data stream. In that case, the new index name will be in the form .ds-<rollover-target>-000001 where the zero-padded number of length 6 is the generation of the data stream.

If rollover-target is an alias that is assigned to an index name that ends with - and a number such as logs-000001. In this case, the name of the new index follows the same pattern and increments the number. For example, logs-000001 increments to logs-000002. This number is zero-padded with a length of 6, regardless of the prior index name.

If the existing index for the alias does not match this pattern, this parameter is required.

Query parameters

dry_run

(Optional, boolean) If true, the request checks whether the index matches provided conditions but does not perform a rollover. Defaults to false.

include_type_name

[7.0.0] Deprecated in 7.0.0. Mapping types have been deprecated. See Removal of mapping types. (Optional, boolean) If true, a mapping type is expected in the body of mappings. Defaults to false.

wait_for_active_shards

(Optional, string) The number of shard copies that must be active before proceeding with the operation. Set to all or any positive integer up to the total number of shards in the index (number_of_replicas+1). Default: 1, the primary shard.

See Active shards.

master_timeout

(Optional, time units) Specifies the period of time to wait for a connection to the master node. If no response is received before the timeout expires, the request fails and returns an error. Defaults to 30s.

timeout

(Optional, time units) Specifies the period of time to wait for a response. If no response is received before the timeout expires, the request fails and returns an error. Defaults to 30s.

Request body

aliases

(Optional, alias object) Index aliases which include the index. See Update index alias.

conditions

(Optional, object) If supplied, the set of conditions the rollover target’s existing index must meet to roll over. If omitted, the rollover will be performed unconditionally.

Parameters include:

  • max_age

    (Optional, time units) Maximum age of the index.

    max_docs

    (Optional, integer) Maximum number of documents in the index. Documents added since the last refresh are not included in the document count. The document count does not include documents in replica shards.

    max_size

    (Optional, byte units) Maximum index size. This is the total size of all primary shards in the index. Replicas are not counted toward the maximum index size.

To see the current index size, use the _cat indices API. The pri.store.size value shows the combined size of all primary shards.

mappings

(Optional, mapping object) Mapping for fields in the index. If specified, this mapping can include:

See Mapping.

settings

(Optional, index setting object) Configuration options for the index. See Index Settings.

Examples

Basic example

  1. PUT /logs-000001 
  2. {
  3.   "aliases": {
  4.     "logs_write": {}
  5.   }
  6. }
  8. # Add > 1000 documents to logs-000001
  10. POST /logs_write/_rollover 
  11. {
  12.   "conditions": {
  13.     "max_age":   "7d",
  14.     "max_docs":  1000,
  15.     "max_size":  "5gb"
  16.   }
  17. }

Creates an index called logs-0000001 with the alias logs_write.

If the index pointed to by logs_write was created 7 or more days ago, or contains 1,000 or more documents, or has an index size at least around 5GB, then the logs-000002 index is created and the logs_write alias is updated to point to logs-000002.

The API returns the following response:

  1. {
  2.   "acknowledged": true,
  3.   "shards_acknowledged": true,
  4.   "old_index": "logs-000001",
  5.   "new_index": "logs-000002",
  6.   "rolled_over": true, 
  7.   "dry_run": false, 
  8.   "conditions": { 
  9.     "[max_age: 7d]": false,
  10.     "[max_docs: 1000]": true,
  11.     "[max_size: 5gb]": false,
  12.   }
  13. }

Whether the index was rolled over.

Whether the rollover was dry run.

The result of each condition.

Roll over a data stream

  1. PUT _index_template/template
  2. {
  3.   "index_patterns": ["my-data-stream*"],
  4.   "data_stream": { }
  5. }
  1. PUT /_data_stream/my-data-stream 
  3. # Add > 1000 documents to my-data-stream
  5. POST /my-data-stream/_rollover 
  6. {
  7.   "conditions" : {
  8.     "max_age": "7d",
  9.     "max_docs": 1000,
  10.     "max_size": "5gb"
  11.   }
  12. }

Creates a data stream called my-data-stream with one initial backing index named my-data-stream-000001.

This request creates a new backing index, my-data-stream-000002, and adds it as the write index for my-data-stream if the current write index meets at least one of the following conditions:

  • The index was created 7 or more days ago.
  • The index has an index size of 5GB or greater.
  • The index contains 1,000 or more documents.

The API returns the following response:

  1. {
  2.   "acknowledged": true,
  3.   "shards_acknowledged": true,
  4.   "old_index": ".ds-my-data-stream-000001", 
  5.   "new_index": ".ds-my-data-stream-000002", 
  6.   "rolled_over": true, 
  7.   "dry_run": false, 
  8.   "conditions": { 
  9.     "[max_age: 7d]": false,
  10.     "[max_docs: 1000]": true,
  11.     "[max_size: 5gb]": false,
  12.   }
  13. }

The previous write index for the data stream.

The new write index for the data stream.

Whether the index was rolled over.

Whether the rollover was dry run.

The result of each condition.

Specify settings for the target index

The settings, mappings, and aliases for the new index are taken from any matching index templates. If the rollover target is an index alias, you can specify settings, mappings, and aliases in the body of the request just like the create index API. Values specified in the request override any values set in matching index templates. For example, the following rollover request overrides the index.number_of_shards setting:

  1. PUT /logs-000001
  2. {
  3.   "aliases": {
  4.     "logs_write": {}
  5.   }
  6. }
  8. POST /logs_write/_rollover
  9. {
  10.   "conditions" : {
  11.     "max_age": "7d",
  12.     "max_docs": 1000,
  13.     "max_size": "5gb"
  14.   },
  15.   "settings": {
  16.     "index.number_of_shards": 2
  17.   }
  18. }

Specify a target index name

If the rollover target is an index alias and the name of the existing index ends with - and a number — e.g. logs-000001 — then the name of the new index will follow the same pattern, incrementing the number (logs-000002). The number is zero-padded with a length of 6, regardless of the old index name.

If the old name doesn’t match this pattern then you must specify the name for the new index as follows:

  1. POST /my_alias/_rollover/my_new_index_name
  2. {
  3.   "conditions": {
  4.     "max_age":   "7d",
  5.     "max_docs":  1000,
  6.     "max_size": "5gb"
  7.   }
  8. }

Use date math with a rollover

If the rollover target is an index alias, it can be useful to use date math to name the rollover index according to the date that the index rolled over, e.g. logstash-2016.02.03. The rollover API supports date math, but requires the index name to end with a dash followed by a number, e.g. logstash-2016.02.03-1 which is incremented every time the index is rolled over. For instance:

  1. # PUT /<logs-{now/d}-1> with URI encoding:
  2. PUT /%3Clogs_%7Bnow%2Fd%7D-1%3E 
  3. {
  4.   "aliases": {
  5.     "logs_write": {}
  6.   }
  7. }
  9. PUT logs_write/_doc/1
  10. {
  11.   "message": "a dummy log"
  12. }
  14. POST logs_write/_refresh
  16. # Wait for a day to pass
  18. POST /logs_write/_rollover 
  19. {
  20.   "conditions": {
  21.     "max_docs":   "1"
  22.   }
  23. }

Creates an index named with today’s date (e.g.) logs_2016.10.31-1

Rolls over to a new index with today’s date, e.g. logs_2016.10.31-000002 if run immediately, or logs-2016.11.01-000002 if run after 24 hours

These indices can then be referenced as described in the date math documentation. For example, to search over indices created in the last three days, you could do the following:

  1. # GET /<logs-{now/d}-*>,<logs-{now/d-1d}-*>,<logs-{now/d-2d}-*>/_search
  2. GET /%3Clogs-%7Bnow%2Fd%7D-*%3E%2C%3Clogs-%7Bnow%2Fd-1d%7D-*%3E%2C%3Clogs-%7Bnow%2Fd-2d%7D-*%3E/_search

Dry run

The rollover API supports dry_run mode, where request conditions can be checked without performing the actual rollover.

  1. POST /logs_write/_rollover?dry_run
  2. {
  3.   "conditions" : {
  4.     "max_age": "7d",
  5.     "max_docs": 1000,
  6.     "max_size": "5gb"
  7.   }
  8. }

Roll over a write index

If the rollover target is an index alias for a write index that has is_write_index explicitly set to true, it is not swapped during rollover actions. Since having an alias point to multiple indices is ambiguous in distinguishing which is the correct write index to roll over, it is not valid to rollover an alias that points to multiple indices. For this reason, the default behavior is to swap which index is being pointed to by the write-oriented alias. This was logs_write in some of the above examples. Since setting is_write_index enables an alias to point to multiple indices while also being explicit as to which is the write index that rollover should target, removing the alias from the rolled over index is not necessary. This simplifies things by allowing for one alias to behave both as the write and read aliases for indices that are being managed with Rollover.

Look at the behavior of the aliases in the following example where is_write_index is set on the rolled over index.

  1. PUT my_logs_index-000001
  2. {
  3.   "aliases": {
  4.     "logs": { "is_write_index": true } 
  5.   }
  6. }
  8. PUT logs/_doc/1
  9. {
  10.   "message": "a dummy log"
  11. }
  13. POST logs/_refresh
  15. POST /logs/_rollover
  16. {
  17.   "conditions": {
  18.     "max_docs":   "1"
  19.   }
  20. }
  22. PUT logs/_doc/2 
  23. {
  24.   "message": "a newer log"
  25. }

configures my_logs_index as the write index for the logs alias

newly indexed documents against the logs alias will write to the new index

  1. {
  2.   "_index" : "my_logs_index-000002",
  3.   "_type" : "_doc",
  4.   "_id" : "2",
  5.   "_version" : 1,
  6.   "result" : "created",
  7.   "_shards" : {
  8.     "total" : 2,
  9.     "successful" : 1,
  10.     "failed" : 0
  11.   },
  12.   "_seq_no" : 0,
  13.   "_primary_term" : 1
  14. }

After the rollover, the alias metadata for the two indices will have the is_write_index setting reflect each index’s role, with the newly created index as the write index.

  1. {
  2.   "my_logs_index-000002": {
  3.     "aliases": {
  4.       "logs": { "is_write_index": true }
  5.     }
  6.   },
  7.   "my_logs_index-000001": {
  8.     "aliases": {
  9.       "logs": { "is_write_index" : false }
  10.     }
  11.   }
  12. }