_routing field

A document is routed to a particular shard in an index using the following formula:

  1. shard_num = hash(_routing) % num_primary_shards

The default value used for _routing is the document’s _id.

Custom routing patterns can be implemented by specifying a custom routing value per document. For instance:

  1. PUT my-index-000001/_doc/1?routing=user1&refresh=true 
  2. {
  3.   "title": "This is a document"
  4. }
  6. GET my-index-000001/_doc/1?routing=user1

This document uses user1 as its routing value, instead of its ID.

The same routing value needs to be provided when getting, deleting, or updating the document.

The value of the _routing field is accessible in queries:

  1. GET my-index-000001/_search
  2. {
  3.   "query": {
  4.     "terms": {
  5.       "_routing": [ "user1" ] 
  6.     }
  7.   }
  8. }

Querying on the _routing field (also see the ids query)

Data streams do not support custom routing. Instead, target the appropriate backing index for the stream.

Searching with custom routing

Custom routing can reduce the impact of searches. Instead of having to fan out a search request to all the shards in an index, the request can be sent to just the shard that matches the specific routing value (or values):

  1. GET my-index-000001/_search?routing=user1,user2 
  2. {
  3.   "query": {
  4.     "match": {
  5.       "title": "document"
  6.     }
  7.   }
  8. }

This search request will only be executed on the shards associated with the user1 and user2 routing values.

Making a routing value required

When using custom routing, it is important to provide the routing value whenever indexing, getting, deleting, or updating a document.

Forgetting the routing value can lead to a document being indexed on more than one shard. As a safeguard, the _routing field can be configured to make a custom routing value required for all CRUD operations:

  1. PUT my-index-000002
  2. {
  3.   "mappings": {
  4.     "_routing": {
  5.       "required": true 
  6.     }
  7.   }
  8. }
  10. PUT my-index-000002/_doc/1 
  11. {
  12.   "text": "No routing value provided"
  13. }

Routing is required for all documents.

This index request throws a routing_missing_exception.

Unique IDs with custom routing

When indexing documents specifying a custom _routing, the uniqueness of the _id is not guaranteed across all of the shards in the index. In fact, documents with the same _id might end up on different shards if indexed with different _routing values.

It is up to the user to ensure that IDs are unique across the index.

Routing to an index partition

An index can be configured such that custom routing values will go to a subset of the shards rather than a single shard. This helps mitigate the risk of ending up with an imbalanced cluster while still reducing the impact of searches.

This is done by providing the index level setting index.routing_partition_size at index creation. As the partition size increases, the more evenly distributed the data will become at the expense of having to search more shards per request.

When this setting is present, the formula for calculating the shard becomes:

  1. shard_num = (hash(_routing) + hash(_id) % routing_partition_size) % num_primary_shards

That is, the _routing field is used to calculate a set of shards within the index and then the _id is used to pick a shard within that set.

To enable this feature, the index.routing_partition_size should have a value greater than 1 and less than index.number_of_shards.

Once enabled, the partitioned index will have the following limitations:

  • Mappings with join field relationships cannot be created within it.
  • All mappings within the index must have the _routing field marked as required.