Example: Enrich your data by matching a value to a range

Example: Enrich your data by matching a value to a range

A range enrich policy uses a term query to match a number, date, or IP address in incoming documents to a range of the same type in the enrich index. Matching a range to a range is not supported.

The following example creates a range enrich policy that adds a descriptive network name and responsible department to incoming documents based on an IP address. It then adds the enrich policy to a processor in an ingest pipeline.

Use the create index API with the appropriate mappings to create a source index.

  1. PUT /networks
  2. {
  3.   "mappings": {
  4.     "properties": {
  5.       "range": { "type": "ip_range" },
  6.       "name": { "type": "keyword" },
  7.       "department": { "type": "keyword" }
  8.     }
  9.   }
  10. }

The following index API request indexes a new document to that index.

  1. PUT /networks/_doc/1?refresh=wait_for
  2. {
  3.   "range": "10.100.0.0/16",
  4.   "name": "production",
  5.   "department": "OPS"
  6. }

Use the create enrich policy API to create an enrich policy with the range policy type. This policy must include:

  • One or more source indices
  • A match_field, the field from the source indices used to match incoming documents
  • Enrich fields from the source indices you’d like to append to incoming documents

Since we plan to enrich documents based on an IP address, the policy’s match_field must be an ip_range field.

  1. PUT /_enrich/policy/networks-policy
  2. {
  3.   "range": {
  4.     "indices": "networks",
  5.     "match_field": "range",
  6.     "enrich_fields": ["name", "department"]
  7.   }
  8. }

Use the execute enrich policy API to create an enrich index for the policy.

  1. POST /_enrich/policy/networks-policy/_execute

Use the create or update pipeline API to create an ingest pipeline. In the pipeline, add an enrich processor that includes:

  • Your enrich policy.
  • The field of incoming documents used to match documents from the enrich index.
  • The target_field used to store appended enrich data for incoming documents. This field contains the match_field and enrich_fields specified in your enrich policy.
  1. PUT /_ingest/pipeline/networks_lookup
  2. {
  3.   "processors" : [
  4.     {
  5.       "enrich" : {
  6.         "description": "Add 'network' data based on 'ip'",
  7.         "policy_name": "networks-policy",
  8.         "field" : "ip",
  9.         "target_field": "network",
  10.         "max_matches": "10"
  11.       }
  12.     }
  13.   ]
  14. }

Use the ingest pipeline to index a document. The incoming document should include the field specified in your enrich processor.

  1. PUT /my-index-000001/_doc/my_id?pipeline=networks_lookup
  2. {
  3.   "ip": "10.100.34.1"
  4. }

To verify the enrich processor matched and appended the appropriate field data, use the get API to view the indexed document.

  1. GET /my-index-000001/_doc/my_id

The API returns the following response:

  1. {
  2.   "_index" : "my-index-000001",
  3.   "_type" : "_doc",
  4.   "_id" : "my_id",
  5.   "_version" : 1,
  6.   "_seq_no" : 0,
  7.   "_primary_term" : 1,
  8.   "found" : true,
  9.   "_source" : {
  10.     "ip" : "10.100.34.1",
  11.     "network" : [
  12.       {
  13.         "name" : "production",
  14.         "range" : "10.100.0.0/16",
  15.         "department" : "OPS"
  16.       }
  17.     ]
  18.   }
  19. }