Handling Nested Fields in Conditionals

Source documents often contain nested fields. Care should be taken to avoid NullPointerExceptions if the parent object does not exist in the document. For example ctx.a.b.c can throw an NullPointerExceptions if the source document does not have top level a object, or a second level b object.

To help protect against NullPointerExceptions, null safe operations should be used. Fortunately, Painless makes null safe operations easy with the ?. operator.

  1. PUT _ingest/pipeline/drop_guests_network
  2. {
  3.   "processors": [
  4.     {
  5.       "drop": {
  6.         "if": "ctx.network?.name == 'Guest'"
  7.       }
  8.     }
  9.   ]
  10. }

The following document will get dropped correctly:

  1. POST test/_doc/1?pipeline=drop_guests_network
  2. {
  3.   "network": {
  4.     "name": "Guest"
  5.   }
  6. }

Thanks to the ?. operator the following document will not throw an error. If the pipeline used a . the following document would throw a NullPointerException since the network object is not part of the source document.

  1. POST test/_doc/2?pipeline=drop_guests_network
  2. {
  3.   "foo" : "bar"
  4. }

The source document can also use dot delimited fields to represent nested fields.

For example instead the source document defining the fields nested:

  1. {
  2.   "network": {
  3.     "name": "Guest"
  4.   }
  5. }

The source document may have the nested fields flattened as such:

  1. {
  2.   "network.name": "Guest"
  3. }

If this is the case, use the Dot Expand Processor so that the nested fields may be used in a conditional.

  1. PUT _ingest/pipeline/drop_guests_network
  2. {
  3.   "processors": [
  4.     {
  5.       "dot_expander": {
  6.         "field": "network.name"
  7.       }
  8.     },
  9.     {
  10.       "drop": {
  11.         "if": "ctx.network?.name == 'Guest'"
  12.       }
  13.     }
  14.   ]
  15. }

Now the following input document can be used with a conditional in the pipeline.

  1. POST test/_doc/3?pipeline=drop_guests_network
  2. {
  3.   "network.name": "Guest"
  4. }

The ?. operators works well for use in the if conditional because the null safe operator returns null if the object is null and == is null safe (as well as many other painless operators).

However, calling a method such as .equalsIgnoreCase is not null safe and can result in a NullPointerException.

Some situations allow for the same functionality but done so in a null safe manner. For example: 'Guest'.equalsIgnoreCase(ctx.network?.name) is null safe because Guest is always non null, but ctx.network?.name.equalsIgnoreCase('Guest') is not null safe since ctx.network?.name can return null.

Some situations require an explicit null check. In the following example there is not null safe alternative, so an explicit null check is needed.

  1. {
  2.   "drop": {
  3.     "if": "ctx.network?.name != null && ctx.network.name.contains('Guest')"
  4.   }
  5. }