User agent processor

The user_agent processor extracts details from the user agent string a browser sends with its web requests. This processor adds this information by default under the user_agent field.

The ingest-user-agent module ships by default with the regexes.yaml made available by uap-java with an Apache 2.0 license. For more details see https://github.com/ua-parser/uap-core.

Using the user_agent Processor in a Pipeline

Table 39. User-agent options

NameRequiredDefaultDescription

field

yes

-

The field containing the user agent string.

target_field

no

user_agent

The field that will be filled with the user agent details.

regex_file

no

-

The name of the file in the config/ingest-user-agent directory containing the regular expressions for parsing the user agent string. Both the directory and the file have to be created before starting Elasticsearch. If not specified, ingest-user-agent will use the regexes.yaml from uap-core it ships with (see below).

properties

no

[name, major, minor, patch, build, os, os_name, os_major, os_minor, device]

Controls what properties are added to target_field.

ignore_missing

no

false

If true and field does not exist, the processor quietly exits without modifying the document

ecs

no

true

Whether to return the output in Elastic Common Schema format. NOTE: This setting is deprecated and will be removed in a future version.

Here is an example that adds the user agent details to the user_agent field based on the agent field:

  1. PUT _ingest/pipeline/user_agent
  2. {
  3.   "description" : "Add user agent information",
  4.   "processors" : [
  5.     {
  6.       "user_agent" : {
  7.         "field" : "agent"
  8.       }
  9.     }
  10.   ]
  11. }
  12. PUT my-index-000001/_doc/my_id?pipeline=user_agent
  13. {
  14.   "agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36"
  15. }
  16. GET my-index-000001/_doc/my_id

Which returns

  1. {
  2.   "found": true,
  3.   "_index": "my-index-000001",
  4.   "_type": "_doc",
  5.   "_id": "my_id",
  6.   "_version": 1,
  7.   "_seq_no": 22,
  8.   "_primary_term": 1,
  9.   "_source": {
  10.     "agent": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36",
  11.     "user_agent": {
  12.       "name": "Chrome",
  13.       "original": "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/51.0.2704.103 Safari/537.36",
  14.       "version": "51.0.2704.103",
  15.       "os": {
  16.         "name": "Mac OS X",
  17.         "version": "10.10.5",
  18.         "full": "Mac OS X 10.10.5"
  19.       },
  20.       "device" : {
  21.         "name" : "Mac"
  22.       },
  23.     }
  24.   }
  25. }

Using a custom regex file

To use a custom regex file for parsing the user agents, that file has to be put into the config/ingest-user-agent directory and has to have a .yml filename extension. The file has to be present at node startup, any changes to it or any new files added while the node is running will not have any effect.

In practice, it will make most sense for any custom regex file to be a variant of the default file, either a more recent version or a customised version.

The default file included in ingest-user-agent is the regexes.yaml from uap-core: https://github.com/ua-parser/uap-core/blob/master/regexes.yaml

Node Settings

The user_agent processor supports the following setting:

ingest.user_agent.cache_size

The maximum number of results that should be cached. Defaults to 1000.

Note that these settings are node settings and apply to all user_agent processors, i.e. there is one cache for all defined user_agent processors.