Kibana Query Language

In Kibana 6.3, we introduced a number of exciting experimental query language enhancements. These features are now available by default in 7.0. Out of the box, Kibana’s query language now includes scripted field support and a simplified, easier to use syntax. If you have a Basic license or above, autocomplete functionality will also be enabled.

Language syntax

If you’re familiar with Kibana’s old Lucene query syntax, you should feel right at home with the new syntax. The basics stay the same, we’ve simply refined things to make the query language easier to use.

response:200 will match documents where the response field matches the value 200.

Quotes around a search term will initiate a phrase search. For example, message:"Quick brown fox" will search for the phrase “quick brown fox” in the message field. Without the quotes, your query will get broken down into tokens via the message field’s configured analyzer and will match documents that contain those tokens, regardless of the order in which they appear. This means documents with “quick brown fox” will match, but so will “quick fox brown”. Remember to use quotes if you want to search for a phrase.

The query parser will no longer split on whitespace. Multiple search terms must be separated by explicit boolean operators. Lucene will combine search terms with an or by default, so response:200 extension:php would become response:200 or extension:php in KQL. This will match documents where response matches 200, extension matches php, or both. Note that boolean operators are not case sensitive.

We can make terms required by using and.

response:200 and extension:php will match documents where response matches 200 and extension matches php.

By default, and has a higher precedence than or.

response:200 and extension:php or extension:css will match documents where response is 200 and extension is php OR documents where extension is css and response is anything.

We can override the default precedence with grouping.

response:200 and (extension:php or extension:css) will match documents where response is 200 and extension is either php or css.

A shorthand exists that allows us to easily search a single field for multiple values.

response:(200 or 404) searches for docs where the response field matches 200 or 404. We can also search for docs with multi-value fields that contain a list of terms, for example: tags:(success and info and security)

Terms can be inverted by prefixing them with not.

not response:200 will match all documents where response is not 200.

Entire groups can also be inverted.

response:200 and not (extension:php or extension:css)

Ranges are similar to lucene with a small syntactical difference.

Instead of bytes:>1000, we omit the colon: bytes > 1000.

>, >=, <, <= are all valid range operators.

Exist queries are simple and do not require a special operator. response:* will find all docs where the response field exists.

Wildcard queries are available. machine.os:win* would match docs where the machine.os field starts with “win”, which would match values like “windows 7” and “windows 10”.

Wildcards also allow us to search multiple fields at once. This can come in handy when you have both text and keyword versions of a field. Let’s say we have machine.os and machine.os.keyword fields and we want to check both for the term “windows 10”. We can do it like this: `machine.os*:windows 10”.

Terms without fields will be matched against the default field in your index settings. If a default field is not set these terms will be matched against all fields. For example, a query for response:200 will search for the value 200 in the response field, but a query for just 200 will search for 200 across all fields in your index.

Nested field support

KQL supports querying on nested fields through a special syntax. You can query nested fields in subtly different ways, depending on the results you want, so crafting nested queries requires extra thought.

One main consideration is how to match parts of the nested query to the individual nested documents. There are two main approaches to take:

  • Parts of the query may only match a single nested document. This is what most users want when querying on a nested field.
  • Parts of the query can match different nested documents. This is how a regular object field works. Although generally less useful, there might be occasions where you want to query a nested field in this way.

Let’s take a look at the first approach. In the following document, items is a nested field. Each document in the nested field contains a name, stock, and category.

  1. {
  2.   "grocery_name": "Elastic Eats",
  3.   "items": [
  4.     {
  5.       "name": "banana",
  6.       "stock": "12",
  7.       "category": "fruit"
  8.     },
  9.     {
  10.       "name": "peach",
  11.       "stock": "10",
  12.       "category": "fruit"
  13.     },
  14.     {
  15.       "name": "carrot",
  16.       "stock": "9",
  17.       "category": "vegetable"
  18.     },
  19.     {
  20.       "name": "broccoli",
  21.       "stock": "5",
  22.       "category": "vegetable"
  23.     }
  24.   ]
  25. }

Match a single nested document

To find stores that have more than 10 bananas in stock, you would write a query like this:

items:{ name:banana and stock > 10 }

items is the “nested path”. Everything inside the curly braces (the “nested group”) must match a single nested document.

The following example returns no matches because no single nested document has bananas with a stock of 9.

items:{ name:banana and stock:9 }

Match different nested documents

The subqueries in this example are in separate nested groups and can match different nested documents.

items:{ name:banana } and items:{ stock:9 }

name:banana matches the first document in the array and stock:9 matches the third document in the array.

Combine approaches

You can combine these two approaches to create complex queries. What if you wanted to find a store with more than 10 bananas that also stocks vegetables? You could do this:

items:{ name:banana and stock > 10 } and items:{ category:vegetable }

The first nested group (name:banana and stock > 10) must still match a single document, but the category:vegetables subquery can match a different nested document because it is in a separate group.

Nested fields inside other nested fields

KQL’s syntax also supports nested fields inside of other nested fields—you simply have to specify the full path. Suppose you have a document where level1 and level2 are both nested fields:

  1. {
  2.   "level1": [
  3.     {
  4.       "level2": [
  5.         {
  6.           "prop1": "foo",
  7.           "prop2": "bar"
  8.         },
  9.         {
  10.           "prop1": "baz",
  11.           "prop2": "qux"
  12.         }
  13.       ]
  14.     }
  15.   ]
  16. }

You can match on a single nested document by specifying the full path:

level1.level2:{ prop1:foo and prop2:bar }

Most Popular