Reference for Kibana extensions

Kibana has extended Vega and Vega-Lite with extensions that support:

  • Default height and width
  • Default theme to match Kibana
  • Writing Elasticsearch queries using the time range and filters from dashboards
  • Using the Elastic Map Service in Vega maps
  • Additional tooltip styling
  • Advanced setting to enable URL loading from any domain
  • Limited debugging support using the browser dev tools
  • (Vega only) Expression functions which can update the time range and dashboard filters

Default height and width

By default, Vega visualizations use the autosize = { type: 'fit', contains: 'padding' } layout. fit uses all available space, ignores width and height values, and respects the padding values. To override this behavior, change the autosize value.

Default theme to match Kibana

Kibana registers a default Vega color scheme with the id elastic, and sets a default color for each mark type. Override it by providing a different stroke, fill, or color (Vega-Lite) value.

Writing Elasticsearch queries in Vega

[experimental] This functionality is experimental and may be changed or removed completely in a future release. Elastic will take a best effort approach to fix any issues, but experimental features are not subject to the support SLA of official GA features. Kibana extends the Vega data elements with support for direct Elasticsearch queries specified as a url.

Because of this, Kibana is unable to support dynamically loaded data, which would otherwise work in Vega. All data is fetched before it’s passed to the Vega renderer.

To define an Elasticsearch query in Vega, set the url to an object. Kibana will parse the object looking for special tokens that allow your query to integrate with Kibana. These tokens are:

  • %context%: true: Set at the top level, and replaces the query section with filters from dashboard
  • %timefield%: <name>: Set at the top level, integrates the query with the dashboard time filter
  • {%timefilter%: true}: Replaced by an Elasticsearch range query with upper and lower bounds
  • {%timefilter%: "min" | "max"}: Replaced only by the upper or lower bounds
  • {%timefilter: true, shift: -1, unit: 'hour'}: Generates a time range query one hour in the past
  • {%autointerval%: true}: Replaced by the string which contains the automatic Kibana time interval, such as 1h
  • {%autointerval%: 10}: Replaced by a string which is approximately dividing the time into 10 ranges, allowing you to influence the automatic interval
  • "%dashboard_context-must_clause%": String replaced by object containing filters
  • "%dashboard_context-filter_clause%": String replaced by an object containing filters
  • "%dashboard_context-must_not_clause%": String replaced by an object containing filters

Putting this together, an example query that counts the number of documents in a specific index:

  1. // An object instead of a string for the URL value
  2. // is treated as a context-aware Elasticsearch query.
  3. url: {
  4.   // Specify the time filter.
  5.   %timefield%: @timestamp
  6.   // Apply dashboard context filters when set
  7.   %context%: true
  9.   // Which indexes to search
  10.   index: kibana_sample_data_logs
  11.   // The body element may contain "aggs" and "query" keys
  12.   body: {
  13.     aggs: {
  14.       time_buckets: {
  15.         date_histogram: {
  16.           // Use date histogram aggregation on @timestamp field
  17.           field: @timestamp 
  18.           // interval value will depend on the time filter
  19.           // Use an integer to set approximate bucket count
  20.           interval: { %autointerval%: true }
  21.           // Make sure we get an entire range, even if it has no data
  22.           extended_bounds: {
  23.             min: { %timefilter%: "min" }
  24.             max: { %timefilter%: "max" }
  25.           }
  26.           // Use this for linear (e.g. line, area) graphs
  27.           // Without it, empty buckets will not show up
  28.           min_doc_count: 0
  29.         }
  30.       }
  31.     }
  32.     // Speed up the response by only including aggregation results
  33.     size: 0
  34.   }
  35. }

@timestamp — Filters the time range and breaks it into histogram buckets.

The full result includes the following structure:

  1. {
  2.   "aggregations": {
  3.     "time_buckets": {
  4.       "buckets": [{
  5.           "key_as_string": "2015-11-30T22:00:00.000Z",
  6.           "key": 1448920800000,
  7.           "doc_count": 28
  8.         }, {
  9.           "key_as_string": "2015-11-30T23:00:00.000Z",
  10.           "key": 1448924400000, 
  11.           "doc_count": 330
  12.         }, ...

“key” — The unix timestamp you can use without conversions by the Vega date expressions.

For most visualizations, you only need the list of bucket values. To focus on only the data you need, use format: {property: "aggregations.time_buckets.buckets"}.

Specify a query with individual range and dashboard context. The query is equivalent to "%context%": true, "%timefield%": "@timestamp", except that the time range is shifted back by 10 minutes:

  1. {
  2.   body: {
  3.     query: {
  4.       bool: {
  5.         must: [
  6.           // This string will be replaced
  7.           // with the auto-generated "MUST" clause
  8.           "%dashboard_context-must_clause%"
  9.           {
  10.             range: {
  11.               // apply timefilter (upper right corner)
  12.               // to the @timestamp variable
  13.               @timestamp: {
  14.                 // "%timefilter%" will be replaced with
  15.                 // the current values of the time filter
  16.                 // (from the upper right corner)
  17.                 "%timefilter%": true
  18.                 // Only work with %timefilter%
  19.                 // Shift current timefilter by 10 units back
  20.                 shift: 10
  21.                 // week, day (default), hour, minute, second
  22.                 unit: minute
  23.               }
  24.             }
  25.           }
  26.         ]
  27.         must_not: [
  28.           // This string will be replaced with
  29.           // the auto-generated "MUST-NOT" clause
  30.           "%dashboard_context-must_not_clause%"
  31.         ]
  32.         filter: [
  33.           // This string will be replaced
  34.           // with the auto-generated "FILTER" clause
  35.           "%dashboard_context-filter_clause%"
  36.         ]
  37.       }
  38.     }
  39.   }
  40. }

When using "%context%": true or defining a value for "%timefield%" the body cannot contain a query. To customize the query within the VEGA specification (e.g. add an additional filter, or shift the timefilter), define your query and use the placeholders as in the example above. The placeholders will be replaced by the actual context of the dashboard or visualization once parsed.

The "%timefilter%" can also be used to specify a single min or max value. The date_histogram’s extended_bounds can be set with two values - min and max. Instead of hardcoding a value, you may use "min": {"%timefilter%": "min"}, which will be replaced with the beginning of the current time range. The shift and unit values are also supported. The "interval" can also be set dynamically, depending on the currently picked range: "interval": {"%autointerval%": 10} will try to get about 10-15 data points (buckets).

Access Elastic Map Service files

[experimental] This functionality is experimental and may be changed or removed completely in a future release. Elastic will take a best effort approach to fix any issues, but experimental features are not subject to the support SLA of official GA features. Access the Elastic Map Service files via the same mechanism:

  1. url: {
  2.   // "type" defaults to "elasticsearch" otherwise
  3.   type: emsfile
  4.   // Name of the file, exactly as in the Region map visualization
  5.   name: World Countries
  6. }
  7. // The result is a geojson file, get its features to use
  8. // this data source with the "shape" marks
  9. // https://vega.github.io/vega/docs/marks/shape/
  10. format: {property: "features"}

To enable Maps, the graph must specify type=map in the host configuration:

  1. {
  2.   "config": {
  3.     "kibana": {
  4.       "type": "map",
  6.       // Initial map position
  7.       "latitude": 40.7,   // default 0
  8.       "longitude": -74,   // default 0
  9.       "zoom": 7,          // default 2
  11.       // defaults to "default". Use false to disable base layer.
  12.       "mapStyle": false,
  14.       // default 0
  15.       "minZoom": 5,
  17.       // defaults to the maximum for the given style,
  18.       // or 25 when base is disabled
  19.       "maxZoom": 13,
  21.       // defaults to true, shows +/- buttons to zoom in/out
  22.       "zoomControl": false,
  24.       // Defaults to 'false', disables mouse wheel zoom. If set to
  25.       // 'true', map may zoom unexpectedly while scrolling dashboard
  26.       "scrollWheelZoom": false,
  28.       // When false, repaints on each move frame.
  29.       // Makes the graph slower when moving the map
  30.       "delayRepaint": true, // default true
  31.     }
  32.   },
  33.   /* the rest of Vega JSON */
  34. }

The visualization automatically injects a "projection", which you can use to calculate the position of all geo-aware marks. Additionally, you can use latitude, longitude, and zoom signals. These signals can be used in the graph, or can be updated to modify the position of the map.

Additional tooltip styling

Kibana has installed the Vega tooltip plugin, so tooltips can be defined in the ways documented there. Beyond that, Kibana also supports a configuration option for changing the tooltip position and padding:

  1. {
  2.   config: {
  3.     kibana: {
  4.       tooltips: {
  5.         position: 'top',
  6.         padding: 15
  7.       }
  8.     }
  9.   }
  10. }

Advanced setting to enable URL loading from any domain

Vega can load data from any URL, but this is disabled by default in Kibana. To change this, set vis_type_vega.enableExternalUrls: true in kibana.yml, then restart Kibana.

Browser debugging console

[experimental] This functionality is experimental and may be changed or removed completely in a future release. Elastic will take a best effort approach to fix any issues, but experimental features are not subject to the support SLA of official GA features. Use browser debugging tools (for example, F12 or Ctrl+Shift+J in Chrome) to inspect the VEGA_DEBUG variable:

  • view — Access to the Vega View object. See Vega Debugging Guide on how to inspect data and signals at runtime. For Vega-Lite, VEGA_DEBUG.view.data('source_0') gets the pre-transformed data, and VEGA_DEBUG.view.data('data_0') gets the encoded data. For Vega, it uses the data name as defined in your Vega spec.
  • vega_spec — Vega JSON graph specification after some modifications by Kibana. In case of Vega-Lite, this is the output of the Vega-Lite compiler.
  • vegalite_spec — If this is a Vega-Lite graph, JSON specification of the graph before Vega-Lite compilation.

Debugging data

[experimental] This functionality is experimental and may be changed or removed completely in a future release. Elastic will take a best effort approach to fix any issues, but experimental features are not subject to the support SLA of official GA features. If you are using an Elasticsearch query, make sure your resulting data is what you expected. The easiest way to view it is by using the “networking” tab in the browser debugging tools (for example, F12). Modify the graph slightly so that it makes a search request, and view the response from the server. Another approach is to use Dev Tools. Place the index name into the first line: GET <INDEX_NAME>/_search, then add your query as the following lines (just the value of the "query" field).

Asking for help with a Vega spec

Because of the dynamic nature of the data in Elasticsearch, it is hard to help you with Vega specs unless you can share a dataset. To do this, use the browser developer tools and type:

JSON.stringify(VEGA_DEBUG.vegalite_spec, null, 2)

Copy the response to gist.github.com, possibly with a .json extension, use the [raw] button, and share that when asking for help.

(Vega only) Expression functions which can update the time range and dashboard filters

Kibana has extended the Vega expression language with these functions:

  1. /**
  2.   * @param {object} query Elastic Query DSL snippet, as used in the query DSL editor
  3.   * @param {string} [index] as defined in Kibana, or default if missing
  4.   */
  5. kibanaAddFilter(query, index)
  7. /**
  8.   * @param {object} query Elastic Query DSL snippet, as used in the query DSL editor
  9.   * @param {string} [index] as defined in Kibana, or default if missing
  10.   */
  11. kibanaRemoveFilter(query, index)
  13. kibanaRemoveAllFilters()
  15. /**
  16.   * Update dashboard time filter to the new values
  17.   * @param {number|string|Date} start
  18.   * @param {number|string|Date} end
  19.   */
  20. kibanaSetTimeFilter(start, end)

Additional configuration options

  1. {
  2.   config: {
  3.     kibana: {
  4.       // Placement of the Vega-defined signal bindings.
  5.       // Can be `left`, `right`, `top`, or `bottom` (default).
  6.       controlsLocation: top
  7.       // Can be `vertical` or `horizontal` (default).
  8.       controlsDirection: vertical
  9.       // If true, hides most of Vega and Vega-Lite warnings
  10.       hideWarnings: true
  11.       // Vega renderer to use: `svg` or `canvas` (default)
  12.       renderer: canvas
  13.     }
  14.   }
  15. }

Most Popular