Timelion

Timelion is a time series data visualizer that enables you to combine totally independent data sources within a single visualization. It’s driven by a simple expression language you use to retrieve time series data, perform calculations to tease out the answers to complex questions, and visualize the results.

For example, Timelion enables you to easily get the answers to questions like:

Before you begin

In this tutorial, you’ll use the time series data from Metricbeat. To ingest the data locally, download Metricbeat.

Create time series visualizations

To compare the real-time percentage of CPU time spent in user space to the results offset by one hour, create a time series visualization.

Define the functions

To start tracking the real-time percentage of CPU, enter the following in the Timelion Expression field:

  1. .es(index=metricbeat-*,
  2.     timefield='@timestamp',
  3.     metric='avg:system.cpu.user.pct')

timelion create01

Compare the data

To compare the two data sets, add another series with data from the previous hour, separated by a comma:

  1. .es(index=metricbeat-*,
  2.     timefield='@timestamp',
  3.     metric='avg:system.cpu.user.pct'),
  4. .es(offset=-1h,             
  5.     index=metricbeat-*,
  6.     timefield='@timestamp',
  7.     metric='avg:system.cpu.user.pct')

offset offsets the data retrieval by a date expression. In this example, -1h offsets the data back by one hour.

timelion create02

Add label names

To easily distinguish between the two data sets, add the label names:

  1. .es(offset=-1h,index=metricbeat-*,
  2.     timefield='@timestamp',
  3.     metric='avg:system.cpu.user.pct').label('last hour'),
  4. .es(index=metricbeat-*,
  5.     timefield='@timestamp',
  6.     metric='avg:system.cpu.user.pct').label('current hour')

.label() adds custom labels to the visualization.

timelion create03

Add a title

Add a meaningful title:

  1. .es(offset=-1h,
  2.     index=metricbeat-*,
  3.     timefield='@timestamp',
  4.     metric='avg:system.cpu.user.pct')
  5.   .label('last hour'),
  6. .es(index=metricbeat-*,
  7.     timefield='@timestamp',
  8.     metric='avg:system.cpu.user.pct')
  9.   .label('current hour')
  10.   .title('CPU usage over time')

.title() adds a title with a meaningful name. Titles make is easier for unfamiliar users to understand the purpose of the visualization.

timelion customize01

Change the chart type

To differentiate between the current hour data and the last hour data, change the chart type:

  1. .es(offset=-1h,
  2.     index=metricbeat-*,
  3.     timefield='@timestamp',
  4.     metric='avg:system.cpu.user.pct')
  5.   .label('last hour')
  6.   .lines(fill=1,width=0.5), 
  7. .es(index=metricbeat-*,
  8.     timefield='@timestamp',
  9.     metric='avg:system.cpu.user.pct')
  10.   .label('current hour')
  11.   .title('CPU usage over time')

.lines() changes the appearance of the chart lines. In this example, .lines(fill=1,width=0.5) sets the fill level to 1, and the border width to 0.5.

timelion customize02

Change the line colors

To make the current hour data stand out, change the line colors:

  1. .es(offset=-1h,
  2.     index=metricbeat-*,
  3.     timefield='@timestamp',
  4.     metric='avg:system.cpu.user.pct')
  5.   .label('last hour')
  6.   .lines(fill=1,width=0.5)
  7.   .color(gray), 
  8. .es(index=metricbeat-*,
  9.     timefield='@timestamp',
  10.     metric='avg:system.cpu.user.pct')
  11.   .label('current hour')
  12.   .title('CPU usage over time')
  13.   .color(#1E90FF)

.color() changes the color of the data. Supported color types include standard color names, hexadecimal values, or a color schema for grouped data. In this example, .color(gray) represents the last hour, and .color(#1E90FF) represents the current hour.

timelion customize03

Make adjustments to the legend

Change the position and style of the legend:

  1. .es(offset=-1h,
  2.     index=metricbeat-*,
  3.     timefield='@timestamp',
  4.     metric='avg:system.cpu.user.pct')
  5.   .label('last hour')
  6.   .lines(fill=1,width=0.5)
  7.   .color(gray),
  8. .es(index=metricbeat-*,
  9.     timefield='@timestamp',
  10.     metric='avg:system.cpu.user.pct')
  11.   .label('current hour')
  12.   .title('CPU usage over time')
  13.   .color(#1E90FF)
  14.   .legend(columns=2, position=nw)

.legend() sets the position and style of the legend. In this example, .legend(columns=2, position=nw) places the legend in the north west position of the visualization with two columns.

timelion customize04

Create visualizations with mathematical functions

To create a visualization for inbound and outbound network traffic, use mathematical functions.

Define the functions

To start tracking the inbound and outbound network traffic, enter the following in the Timelion Expression field:

  1. .es(index=metricbeat*,
  2.     timefield=@timestamp,
  3.     metric=max:system.network.in.bytes)

timelion math01

Plot the rate of change

Change how the data is displayed so that you can easily monitor the inbound traffic:

  1. .es(index=metricbeat*,
  2.     timefield=@timestamp,
  3.     metric=max:system.network.in.bytes)
  4.   .derivative()

.derivative plots the change in values over time.

timelion math02

Add a similar calculation for outbound traffic:

  1. .es(index=metricbeat*,
  2.     timefield=@timestamp,
  3.     metric=max:system.network.in.bytes)
  4.   .derivative(),
  5. .es(index=metricbeat*,
  6.     timefield=@timestamp,
  7.     metric=max:system.network.out.bytes)
  8.   .derivative()
  9.   .multiply(-1)

.multiply() multiplies the data series by a number, the result of a data series, or a list of data series. For this example, .multiply(-1) converts the outbound network traffic to a negative value since the outbound network traffic is leaving your machine.

timelion math03

Change the data metric

To make the visualization easier to analyze, change the data metric from bytes to megabytes:

  1. .es(index=metricbeat*,
  2.     timefield=@timestamp,
  3.     metric=max:system.network.in.bytes)
  4.   .derivative()
  5.   .divide(1048576),
  6. .es(index=metricbeat*,
  7.     timefield=@timestamp,
  8.     metric=max:system.network.out.bytes)
  9.   .derivative()
  10.   .multiply(-1)
  11.   .divide(1048576)

.divide() accepts the same input as .multiply(), then divides the data series by the defined divisor.

timelion math04

Customize and format the visualization

Customize and format the visualization using functions:

  1. .es(index=metricbeat*,
  2.     timefield=@timestamp,
  3.     metric=max:system.network.in.bytes)
  4.   .derivative()
  5.   .divide(1048576)
  6.   .lines(fill=2, width=1)
  7.   .color(green)
  8.   .label("Inbound traffic")         
  9.   .title("Network traffic (MB/s)"), 
  10. .es(index=metricbeat*,
  11.     timefield=@timestamp,
  12.     metric=max:system.network.out.bytes)
  13.   .derivative()
  14.   .multiply(-1)
  15.   .divide(1048576)
  16.   .lines(fill=2, width=1)           
  17.   .color(blue)                      
  18.   .label("Outbound traffic")
  19.   .legend(columns=2, position=nw)

.label() adds custom labels to the visualization.

.title() adds a title with a meaningful name.

.lines() changes the appearance of the chart lines. In this example, .lines(fill=2, width=1) sets the fill level to 2, and the border width to 1.

.color() changes the color of the data. Supported color types include standard color names, hexadecimal values, or a color schema for grouped data. In this example, .color(green) represents the inbound network traffic, and .color(blue) represents the outbound network traffic.

.legend() sets the position and style of the legend. For this example, legend(columns=2, position=nw) places the legend in the north west position of the visualization with two columns.

timelion math05

Create visualizations with conditional logic and tracking trends

To easily detect outliers and discover patterns over time, modify time series data with conditional logic and create a trend with a moving average.

With Timelion conditional logic, you can use the following operator values to compare your data:

eq

equal

ne

not equal

lt

less than

lte

less than or equal to

gt

greater than

gte

greater than or equal to

Define the functions

To chart the maximum value of system.memory.actual.used.bytes, enter the following in the Timelion Expression field:

  1. .es(index=metricbeat-*,
  2.     timefield='@timestamp',
  3.     metric='max:system.memory.actual.used.bytes')

timelion conditional01

Track used memory

To track the amount of memory used, create two thresholds:

  1. .es(index=metricbeat-*,
  2.     timefield='@timestamp',
  3.     metric='max:system.memory.actual.used.bytes'),
  4. .es(index=metricbeat-*,
  5.     timefield='@timestamp',
  6.     metric='max:system.memory.actual.used.bytes')
  7.   .if(gt,                             
  8.       11300000000,                    
  9.       .es(index=metricbeat-*,
  10.           timefield='@timestamp',
  11.           metric='max:system.memory.actual.used.bytes'),
  12.       null)
  13.     .label('warning')
  14.     .color('#FFCC11'),
  15. .es(index=metricbeat-*,
  16.     timefield='@timestamp',
  17.     metric='max:system.memory.actual.used.bytes')
  18.   .if(gt,
  19.       11375000000,
  20.       .es(index=metricbeat-*,
  21.           timefield='@timestamp',
  22.           metric='max:system.memory.actual.used.bytes'),
  23.       null)
  24.   .label('severe')
  25.   .color('red')

Timelion conditional logic for the greater than operator. In this example, the warning threshold is 11.3GB (11300000000), and the severe threshold is 11.375GB (11375000000). If the threshold values are too high or low for your machine, adjust the values accordingly.

if() compares each point to a number. If the condition evaluates to true, adjust the styling. If the condition evaluates to false, use the default styling.

timelion conditional02

Determine the trend

To determine the trend, create a new data series:

  1. .es(index=metricbeat-*,
  2.     timefield='@timestamp',
  3.     metric='max:system.memory.actual.used.bytes'),
  4. .es(index=metricbeat-*,
  5.     timefield='@timestamp',
  6.     metric='max:system.memory.actual.used.bytes')
  7.   .if(gt,11300000000,
  8.       .es(index=metricbeat-*,
  9.           timefield='@timestamp',
  10.           metric='max:system.memory.actual.used.bytes'),
  11.       null)
  12.       .label('warning')
  13.       .color('#FFCC11'),
  14. .es(index=metricbeat-*,
  15.     timefield='@timestamp',
  16.     metric='max:system.memory.actual.used.bytes')
  17.   .if(gt,11375000000,
  18.       .es(index=metricbeat-*,
  19.           timefield='@timestamp',
  20.           metric='max:system.memory.actual.used.bytes'),
  21.       null).
  22.       label('severe')
  23.       .color('red'),
  24. .es(index=metricbeat-*,
  25.     timefield='@timestamp',
  26.     metric='max:system.memory.actual.used.bytes')
  27.   .mvavg(10)

mvavg() calculates the moving average over a specified period of time. In this example, .mvavg(10) creates a moving average with a window of 10 data points.

timelion conditional03

Customize and format the visualization

Customize and format the visualization using functions:

  1. .es(index=metricbeat-*,
  2.     timefield='@timestamp',
  3.     metric='max:system.memory.actual.used.bytes')
  4.   .label('max memory')                    
  5.   .title('Memory consumption over time'), 
  6. .es(index=metricbeat-*,
  7.     timefield='@timestamp',
  8.     metric='max:system.memory.actual.used.bytes')
  9.   .if(gt,
  10.       11300000000,
  11.       .es(index=metricbeat-*,
  12.           timefield='@timestamp',
  13.           metric='max:system.memory.actual.used.bytes'),
  14.       null)
  15.     .label('warning')
  16.     .color('#FFCC11')                 
  17.     .lines(width=5),                  
  18. .es(index=metricbeat-*,
  19.     timefield='@timestamp',
  20.     metric='max:system.memory.actual.used.bytes')
  21.   .if(gt,
  22.       11375000000,
  23.       .es(index=metricbeat-*,
  24.           timefield='@timestamp',
  25.           metric='max:system.memory.actual.used.bytes'),
  26.       null)
  27.     .label('severe')
  28.     .color('red')
  29.     .lines(width=5),
  30. .es(index=metricbeat-*,
  31.     timefield='@timestamp',
  32.     metric='max:system.memory.actual.used.bytes')
  33.   .mvavg(10)
  34.   .label('mvavg')
  35.   .lines(width=2)
  36.   .color(#5E5E5E)
  37.   .legend(columns=4, position=nw)

.label() adds custom labels to the visualization.

.title() adds a title with a meaningful name.

.color() changes the color of the data. Supported color types include standard color names, hexadecimal values, or a color schema for grouped data.

.lines() changes the appearance of the chart lines. In this example, .lines(width=5) sets border width to 5.

.legend() sets the position and style of the legend. For this example, (columns=4, position=nw) places the legend in the north west position of the visualization with four columns.

timelion conditional04

For additional information on Timelion conditional capabilities, go to I have but one .condition().

Most Popular