Query data with Flux

The following guides walk through both common and complex queries and use cases for Flux.

Example data variable

Many of the examples provided in the following guides use a data variable, which represents a basic query that filters data by measurement and field. data is defined as:

data = from(bucket: "db/rp")
  |> range(start: -1h)
  |> filter(fn: (r) =>
    r._measurement == "example-measurement" and
    r._field == "example-field"
  )

Flux query guides

• Execute Flux queries
• Query fields and tags
• Group
• Sort and limit
• Window & aggregate
• Transform data with math
• Calculate percentages
• Increase
• Moving Average
• Rate
• Histograms
• Fill
• Median
• Percentile & quantile
• Join
• Cumulative sum
• First and last
• Exists
• Extract scalar values
• Manipulate timestamps
• Monitor states
• Query SQL data
• Conditional logic
• Regular expressions
• Geo-temporal data
• Optimize Flux queries
• Use Flux in Chronograf dashboards

Execute Flux queries

Use the InfluxDB CLI, API, and the Chronograf Data Explorer to execute Flux queries.

Query fields and tags

Use the filter() function to query data based on fields, tags, or any other column value. filter() performs operations similar to the SELECT statement and the WHERE clause in InfluxQL and other SQL-like query languages.

from(bucket: "db/rp")
  |> range(start: -1h)
  |> filter(fn: (r) =>
      r._measurement == "example-measurement" and
      r._field == "example-field" and
      r.tag == "example-tag"
  )

Group

Use the group() function to group data with common values in specific columns.

data
  |> group(columns: ["host"], mode: "by")

Input:

Output:

Sort and limit

Use the sort()function to order records within each table by specific columns and the limit() function to limit the number of records in output tables to a fixed number, n.

data
  |> sort(columns: ["host", "_value"])
  |> limit(n: 4)

Input:

Output:

Window & aggregate

This guide walks through windowing and aggregating data with Flux and outlines how it shapes your data in the process.

data
  |> aggregateWindow(every: 20m, fn: mean)

Input:

Output:

Transform data with math

Use the map() function to remap column values and apply mathematic operations.

data
  |> map(fn: (r) => ({ r with _value: r._value * r._value }))

Input:

Output:

Calculate percentages

Use pivot() or join() and the map() function to align operand values into rows and calculate a percentage.

data
  |> pivot(rowKey:["_time"], columnKey: ["_field"], valueColumn: "_value")
  |> map(fn: (r) => ({
    _time: r._time,
    _field: "used_percent",
    _value: float(v: r.used) / float(v: r.total) * 100.0
  }))

Input:

Output:

Increase

Use the increase() function to track increases across multiple columns in a table. This function is especially useful when tracking changes in counter values that wrap over time or periodically reset.

data
  |> increase()

Input:

Output:

Moving Average

Use the movingAverage() or timedMovingAverage() functions to return the moving average of data.

data
  |> movingAverage(n: 5)

Input:

Output:

data
  |> timedMovingAverage(every: 2m, period: 4m)

Input:

Output:

Rate

Use the derivative() function to calculate the rate of change between subsequent values or the aggregate.rate() function to calculate the average rate of change per window of time. If time between points varies, these functions normalize points to a common time interval making values easily comparable.

data
  |> derivative(unit: 1m, nonNegative: true)

Input:

Output:

import "experimental/aggregate"
data
  |> aggregate.rate(every: 20m, unit: 1m)

Input:

Output:

Histograms

Use the histogram() function to create cumulative histograms with Flux.

data
  |> histogram(
    column: "_value",
    upperBoundColumn: "le",
    countColumn: "_value",
    bins: [100.0, 200.0, 300.0, 400.0],
  )

Input:

Output:

Fill

Use the fill() function to replace null values.

data
  |> fill(usePrevious: true)

Input:

Output:

data
  |> fill(value: 0.0)

Input:

Output:

Median

Use the median() function to return a value representing the 0.5 quantile (50th percentile) or median of input data.

data
  |> median()

Input:

Output:

Percentile & quantile

Use the quantile() function to return all values within the q quantile or percentile of input data.

data
  |> quantile(q: 0.99, method: "estimate_tdigest")

Input:

Output:

Join

This guide walks through joining data with Flux and outlines how it shapes your data in the process.

t1 = from(bucket: "example-bucket")
  |> range(start: 2020-01-01T00:00:00Z)
  |> filter(fn: (r) => r.m == "foo")
t2 = from(bucket: "example-bucket")
  |> range(start: 2020-01-01T00:00:00Z)
  |> filter(fn: (r) => r.m == "bar")
join(tables: {t1: t1, t2: t2}, on: ["_time"])

Input:

t1

t2

Output:

Cumulative sum

Use the cumulativeSum() function to calculate a running total of values.

data
  |> cumulativeSum()

Input:

Output:

First and last

Use the first() or last() functions to return the first or last point in an input table.

data
  |> first()

Input:

Output:

data
  |> last()

Input:

Output:

Exists

Use the Flux exists operator to check if a record contains a key or if that key’s value is null.

Filter null values

data
  |> filter(fn: (r) => exists r._value)

Extract scalar values

Use Flux stream and table functions to extract scalar values from Flux query output. This lets you, for example, dynamically set variables using query results.

scalarValue = {
  _record =
    data
      |> tableFind(fn: key => true)
      |> getRecord(idx: 0)
  return _record._value
}

Manipulate timestamps

Use Flux to process and manipulate timestamps.

Monitor states

Flux provides several functions to help monitor states and state changes in your data.

Query SQL data

The Flux sql package provides functions for working with SQL data sources. Use sql.from() to query SQL databases like PostgreSQL and MySQL

import "sql"
sql.from(
  driverName: "postgres",
  dataSourceName: "postgresql://user:password@localhost",
  query: "SELECT * FROM example_table"
)

Conditional logic

This guide describes how to use Flux conditional expressions, such as if, else, and then, to query and transform data. Flux evaluates statements from left to right and stops evaluating once a condition matches.

if color == "green" then "008000" else "ffffff"

Regular expressions

This guide walks through using regular expressions in evaluation logic in Flux functions.

data
  |> filter(fn: (r) => r.tag =~ /^foo[1-3]/)

Input:

Output:

Geo-temporal data

Use the Flux Geo package to filter geo-temporal data and group by geographic location or track.

import "experimental/geo"
sampleGeoData
  |> geo.filterRows(region: {lat: 30.04, lon: 31.23, radius: 200.0})
  |> geo.groupByArea(newColumn: "geoArea", level: 5)

Optimize Flux queries

Optimize your Flux queries to reduce their memory and compute (CPU) requirements.

Use Flux in Chronograf dashboards

This guide walks through using Flux queries in Chronograf dashboard cells, what template variables are available, and how to use them.