TimescaleDB API Reference

Hypertable management

add_dimension()

Add an additional partitioning dimension to a TimescaleDB hypertable. The column selected as the dimension can either use interval partitioning (e.g., for a second time partition) or hash partitioning.

WARNING:Theadd_dimension command can only be executed after a table has been converted to a hypertable (viacreate_hypertable), but must similarly be run only on an empty hypertable.

Space partitions: Using space partitions is highly recommended fordistributed hypertables to achieve efficient scale-out performance. Forregular hypertables that exist only on a single node, additional partitioning can be used for specialized use cases and not recommended for most users.

Space partitions use hashing: Every distinct item is hashed to one ofN buckets. Remember that we are already using (flexible) time intervals to manage chunk sizes; the main purpose of space partitioning is to enable parallelization across multiple data nodes (in the case of distributed hypertables) or across multiple disks within the same time interval (in the case of single-node deployments).

Parallelizing queries across multiple data nodes

In a distributed hypertable, space partitioning enables inserts to be parallelized across data nodes, even while the inserted rows share timestamps from the same time interval, and thus increases the ingest rate. Query performance also benefits by being able to parallelize queries across nodes, particularly when full or partial aggregations can be “pushed down” to data nodes (e.g., as in the queryavg(temperature) FROM conditions GROUP BY hour, location when usinglocation as a space partition). Please see ourbest practices about partitioning in distributed hypertables for more information.

Parallelizing disk I/O on a single node

Parallel I/O can benefit in two scenarios: (a) two or more concurrent queries should be able to read from different disks in parallel, or (b) a single query should be able to use query parallelization to read from multiple disks in parallel.

Thus, users looking for parallel I/O have two options:

1. Use a RAID setup across multiple physical disks, and expose a single logical disk to the hypertable (i.e., via a single tablespace).

2. For each physical disk, add a separate tablespace to the database. TimescaleDB allows you to actually add multiple tablespaces to asingle hypertable (although under the covers, a hypertable’s chunks are spread across the tablespaces associated with that hypertable).

We recommend a RAID setup when possible, as it supports both forms of parallelization described above (i.e., separate queries to separate disks, single query to multiple disks in parallel). The multiple tablespace approach only supports the former. With a RAID setup,no spatial partitioning is required.

That said, when using space partitions, we recommend using 1 space partition per disk.

TimescaleDB doesnot benefit from a very large number of space partitions (such as the number of unique items you expect in partition field). A very large number of such partitions leads both to poorer per-partition load balancing (the mapping of items to partitions using hashing), as well as much increased planning latency for some types of queries.

Required Arguments

Optional Arguments

Returns

When executing this function, eithernumber_partitions orchunk_time_interval must be supplied, which will dictate if the dimension will use hash or interval partitioning.

Thechunk_time_interval should be specified as follows:

• If the column to be partitioned is a TIMESTAMP, TIMESTAMPTZ, or DATE, this length should be specified either as an INTERVAL type or an integer value inmicroseconds.

• If the column is some other integer type, this length should be an integer that reflects the column’s underlying semantics (e.g., thechunk_time_interval should be given in milliseconds if this column is the number of milliseconds since the UNIX epoch).

WARNING:Supporting more thanone additional dimension is currently experimental. For any production environments, users are recommended to use at most one “space” dimension.

Sample Usage

First convert tableconditions to hypertable with just time partitioning on columntime, then add an additional partition key onlocation with four partitions:

SELECT create_hypertable('conditions', 'time');
SELECT add_dimension('conditions', 'location', number_partitions => 4);

Convert tableconditions to hypertable with time partitioning ontime and space partitioning (2 partitions) onlocation, then add two additional dimensions.

SELECT create_hypertable('conditions', 'time', 'location', 2);
SELECT add_dimension('conditions', 'time_received', chunk_time_interval => INTERVAL '1 day');
SELECT add_dimension('conditions', 'device_id', number_partitions => 2);
SELECT add_dimension('conditions', 'device_id', number_partitions => 2, if_not_exists => true);

Now in a multi-node example for distributed hypertables with a cluster of one access node and two data nodes, configure the access node for access to the two data nodes. Then, convert tableconditions to a distributed hypertable with just time partitioning on columntime, and finally add a space partitioning dimension onlocation with two partitions (as the number of the attached data nodes).

SELECT add_data_node('dn1', host => 'dn1.example.com');
SELECT add_data_node('dn2', host => 'dn2.example.com');
SELECT create_distributed_hypertable('conditions', 'time');
SELECT add_dimension('conditions', 'location', number_partitions => 2);

attach_data_node()

Attach a data node to a hypertable. The data node should have been previously created usingadd_data_node.

When a distributed hypertable is created it will by default use all available data nodes for the hypertable, but if a data node is addedafter a hypertable is created, the data node will not automatically be used by existing distributed hypertables.

If you want a hypertable to use a data node that was created later, you must attach the data node to the hypertable using this function.

Required Arguments

Optional Arguments

Returns

Sample Usage

Attach a data nodedn3 to a distributed hypertableconditions previously created withcreate_distributed_hypertable.

SELECT * FROM attach_data_node('dn3','conditions');
hypertable_id | node_hypertable_id | node_name
--------------+--------------------+-------------
            5 | 3 | dn3
(1 row)

TIP:You must add a data node to your distributed database first withadd_data_node first before attaching it.

add_data_node()

Add a new data node on the access node to be used by distributed hypertables. The data node will automatically be used by distributed hypertables that are created after the data node has been added, while existing distributed hypertables require an additionalattach_data_node.

If the data node already exists, the command will abort with either an error or a notice depending on the value ofif_not_exists.

For security purposes, only superusers or users with necessary privileges can add data nodes (see below for details). When adding a data node, the access node will also try to connect to the data node and therefore needs a way to authenticate with it. TimescaleDB currently supports several different such authentication methods for flexibility (including trust, user mappings, password, and certificate methods). Please refer toSetting up Multi-Node TimescaleDB for more information about node-to-node authentication.

Unlessbootstrap is false, the function will attempt to bootstrap the data node by:

1. Creating the database given indatabase that will serve as the new data node.
2. Loading the TimescaleDB extension in the new database.
3. Setting metadata to make the data node part of the distributed database.

Note that user roles are not automatically created on the new data node during bootstrapping. Thedistributed_exec procedure can be used to create additional roles on the data node after it is added.

Required Arguments

Optional Arguments

Returns

Errors

An error will be given if:

• The function is executed inside a transaction.
• The function is executed in a database that is already a data node.
• The data node already exists andif_not_exists isFALSE.
• The access node cannot connect to the data node due to a network failure or invalid configuration (e.g., wrong port, or there is no way to authenticate the user).
• Ifbootstrap isFALSE and the database was not previously bootstrapped.

Privileges

To add a data node, you must be a superuser or have theUSAGE privilege on thetimescaledb_fdw foreign data wrapper. To grant such privileges to a regular user role, do:

GRANT USAGE ON FOREIGN DATA WRAPPER timescaledb_fdw TO <newrole>;

Note, however, that superuser privileges might still be necessary on the data node in order to bootstrap it, including creating the TimescaleDB extension on the data node unless it is already installed.

Sample usage

Let’s assume that you have an existing hypertableconditions and want to usetime as the time partitioning column andlocation as the space partitioning column. You also want to distribute the chunks of the hypertable on two data nodesdn1.example.com anddn2.example.com:

SELECT add_data_node('dn1', host => 'dn1.example.com');
SELECT add_data_node('dn2', host => 'dn2.example.com');
SELECT create_distributed_hypertable('conditions', 'time', 'location');

If you want to create a distributed database with the two data nodes local to this instance, you can write:

SELECT add_data_node('dn1', host => 'localhost', database => 'dn1');
SELECT add_data_node('dn2', host => 'localhost', database => 'dn2');
SELECT create_distributed_hypertable('conditions', 'time', 'location');

Note that this does not offer any performance advantages over using a regular hypertable, but it can be useful for testing.

attach_tablespace()

Attach a tablespace to a hypertable and use it to store chunks. Atablespace is a directory on the filesystem that allows control over where individual tables and indexes are stored on the filesystem. A common use case is to create a tablespace for a particular storage disk, allowing tables to be stored there. Please review the standard PostgreSQL documentation for moreinformation on tablespaces.

TimescaleDB can manage a set of tablespaces for each hypertable, automatically spreading chunks across the set of tablespaces attached to a hypertable. If a hypertable is hash partitioned, TimescaleDB will try to place chunks that belong to the same partition in the same tablespace. Changing the set of tablespaces attached to a hypertable may also change the placement behavior. A hypertable with no attached tablespaces will have its chunks placed in the database’s default tablespace.

Required Arguments

Tablespaces need to becreated before being attached to a hypertable. Once created, tablespaces can be attached to multiple hypertables simultaneously to share the underlying disk storage. Associating a regular table with a tablespace using theTABLESPACE option toCREATE TABLE, prior to callingcreate_hypertable, will have the same effect as callingattach_tablespace immediately followingcreate_hypertable.

Optional Arguments

Sample Usage

Attach the tablespacedisk1 to the hypertableconditions:

SELECT attach_tablespace('disk1', 'conditions');
SELECT attach_tablespace('disk2', 'conditions', if_not_attached => true);

WARNING:The management of tablespaces on hypertables is currently an experimental feature.

create_hypertable()

Creates a TimescaleDB hypertable from a PostgreSQL table (replacing the latter), partitioned on time and with the option to partition on one or more other columns (i.e., space). All actions, such asALTER TABLE,SELECT, etc., still work on the resulting hypertable.

Required Arguments

Optional Arguments

Returns

TIP:If you useSELECT * FROM create_hypertable(...) you will get the return value formatted as a table with column headings.

WARNING:The use of themigrate_data argument to convert a non-empty table can lock the table for a significant amount of time, depending on how much data is in the table. It can also run into deadlock if foreign key constraints exist to other tables.

If you would like finer control over index formation and other aspects of your hypertable,follow these migration instructions instead.

When converting a normal SQL table to a hypertable, pay attention to how you handle constraints. A hypertable can contain foreign keys to normal SQL table columns, but the reverse is not allowed. UNIQUE and PRIMARY constraints must include the partitioning key.

The deadlock is likely to happen when concurrent transactions simultaneously try to insert data into tables that are referenced in the foreign key constraints and into the converting table itself. The deadlock can be prevented by manually obtainingSHARE ROW EXCLUSIVE lock on the referenced tables before callingcreate_hypertable in the same transaction, seePostgreSQL documentation for the syntax.

Units

The ‘time’ column supports the following data types:

TIP:The type flexibility of the ‘time’ column allows the use of non-time-based values as the primary chunk partitioning column, as long as those values can increment.

TIP:For incompatible data types (e.g.jsonb) you can specify a function to thetime_partitioning_func argument which can extract a compatible data type

The units ofchunk_time_interval should be set as follows:

• For time columns having timestamp or DATE types, thechunk_time_interval should be specified either as aninterval type or an integral value inmicroseconds.

• For integer types, thechunk_time_intervalmust be set explicitly, as the database does not otherwise understand the semantics of what each integer value represents (a second, millisecond, nanosecond, etc.). So if your time column is the number of milliseconds since the UNIX epoch, and you wish to each chunk to cover 1 day, you should specifychunk_time_interval => 86400000.

In case of hash partitioning (i.e.,number_partitions is greater than zero), it is possible to optionally specify a custom partitioning function. If no custom partitioning function is specified, the default partitioning function is used. The default partitioning function calls PostgreSQL’s internal hash function for the given type, if one exists. Thus, a custom partitioning function can be used for value types that do not have a native PostgreSQL hash function. A partitioning function should take a singleanyelement type argument and return a positiveinteger hash value. Note that this hash value isnot a partition ID, but rather the inserted value’s position in the dimension’s key space, which is then divided across the partitions.

TIP:The time column increate_hypertable must be defined asNOT NULL. If this is not already specified on table creation,create_hypertable will automatically add this constraint on the table when it is executed.

Sample Usage

Convert tableconditions to hypertable with just time partitioning on columntime:

SELECT create_hypertable('conditions', 'time');

Convert tableconditions to hypertable, settingchunk_time_interval to 24 hours.

SELECT create_hypertable('conditions', 'time', chunk_time_interval => 86400000000);
SELECT create_hypertable('conditions', 'time', chunk_time_interval => INTERVAL '1 day');

Convert tableconditions to hypertable with time partitioning ontime and space partitioning (4 partitions) onlocation:

SELECT create_hypertable('conditions', 'time', 'location', 4);

The same as above, but using a custom partitioning function:

SELECT create_hypertable('conditions', 'time', 'location', 4, partitioning_func => 'location_hash');

Convert tableconditions to hypertable. Do not raise a warning ifconditions is already a hypertable:

SELECT create_hypertable('conditions', 'time', if_not_exists => TRUE);

Time partition tablemeasurements on a composite column typereport using a time partitioning function: Requires an immutable function that can convert the column value into a supported column value:

CREATE TYPE report AS (reported timestamp with time zone, contents jsonb);
CREATE FUNCTION report_reported(report)
  RETURNS timestamptz
  LANGUAGE SQL
  IMMUTABLE AS
  'SELECT $1.reported';
SELECT create_hypertable('measurements', 'report', time_partitioning_func => 'report_reported');

Time partition tableevents, on a column typejsonb (event), which has a top level key (started) containing an ISO 8601 formatted timestamp:

CREATE FUNCTION event_started(jsonb)
  RETURNS timestamptz
  LANGUAGE SQL
  IMMUTABLE AS
  $func$SELECT ($1->>'started')::timestamptz$func$;
SELECT create_hypertable('events', 'event', time_partitioning_func => 'event_started');

Best Practices

One of the most common questions users of TimescaleDB have revolves around configuringchunk_time_interval.

Time intervals: The current release of TimescaleDB enables both the manual and automated adaption of its time intervals. With manually-set intervals, users should specify achunk_time_interval when creating their hypertable (the default value is 1 week). The interval used for new chunks can be changed by callingset_chunk_time_interval().

The key property of choosing the time interval is that the chunk (including indexes) belonging to the most recent interval (or chunks if using space partitions) fit into memory. As such, we typically recommend setting the interval so that these chunk(s) comprise no more than 25% of main memory.

TIP:Make sure that you are planning for recent chunks fromall active hypertables to fit into 25% of main memory, rather than 25% per hypertable.

To determine this, you roughly need to understand your data rate. If you are writing roughly 2GB of data per day and have 64GB of memory, setting the time interval to a week would be good. If you are writing 10GB per day on the same machine, setting the time interval to a day would be appropriate. This interval would also hold if data is loaded more in batches, e.g., you bulk load 70GB of data per week, with data corresponding to records from throughout the week.

While it’s generally safer to make chunks smaller rather than too large, setting intervals too small can lead tomany chunks, which corresponds to increased planning latency for some types of queries.

TIP:One caveat is that the total chunk size is actually dependent on both the underlying data sizeand any indexes, so some care might be taken if you make heavy use of expensive index types (e.g., some PostGIS geospatial indexes). During testing, you might check your total chunk sizes via thechunks_detailed_size function.

Space partitions: In most cases, it is advised for users not to use space partitions. However, if you create a distributed hypertable, it is important to create space partitioning, seecreate_distributed_hypertable. The rare cases in which space partitions may be useful for non-distributed hypertables are described in theadd_dimension section.

create_distributed_hypertable()

Creates a TimescaleDB hypertable distributed across a multinode environment. Use this function in place ofcreate_hypertable when creating distributed hypertables.

Required Arguments

Optional Arguments

Returns

Sample Usage

Create a tableconditions which will be partitioned across data nodes by the ‘location’ column. Note that the number of space partitions is automatically equal to the number of data nodes assigned to this hypertable (all configured data nodes in this case, asdata_nodes is not specified).

SELECT create_distributed_hypertable('conditions', 'time', 'location');

Create a tableconditions using a specific set of data nodes.

SELECT create_distributed_hypertable('conditions', 'time', 'location',
    data_nodes => '{ "data_node_1", "data_node_2", "data_node_4", "data_node_7" }');

Best Practices

Space partitions: As opposed to the normalcreate_hypertable best practices, space partitions are highly recommended for distributed hypertables. Incoming data will be divided among data nodes based upon the space partition (the first one if multiple space partitions have been defined). If there is no space partition, all the data for each time slice will be written to a single data node.

Time intervals: Follow the same guideline in setting thechunk_time_interval as withcreate_hypertable, bearing in mind that the calculation needs to be based on the memory capacity of the data nodes. However, one additional thing to consider, assuming space partitioning is being used, is that the hypertable will be evenly distributed across the data nodes, allowing a larger time interval.

For example, assume you are ingesting 10GB of data per day and you have five data nodes, each with 64GB of memory. If this is the only table being served by these data nodes, then you should use a time interval of 1 week (7 * 10GB / 5 * 64GB ~= 22% main memory used for most recent chunks).

If space partitioning is not being used, thechunk_time_interval should be the same as the non-distributed case, as all of the incoming data will be handled by a single node.

Replication factor: The hypertable’sreplication_factor defines to how many data nodes a newly created chunk will be replicated. That is, a chunk with areplication_factor of three will exist on three separate data nodes, and rows written to that chunk will be inserted (as part of a two-phase commit protocol) to all three chunk copies. For chunks replicated more than once, if a data node fails or is removed, no data will be lost, and writes can continue to succeed on the remaining chunk copies. However, the chunks present on the lost data node will now be under-replicated. Currently, it is not possible to restore under-replicated chunks, although this limitation will be removed in a future release. To avoid such inconsistency, we do not yet recommend usingreplication_factor > 1, and instead rely on physical replication of each data node if such fault-tolerance is required.

CREATE INDEX (Transaction Per Chunk)

CREATE INDEX ... WITH (timescaledb.transaction_per_chunk, ...);

This option extendsCREATE INDEX with the ability to use a separate transaction for each chunk it creates an index on, instead of using a single transaction for the entire hypertable. This allowsINSERTs, and other operations to to be performed concurrently during most of the duration of theCREATE INDEX command. While the index is being created on an individual chunk, it functions as if a regularCREATE INDEX were called on that chunk, however other chunks are completely un-blocked.

TIP:This version ofCREATE INDEX can be used as an alternative toCREATE INDEX CONCURRENTLY, which is not currently supported on hypertables.

WARNING:If the operation fails partway through, indexes may not be created on all hypertable chunks. If this occurs, the index on the root table of the hypertable will be marked as invalid (this can be seen by running\d+ on the hypertable). The index will still work, and will be created on new chunks, but if you wish to ensureall chunks have a copy of the index, drop and recreate it.

Sample Usage

Anonymous index

CREATE INDEX ON conditions(time, device_id) WITH (timescaledb.transaction_per_chunk);

Other index methods

CREATE INDEX ON conditions(time, location) USING brin
  WITH (timescaledb.transaction_per_chunk);

delete_data_node()

This function will remove the data node locally. This willnot affect the remote database in any way, it will just update the local index over all existing data nodes.

The data node will be detached from all hypertables that are using it if permissions and data integrity requirements are satisfied. For more information, seedetach_data_node.

Errors

An error will be generated if the data node cannot be detached from all attached hypertables.

Required Arguments

Optional Arguments

Returns

A boolean indicating if the operation was successful or not.

Sample usage

To delete a data node nameddn1:

SELECT delete_data_node('dn1');

detach_data_node()

Detach a data node from one hypertable or from all hypertables.

Reasons for detaching a data node:

• A data node should no longer be used by a hypertable and needs to be removed from all hypertables that use it
• You want to have fewer data nodes for a distributed hypertable to partition across

Required Arguments

Optional Arguments

Returns

The number of hypertables the data node was detached from.

Errors

Detaching a node is not permitted:

• If it would result in data loss for the hypertable due to the data node containing chunks that are not replicated on other data nodes
• If it would result in under-replicated chunks for the distributed hypertable (without theforce argument)

TIP:Replication is currently experimental, and not a supported feature

Detaching a data node is under no circumstances possible if that would mean data loss for the hypertable. Nor is it possible to detach a data node, unless forced, if that would mean that the distributed hypertable would end up with under-replicated chunks.

The only safe way to detach a data node is to first safely delete any data on it or replicate it to another data node.

Sample Usage

Detach data nodedn3 fromconditions:

SELECT detach_data_node('dn3', 'conditions');

detach_tablespace()

Detach a tablespace from one or more hypertables. Thisonly means thatnew chunks will not be placed on the detached tablespace. This is useful, for instance, when a tablespace is running low on disk space and one would like to prevent new chunks from being created in the tablespace. The detached tablespace itself and any existing chunks with data on it will remain unchanged and will continue to work as before, including being available for queries. Note that newly inserted data rows may still be inserted into an existing chunk on the detached tablespace since existing data is not cleared from a detached tablespace. A detached tablespace can be reattached if desired to once again be considered for chunk placement.

Required Arguments

When giving only the tablespace name as argument, the given tablespace will be detached from all hypertables that the current role has the appropriate permissions for. Therefore, without proper permissions, the tablespace may still receive new chunks after this command is issued.

Optional Arguments

When specifying a specific hypertable, the tablespace will only be detached from the given hypertable and thus may remain attached to other hypertables.

Sample Usage

Detach the tablespacedisk1 from the hypertableconditions:

SELECT detach_tablespace('disk1', 'conditions');
SELECT detach_tablespace('disk2', 'conditions', if_attached => true);

Detach the tablespacedisk1 from all hypertables that the current user has permissions for:

SELECT detach_tablespace('disk1');

detach_tablespaces()

Detach all tablespaces from a hypertable. After issuing this command on a hypertable, it will no longer have any tablespaces attached to it. New chunks will instead be placed in the database’s default tablespace.

Required Arguments

Sample Usage

Detach all tablespaces from the hypertableconditions:

SELECT detach_tablespaces('conditions');

distributed_exec()

This procedure is used on an access node to execute a SQL command across the data nodes of a distributed database. For instance, one use case is to create the roles and permissions needed in a distributed database.

The procedure can run distributed commands transactionally, so a command is executed either everywhere or nowhere. However, not all SQL commands can run in a transaction. This can be toggled with the argumenttransactional. Note if the execution is not transactional, a failure on one of the data node will require manual dealing with any introduced inconsistency.

Note that the command isnot executed on the access node itself and it is not possible to chain multiple commands together in one call.

Required Arguments

Optional Arguments

Sample Usage

Create the roletestrole across all data nodes in a distributed database:

CALL distributed_exec($$ CREATE USER testrole WITH LOGIN $$);

Create the roletestrole on two specific data nodes:

CALL distributed_exec($$ CREATE USER testrole WITH LOGIN $$, node_list => '{ "dn1", "dn2" }');

Create new databasesdist_database on data nodes, which requires to settransactional to FALSE:

CALL distributed_exec('CREATE DATABASE dist_database', transactional => FALSE);

drop_chunks()

Removes data chunks whose time range falls completely before (or after) a specified time. Shows a list of the chunks that were dropped, in the same style as theshow_chunksfunction.

Chunks are constrained by a start and end time and the start time is always before the end time. A chunk is dropped if its end time is older than theolder_than timestamp or, ifnewer_than is given, its start time is newer than thenewer_than timestamp.

Note that, because chunks are removed if and only if their time range falls fully before (or after) the specified timestamp, the remaining data may still contain timestamps that are before (or after) the specified one.

Required Arguments

Optional Arguments

Theolder_than andnewer_than parameters can be specified in two ways:

• interval type: The cut-off point is computed asnow() - older_than and similarlynow() - newer_than. An error will be returned if an INTERVAL is supplied and the time column is not one of aTIMESTAMP,TIMESTAMPTZ, orDATE.

• timestamp, date, or integer type: The cut-off point is explicitly given as aTIMESTAMP /TIMESTAMPTZ /DATE or as aSMALLINT /INT /BIGINT. The choice of timestamp or integer must follow the type of the hypertable’s time column.

WARNING:When using just an interval type, the function assumes that you are are removing thingsin the past. If you want to remove data in the future (i.e., erroneous entries), use a timestamp.

When both arguments are used, the function returns the intersection of the resulting two ranges. For example, specifyingnewer_than => 4 months andolder_than => 3 months will drop all full chunks that are between 3 and 4 months old. Similarly, specifyingnewer_than => '2017-01-01' andolder_than => '2017-02-01' will drop all full chunks between ‘2017-01-01’ and ‘2017-02-01’. Specifying parameters that do not result in an overlapping intersection between two ranges will result in an error.

Sample Usage

Drop all chunks from hypertableconditions older than 3 months:

SELECT drop_chunks('conditions', INTERVAL '3 months');

Example output:

 drop_chunks
----------------------------------------
 _timescaledb_internal._hyper_3_5_chunk
 _timescaledb_internal._hyper_3_6_chunk
 _timescaledb_internal._hyper_3_7_chunk
 _timescaledb_internal._hyper_3_8_chunk
 _timescaledb_internal._hyper_3_9_chunk
(5 rows)

Drop all chunks more than 3 months in the future from hypertableconditions. This is useful for correcting data ingested with incorrect clocks:

SELECT drop_chunks('conditions', newer_than => now() + interval '3 months');

Drop all chunks from hypertableconditions before 2017:

SELECT drop_chunks('conditions', '2017-01-01'::date);

Drop all chunks from hypertableconditions before 2017, where time column is given in milliseconds from the UNIX epoch:

SELECT drop_chunks('conditions', 1483228800000);

Drop all chunks older than 3 months ago and newer than 4 months ago from hypertableconditions:

SELECT drop_chunks('conditions', older_than => interval '3 months', newer_than => interval '4 months')

set_chunk_time_interval()

Sets the chunk_time_interval on a hypertable. The new interval is used when new chunks are created but the time intervals on existing chunks are not affected.

Required Arguments

Optional Arguments

The valid types for thechunk_time_interval depend on the type of hypertable time column:

• TIMESTAMP, TIMESTAMPTZ, DATE: The specifiedchunk_time_interval should be given either as an INTERVAL type (INTERVAL '1 day') or as an integer or bigint value (representing some number of microseconds).

• INTEGER: The specifiedchunk_time_interval should be an integer (smallint, int, bigint) value and represent the underlying semantics of the hypertable’s time column, e.g., given in milliseconds if the time column is expressed in milliseconds (seecreate_hypertableinstructions).

Sample Usage

For a TIMESTAMP column, setchunk_time_interval to 24 hours.

SELECT set_chunk_time_interval('conditions', INTERVAL '24 hours');
SELECT set_chunk_time_interval('conditions', 86400000000);

For a time column expressed as the number of milliseconds since the UNIX epoch, setchunk_time_interval to 24 hours.

SELECT set_chunk_time_interval('conditions', 86400000);

set_number_partitions()

Sets the number of partitions (slices) of a space dimension on a hypertable. The new partitioning only affects new chunks.

Required Arguments

Optional Arguments

Thedimension_name needs to be explicitly specified only if the hypertable has more than one space dimension. An error will be thrown otherwise.

Sample Usage

For a table with a single space dimension:

SELECT set_number_partitions('conditions', 2);

For a table with more than one space dimension:

SELECT set_number_partitions('conditions', 2, 'device_id');

set_integer_now_func()

This function is only relevant for hypertables with integer (as opposed to TIMESTAMP/TIMESTAMPTZ/DATE) time values. For such hypertables, it sets a function that returns thenow() value (current time) in the units of the time column. This is necessary for running some policies on integer-based tables. In particular, many policies only apply to chunks of a certain age and a function that returns the current time is necessary to determine the age of a chunk.

Required Arguments

Optional Arguments

Sample Usage

To set the integer now function for a hypertable with a time column in unix time (number of seconds since the unix epoch, UTC).

CREATE OR REPLACE FUNCTION unix_now() returns BIGINT LANGUAGE SQL STABLE as $$ SELECT extract(epoch from now())::BIGINT $$;
SELECT set_integer_now_func('test_table_bigint', 'unix_now');

set_replication_factor()

Sets the replication factor of a distributed hypertable to the given value. Changing the replication factor does not affect the number of replicas for existing chunks. Chunks created after changing the replication factor will be replicated in accordance with new value of the replication factor. If the replication factor cannot be satisfied, since the amount of attached data nodes is less than new replication factor, the command aborts with an error.

If existing chunks have less replicas than new value of the replication factor, the function will print a warning.

Required Arguments

Errors

An error will be given if:

• hypertable is not a distributed hypertable.
• replication_factor is less than1, which cannot be set on a distributed hypertable.
• replication_factor is bigger than the number of attached data nodes.

If a bigger replication factor is desired, it is necessary to attach more data nodes by usingattach_data_node.

Sample Usage

Update the replication factor for a distributed hypertable to2:

SELECT set_replication_factor('conditions', 2);

Example of the warning if any existing chunk of the distributed hypertable has less than 2 replicas:

WARNING: hypertable "conditions" is under-replicated
DETAIL: Some chunks have less than 2 replicas.

Example of providing too big of a replication factor for a hypertable with 2 attached data nodes:

SELECT set_replication_factor('conditions', 3);
ERROR: too big replication factor for hypertable "conditions"
DETAIL: The hypertable has 2 data nodes attached, while the replication factor is 3.
HINT: Decrease the replication factor or attach more data nodes to the hypertable.

show_chunks()

Get list of chunks associated with hypertables.

Optional Arguments

Function accepts the following arguments. These arguments have the same semantics as thedrop_chunksfunction.

Theolder_than andnewer_than parameters can be specified in two ways:

• interval type: The cut-off point is computed asnow() - older_than and similarlynow() - newer_than. An error will be returned if an INTERVAL is supplied and the time column is not one of a TIMESTAMP, TIMESTAMPTZ, or DATE.

• timestamp, date, or integer type: The cut-off point is explicitly given as a TIMESTAMP / TIMESTAMPTZ / DATE or as a SMALLINT / INT / BIGINT. The choice of timestamp or integer must follow the type of the hypertable’s time column.

When both arguments are used, the function returns the intersection of the resulting two ranges. For example, specifyingnewer_than => 4 months andolder_than => 3 months will show all full chunks that are between 3 and 4 months old. Similarly, specifyingnewer_than => '2017-01-01' andolder_than => '2017-02-01' will show all full chunks between ‘2017-01-01’ and ‘2017-02-01’. Specifying parameters that do not result in an overlapping intersection between two ranges will result in an error.

Sample Usage

Get list of all chunks. Returns 0 if there are no hypertables:

SELECT show_chunks();

The expected output:

 show_chunks
---------------------------------------
 _timescaledb_internal._hyper_1_10_chunk
 _timescaledb_internal._hyper_1_11_chunk
 _timescaledb_internal._hyper_1_12_chunk
 _timescaledb_internal._hyper_1_13_chunk
 _timescaledb_internal._hyper_1_14_chunk
 _timescaledb_internal._hyper_1_15_chunk
 _timescaledb_internal._hyper_1_16_chunk
 _timescaledb_internal._hyper_1_17_chunk
 _timescaledb_internal._hyper_1_18_chunk

Get list of all chunks associated with a table:

SELECT show_chunks('conditions');

Get all chunks older than 3 months:

SELECT show_chunks(older_than => INTERVAL '3 months');

Get all chunks more than 3 months in the future. This is useful for showing data ingested with incorrect clocks:

SELECT show_chunks(newer_than => now() + INTERVAL '3 months');

Get all chunks from hypertableconditions older than 3 months:

SELECT show_chunks('conditions', older_than => INTERVAL '3 months');

Get all chunks from hypertableconditions before 2017:

SELECT show_chunks('conditions', older_than => DATE '2017-01-01');

Get all chunks newer than 3 months:

SELECT show_chunks(newer_than => INTERVAL '3 months');

Get all chunks older than 3 months and newer than 4 months:

SELECT show_chunks(older_than => INTERVAL '3 months', newer_than => INTERVAL '4 months');

reorder_chunk() Community

Reorder a single chunk’s heap to follow the order of an index. This function acts similarly to thePostgreSQL CLUSTER command , however it uses lower lock levels so that, unlike with the CLUSTER command, the chunk and hypertable are able to be read for most of the process. It does use a bit more disk space during the operation.

This command can be particularly useful when data is often queried in an order different from that in which it was originally inserted. For example, data is commonly inserted into a hypertable in loose time order (e.g., many devices concurrently sending their current state), but one might typically query the hypertable about aspecific device. In such cases, reordering a chunk using an index on(device_id, time) can lead to significant performance improvement for these types of queries.

One can call this function directly on individual chunks of a hypertable, but usingadd_reorder_policy is often much more convenient.

Required Arguments

Optional Arguments

Returns

This function returns void.

Sample Usage

SELECT reorder_chunk('_timescaledb_internal._hyper_1_10_chunk', 'conditions_device_id_time_idx');

runs a reorder on the_timescaledb_internal._hyper_1_10_chunk chunk using theconditions_device_id_time_idx index.

move_chunk() Community

TimescaleDB allows users to move data (and indexes) to alternative tablespaces. This allows the user the ability to move data to more cost effective storage as it ages. This function acts like the combination of thePostgreSQL CLUSTER command and thePostgreSQL ALTER TABLE…SET TABLESPACE command.

Unlike these PostgreSQL commands, however, themove_chunk function employs lower lock levels so that the chunk and hypertable are able to be read for most of the process. This comes at a cost of slightly higher disk usage during the operation. For a more detailed discussion of this capability, please see theData Tiering documentation.

Required Arguments

Optional Arguments

Sample Usage

SELECT move_chunk(
  chunk => '_timescaledb_internal._hyper_1_4_chunk',
  destination_tablespace => 'tablespace_2',
  index_destination_tablespace => 'tablespace_3',
  reorder_index => 'conditions_device_id_time_idx',
  verbose => TRUE
);

Compression Community

We highly recommend reading theblog post andtutorial about compression before trying to set it up for the first time.

Setting up compression on TimescaleDB requires users to firstconfigure the hypertable for compression and thenset up a policy for when to compress chunks.

Advanced usage of compression allows users tocompress chunks manually, instead of automatically as they age.

Restrictions

The current version does not support altering or inserting data into compressed chunks. The data can be queried without any modifications, however if you need to backfill or update data in a compressed chunk you will need to decompress the chunk(s) first.

Associated commands

• ALTER TABLE
• add_compression_policy
• remove_compression_policy
• compress_chunk
• decompress_chunk

ALTER TABLE (Compression) Community

‘ALTER TABLE’ statement is used to turn on compression and set compression options.

The syntax is:

ALTER TABLE <table_name> SET (timescaledb.compress, timescaledb.compress_orderby = '<column_name> [ASC | DESC] [ NULLS { FIRST | LAST } ] [, ...]',
timescaledb.compress_segmentby = '<column_name> [, ...]'
);

Required Options

Other Options

Parameters

Sample Usage

Configure a hypertable that ingests device data to use compression.

ALTER TABLE metrics SET (timescaledb.compress, timescaledb.compress_orderby = 'time DESC', timescaledb.compress_segmentby = 'device_id');

add_compression_policy() Community

Allows you to set a policy by which the system will compress a chunk automatically in the background after it reaches a given age.

Note that compression policies can only be created on hypertables that already have compression enabled, e.g., via theALTER TABLE command to settimescaledb.compress and other configuration parameters.

Required Arguments

Thecompress_after parameter should be specified differently depending on the type of the time column of the hypertable:

• For hypertables with TIMESTAMP, TIMESTAMPTZ, and DATE time columns: the time interval should be an INTERVAL type.
• For hypertables with integer-based timestamps: the time interval should be an integer type (this requires theinteger_now_func to be set).

Optional Arguments

Sample Usage

Add a policy to compress chunks older than 60 days on the ‘cpu’ hypertable.

SELECT add_compression_policy('cpu', INTERVAL '60d');

Add a compress chunks policy to a hypertable with an integer-based time column:

SELECT add_compression_policy('table_with_bigint_time', BIGINT '600000');

remove_compression_policy() Community

If you need to remove the compression policy. To re-start policy-based compression again you will need to re-add the policy.

Required Arguments

Optional Arguments

Sample Usage

Remove the compression policy from the ‘cpu’ table:

SELECT remove_compression_policy('cpu');

compress_chunk() Community

The compress_chunk function is used to compress a specific chunk. This is most often used instead of theadd_compression_policy function, when a user wants more control over the scheduling of compression. For most users, we suggest using the policy framework instead.

TIP:You can get a list of chunks belonging to a hypertable using theshow_chunksfunction.

Required Arguments

Optional Arguments

Sample Usage

Compress a single chunk.

SELECT compress_chunk('_timescaledb_internal._hyper_1_2_chunk');

decompress_chunk() Community

If you need to modify or add data to a chunk that has already been compressed, you will need to decompress the chunk first. This is especially useful for backfilling old data.

TIP:Prior to decompressing chunks for the purpose of data backfill or updating you should first stop any compression policy that is active on the hypertable you plan to perform this operation on. Once the update and/or backfill is complete simply turn the policy back on and the system will recompress your chunks.

Required Arguments

Optional Arguments

Sample Usage

Decompress a single chunk

SELECT decompress_chunk('_timescaledb_internal._hyper_2_2_chunk');

Continuous Aggregates Community

TimescaleDB allows users the ability to automatically recompute aggregates at predefined intervals and materialize the results. This is suitable for frequently used queries. For a more detailed discussion of this capability, please seeusing TimescaleDB Continuous Aggregates.

• CREATE MATERIALIZED VIEW
• ALTER MATERIALIZED VIEW
• DROP MATERIALIZED VIEW
• refresh_continuous_aggregate()

CREATE MATERIALIZED VIEW (Continuous Aggregate) Community

CREATE MATERIALIZED VIEW statement is used to create continuous aggregates.

The syntax is:

CREATE MATERIALIZED VIEW <view_name> [ ( column_name [, ...] ) ]
  WITH ( timescaledb.continuous [, timescaledb.<option> = <value> ] )
  AS
    <select_query>
  [WITH [NO] DATA]

<select_query> is of the form :

SELECT <grouping_exprs>, <aggregate_functions>
    FROM <hypertable>
[WHERE ... ]
GROUP BY time_bucket( <const_value>, <partition_col_of_hypertable> ),
         [ optional grouping exprs>]
[HAVING ...]

Note that continuous aggregates have some limitations of what types of queries they can support, described in more length below. For example, theFROM clause must provide only one hypertable, i.e., no joins, CTEs, views or subqueries are supported. TheGROUP BY clause must include a time bucket on the hypertable’s time column, and all aggregates must be parallelizable.

Parameters

RequiredWITH clause options

OptionalWITH clause options

Notes

• The view will be automatically refreshed (as outlined underrefresh_continuous_aggregate) unlessWITH NO DATA is given (WITH DATA is the default).
• TheSELECT query should be of the form specified in the syntax above, which is discussed in the following items.
• Only a single hypertable can be specified in theFROM clause of theSELECT query. This means that including more hypertables, joins, tables, views, subqueries is not supported.
• The hypertable used in theSELECT may not haverow-level-security policies enabled.
• TheGROUP BY clause must include a time_bucket expression. Thetime_bucket expression must use the time dimension column of the hypertable.
• time_bucket_gapfill is not allowed in continuous aggs, but may be run in aSELECT from the continuous aggregate view.
• In general, aggregates which can beparallelized by PostgreSQL are allowed in the view definition, this includes most aggregates distributed with PostgreSQL. Aggregates withORDER BY,DISTINCT andFILTER clauses are not permitted.
• All functions and their arguments included inSELECT,GROUP BY andHAVING clauses must beimmutable.
• The view is not allowed to be asecurity barrier view.
• Window functions cannot be used in conjunction with continuous aggregates.

TIP:You can find thesettings for continuous aggregates andstatistics intimescaledb_information views.

Sample Usage

Create a continuous aggregate view.

CREATE MATERIALIZED VIEW continuous_aggregate_view( timec, minl, sumt, sumh )
WITH (timescaledb.continuous) AS
  SELECT time_bucket('1day', timec), min(location), sum(temperature), sum(humidity)
    FROM conditions
    GROUP BY time_bucket('1day', timec)

Add additional continuous aggregates on top of the same raw hypertable.

CREATE MATERIALIZED VIEW continuous_aggregate_view( timec, minl, sumt, sumh )
WITH (timescaledb.continuous) AS
  SELECT time_bucket('30day', timec), min(location), sum(temperature), sum(humidity)
    FROM conditions
    GROUP BY time_bucket('30day', timec);

CREATE MATERIALIZED VIEW continuous_aggregate_view( timec, minl, sumt, sumh )
WITH (timescaledb.continuous) AS
  SELECT time_bucket('1h', timec), min(location), sum(temperature), sum(humidity)
    FROM conditions
    GROUP BY time_bucket('1h', timec);

ALTER MATERIALIZED VIEW (Continuous Aggregate) Community

ALTER MATERIALIZED VIEW statement can be used to modify some of theWITH clauseoptions for the continuous aggregate view.

ALTER MATERIALIZED VIEW <view_name> SET ( timescaledb.<option> = <value> [, ... ] )

Parameters

Sample Usage

To disablereal-time aggregates for a continuous aggregate:

ALTER MATERIALIZED VIEW contagg_view SET (timescaledb.materialized_only = true);

The only option that currently can be modified withALTER MATERIALIZED VIEW ismaterialized_only. The other optionscontinuous andcreate_group_indexes can only be set when creating the continuous aggregate.

DROP MATERIALIZED VIEW (Continuous Aggregate) Community

Continuous aggregate views can be dropped using theDROP MATERIALIZED VIEW statement.

This statement deletes the continuous aggregate and all its internal objects. To also delete other dependent objects, such as a view defined on the continuous aggregate, add theCASCADE option. Dropping a continuous aggregate does not affect the data in the underlying hypertable from which the continuous aggregate is derived.

DROP MATERIALIZED VIEW <view_name>;

Parameters

Sample Usage

Drop existing continuous aggregate.

DROP MATERIALIZED VIEW contagg_view;

refresh_continuous_aggregate() Community

Refresh all buckets of a continuous aggregate between two points of time.

The function expects the parameter values to have the same time type as used in the continuous aggregate’s time bucket expression (e.g., if the time bucket specifies intimestamptz, then the start and end time supplied should also betimestamptz).

Required Arguments

Sample Usage

Refresh the continuous aggregateconditions between2020-01-01 and2020-02-01 exclusive.

CALL refresh_continuous_aggregate('conditions', '2020-01-01', '2020-02-01');

Automation policies Community

TimescaleDB includes an automation framework for allowing background tasks to run inside the database, controllable by user-supplied policies. These tasks currently include capabilities around data retention and data reordering for improving query performance.

The following functions allow the administrator to create/remove/alter policies that schedule administrative actions to take place on a hypertable. The actions are meant to implement data retention or perform tasks that will improve query performance on older chunks. Each policy is assigned a scheduled job which will be run in the background to enforce it.

Information about jobs created by policies can be viewed by queryingtimescaledb_information.jobs andtimescaledb_information.job_stats.

add_continuous_aggregate_policy() Community

Create a policy that automatically refreshes a continuous aggregate.

Required Arguments

Thestart_offset should be greater thanend_offset. Thestart_offset andend_offset parameters should be specified differently depending on the type of the time column of the hypertable:

• For hypertables with TIMESTAMP, TIMESTAMPTZ, and DATE time columns: the offset should be an INTERVAL type
• For hypertables with integer-based timestamps: the offset should be an integer type.

Optional Arguments

Returns

Sample Usage

Add a policy that refreshes the last month once an hour, excluding the latest hour from the aggregate (for performance reasons, it is recommended to exclude buckets that still see lots of writes):

SELECT add_continuous_aggregate_policy('conditions_summary',
    start_offset => INTERVAL '1 month',
    end_offset => INTERVAL '1 hour',
    schedule_interval => INTERVAL '1 hour');

add_job() Community

Register an action to be scheduled by our automation framework. Please read theinstructions for more details including multiple example actions.

Required Arguments

Optional Arguments

Returns

Sample Usage

CREATE OR REPLACE PROCEDURE user_defined_action(job_id int, config jsonb) LANGUAGE PLPGSQL AS
$$
BEGIN
  RAISE NOTICE 'Executing action % with config %', job_id, config;
END
$$;
SELECT add_job('user_defined_action','1h');

Register the procedureuser_defined_action to be run every hour.

delete_job() Community

Delete a job registered with the automation framework. This works for user-defined actions as well as policies.

If the job is currently running, the process will be terminated.

Required Arguments

Sample Usage

SELECT delete_job(1000);

Delete the job with the job id 1000.

run_job() Community

Run a previously registered job in the current session. This works for user-defined actions as well as policies. Sincerun_job is implemented as stored procedure it cannot be executed inside a SELECT query but has to be executed withCALL.

TIP:Any background worker job can be run in foreground when executed withrun_job. This can be useful to debug problems when combined with increased log level.

Required Arguments

Sample Usage

SET client_min_messages TO DEBUG1;
CALL run_job(1000);

Set log level shown to client to DEBUG1 and run the job with the job id 1000.

remove_continuous_aggregate_policy() Community

Remove refresh policy for a continuous aggregate.

Required Arguments

Sample Usage

Remove the refresh policy from the ‘cpu_view’ continuous aggregate:

SELECT remove_continuous_aggregate_policy('cpu_view');

add_retention_policy() Community

Create a policy to drop chunks older than a given interval of a particular hypertable or continuous aggregate on a schedule in the background. (Seedrop_chunks). This implements a data retention policy and will remove data on a schedule. Only one retention policy may exist per hypertable.

Required Arguments

Thedrop_after parameter should be specified differently depending on the type of the time column of the hypertable:

• For hypertables with TIMESTAMP, TIMESTAMPTZ, and DATE time columns: the time interval should be an INTERVAL type.
• For hypertables with integer-based timestamps: the time interval should be an integer type (this requires theinteger_now_func to be set).

Optional Arguments

Returns

Sample Usage

Create a data retention policy to discard chunks greater than 6 months old:

SELECT add_retention_policy('conditions', INTERVAL '6 months');

Create a data retention policy with an integer-based time column:

SELECT add_retention_policy('conditions', BIGINT '600000');

remove_retention_policy() Community

Remove a policy to drop chunks of a particular hypertable.

Required Arguments

Optional Arguments

Sample Usage

SELECT remove_retention_policy('conditions');

removes the existing data retention policy for theconditions table.

add_reorder_policy() Community

Create a policy to reorder chunks on a given hypertable index in the background. (Seereorder_chunk). Only one reorder policy may exist per hypertable. Only chunks that are the 3rd from the most recent will be reordered to avoid reordering chunks that are still being inserted into.

TIP:Once a chunk has been reordered by the background worker it will not be reordered again. So if one were to insert significant amounts of data in to older chunks that have already been reordered, it might be necessary to manually re-run thereorder_chunk function on older chunks, or to drop and re-create the policy if many older chunks have been affected.

Required Arguments

Optional Arguments

Returns

Sample Usage

SELECT add_reorder_policy('conditions', 'conditions_device_id_time_idx');

creates a policy to reorder completed chunks by the existing(device_id, time) index. (Seereorder_chunk).

remove_reorder_policy() Community

Remove a policy to reorder a particular hypertable.

Required Arguments

Optional Arguments

Sample Usage

SELECT remove_reorder_policy('conditions', if_exists => true);

removes the existing reorder policy for theconditions table if it exists.

alter_job() Community

Actions scheduled via TimescaleDB’s automation framework run periodically in a background worker. You can change the schedule of their execution usingalter_job. To alter an existing job, you must refer to it byjob_id. Thejob_id which executes a given action and its current schedule can be found either in thetimescaledb_information.jobs view, which lists information about every scheduled action, as well as intimescaledb_information.job_stats. Thejob_stats view additionally contains information about when each job was last run and other useful statistics for deciding what the new schedule should be.

Required Arguments

Optional Arguments

Returns

Sample Usage

SELECT alter_job(1000, schedule_interval => INTERVAL '2 days');

Reschedules the job with id 1000 so that it runs every two days.

SELECT alter_job(job_id, scheduled => false)
FROM timescaledb_information.jobs
WHERE proc_name = 'policy_compression' AND hypertable_name = 'conditions'

Disables scheduling of the compression policy on hypertableconditions.

SELECT alter_job(1015, next_start => '2020-03-15 09:00:00.0+00');

Reschedules continuous aggregate job1015 so that the next execution of the job starts at the specified time (9:00:00 am on March 15, 2020).

Analytics

first()

Thefirst aggregate allows you to get the value of one column as ordered by another. For example,first(temperature, time) will return the earliest temperature value based on time within an aggregate group.

Required Arguments

Sample Usage

Get the earliest temperature by device_id:

SELECT device_id, first(temp, time)
FROM metrics
GROUP BY device_id;

WARNING:Thelast andfirst commands donot use indexes, and instead perform a sequential scan through their groups. They are primarily used for ordered selection within aGROUP BY aggregate, and not as an alternative to anORDER BY time DESC LIMIT 1 clause to find the latest value (which will use indexes).

histogram()

Thehistogram() function represents the distribution of a set of values as an array of equal- buckets. It partitions the dataset into a specified number of buckets (nbuckets) ranging from the inputtedmin andmax values.

The return value is an array containingnbuckets+2 buckets, with the middlenbuckets bins for values in the stated range, the first bucket at the head of the array for values under the lowermin bound, and the last bucket for values greater than or equal to themax bound. Each bucket is inclusive on its lower bound, and exclusive on its upper bound. Therefore, values equal to themin are included in the bucket starting withmin, but values equal to themax are in the last bucket.

Required Arguments

Sample Usage

A simple bucketing of device’s battery levels from thereadings dataset:

SELECT device_id, histogram(battery_level, 20, 60, 5)
FROM readings
GROUP BY device_id
LIMIT 10;

The expected output:

 device_id | histogram
------------+------------------------------
 demo000000 | {0,0,0,7,215,206,572}
 demo000001 | {0,12,173,112,99,145,459}
 demo000002 | {0,0,187,167,68,229,349}
 demo000003 | {197,209,127,221,106,112,28}
 demo000004 | {0,0,0,0,0,39,961}
 demo000005 | {12,225,171,122,233,80,157}
 demo000006 | {0,78,176,170,8,40,528}
 demo000007 | {0,0,0,126,239,245,390}
 demo000008 | {0,0,311,345,116,228,0}
 demo000009 | {295,92,105,50,8,8,442}

interpolate() Community

Theinterpolate function does linear interpolation for missing values. It can only be used in an aggregation query withtime_bucket_gapfill. Theinterpolate function call cannot be nested inside other function calls.

Required Arguments

Optional Arguments

Because the interpolation function relies on having values before and after each bucketed period to compute the interpolated value, it might not have enough data to calculate the interpolation for the first and last time bucket if those buckets do not otherwise contain valid values. For example, the interpolation would require looking before this first time bucket period, yet the query’s outer time predicate WHERE time > … normally restricts the function to only evaluate values within this time range. Thus, theprev andnext expression tell the function how to look for values outside of the range specified by the time predicate. These expressions will only be evaluated when no suitable value is returned by the outer query (i.e., the first and/or last bucket in the queried time range is empty). The returned record forprev andnext needs to be a time, value tuple. The datatype of time needs to be the same as the time datatype in thetime_bucket_gapfill call. The datatype of value needs to be the same as thevalue datatype of theinterpolate call.

Sample Usage

Get the temperature every day for each device over the last week interpolating for missing readings:

SELECT
  time_bucket_gapfill('1 day', time, now() - INTERVAL '1 week', now()) AS day,
  device_id,
  avg(temperature) AS value,
  interpolate(avg(temperature))
FROM metrics
WHERE time > now () - INTERVAL '1 week'
GROUP BY day, device_id
ORDER BY day;
           day | device_id | value | interpolate
------------------------+-----------+-------+-------------
 2019-01-10 01:00:00+01 | 1 | |
 2019-01-11 01:00:00+01 | 1 | 5.0 | 5.0
 2019-01-12 01:00:00+01 | 1 | | 6.0
 2019-01-13 01:00:00+01 | 1 | 7.0 | 7.0
 2019-01-14 01:00:00+01 | 1 | | 7.5
 2019-01-15 01:00:00+01 | 1 | 8.0 | 8.0
 2019-01-16 01:00:00+01 | 1 | 9.0 | 9.0
(7 row)

Get the average temperature every day for each device over the last 7 days interpolating for missing readings with lookup queries for values before and after the gapfill time range:

SELECT
  time_bucket_gapfill('1 day', time, now() - INTERVAL '1 week', now()) AS day,
  device_id,
  avg(value) AS value,
  interpolate(avg(temperature),
    (SELECT (time,temperature) FROM metrics m2 WHERE m2.time < now() - INTERVAL '1 week' AND m.device_id = m2.device_id ORDER BY time DESC LIMIT 1),
    (SELECT (time,temperature) FROM metrics m2 WHERE m2.time > now() AND m.device_id = m2.device_id ORDER BY time DESC LIMIT 1)
  ) AS interpolate
FROM metrics m
WHERE time > now () - INTERVAL '1 week'
GROUP BY day, device_id
ORDER BY day;
           day | device_id | value | interpolate
------------------------+-----------+-------+-------------
 2019-01-10 01:00:00+01 | 1 | | 3.0
 2019-01-11 01:00:00+01 | 1 | 5.0 | 5.0
 2019-01-12 01:00:00+01 | 1 | | 6.0
 2019-01-13 01:00:00+01 | 1 | 7.0 | 7.0
 2019-01-14 01:00:00+01 | 1 | | 7.5
 2019-01-15 01:00:00+01 | 1 | 8.0 | 8.0
 2019-01-16 01:00:00+01 | 1 | 9.0 | 9.0
(7 row)

last()

Thelast aggregate allows you to get the value of one column as ordered by another. For example,last(temperature, time) will return the latest temperature value based on time within an aggregate group.

Required Arguments

Sample Usage

Get the temperature every 5 minutes for each device over the past day:

SELECT device_id, time_bucket('5 minutes', time) AS interval,
  last(temp, time)
FROM metrics
WHERE time > now () - INTERVAL '1 day'
GROUP BY device_id, interval
ORDER BY interval DESC;

WARNING:Thelast andfirst commands donot use indexes, and instead perform a sequential scan through their groups. They are primarily used for ordered selection within aGROUP BY aggregate, and not as an alternative to anORDER BY time DESC LIMIT 1 clause to find the latest value (which will use indexes).

locf() Community

Thelocf function (last observation carried forward) allows you to carry the last seen value in an aggregation group forward. It can only be used in an aggregation query withtime_bucket_gapfill. Thelocf function call cannot be nested inside other function calls.

Required Arguments

Optional Arguments

Because the locf function relies on having values before each bucketed period to carry forward, it might not have enough data to fill in a value for the first bucket if it does not contain a value. For example, the function would need to look before this first time bucket period, yet the query’s outer time predicate WHERE time > … normally restricts the function to only evaluate values within this time range. Thus, theprev expression tell the function how to look for values outside of the range specified by the time predicate. Theprev expression will only be evaluated when no previous value is returned by the outer query (i.e., the first bucket in the queried time range is empty).

Sample Usage

Get the average temperature every day for each device over the last 7 days carrying forward the last value for missing readings:

SELECT
  time_bucket_gapfill('1 day', time, now() - INTERVAL '1 week', now()) AS day,
  device_id,
  avg(temperature) AS value,
  locf(avg(temperature))
FROM metrics
WHERE time > now () - INTERVAL '1 week'
GROUP BY day, device_id
ORDER BY day;
           day | device_id | value | locf
------------------------+-----------+-------+------
 2019-01-10 01:00:00+01 | 1 | |
 2019-01-11 01:00:00+01 | 1 | 5.0 | 5.0
 2019-01-12 01:00:00+01 | 1 | | 5.0
 2019-01-13 01:00:00+01 | 1 | 7.0 | 7.0
 2019-01-14 01:00:00+01 | 1 | | 7.0
 2019-01-15 01:00:00+01 | 1 | 8.0 | 8.0
 2019-01-16 01:00:00+01 | 1 | 9.0 | 9.0
(7 row)

Get the average temperature every day for each device over the last 7 days carrying forward the last value for missing readings with out-of-bounds lookup

SELECT
  time_bucket_gapfill('1 day', time, now() - INTERVAL '1 week', now()) AS day,
  device_id,
  avg(temperature) AS value,
  locf(
    avg(temperature),
    (SELECT temperature FROM metrics m2 WHERE m2.time < now() - INTERVAL '2 week' AND m.device_id = m2.device_id ORDER BY time DESC LIMIT 1)
  )
FROM metrics m
WHERE time > now () - INTERVAL '1 week'
GROUP BY day, device_id
ORDER BY day;
           day | device_id | value | locf
------------------------+-----------+-------+------
 2019-01-10 01:00:00+01 | 1 | | 1.0
 2019-01-11 01:00:00+01 | 1 | 5.0 | 5.0
 2019-01-12 01:00:00+01 | 1 | | 5.0
 2019-01-13 01:00:00+01 | 1 | 7.0 | 7.0
 2019-01-14 01:00:00+01 | 1 | | 7.0
 2019-01-15 01:00:00+01 | 1 | 8.0 | 8.0
 2019-01-16 01:00:00+01 | 1 | 9.0 | 9.0
(7 row)

time_bucket()

This is a more powerful version of the standard PostgreSQLdate_trunc function. It allows for arbitrary time intervals instead of the second, minute, hour, etc. provided bydate_trunc. The return value is the bucket’s start time. Below is necessary information for using it effectively.

TIP:TIMESTAMPTZ arguments are bucketed by the time at UTC. So the alignment of buckets is on UTC time. One consequence of this is that daily buckets are aligned to midnight UTC, not local time.

If the user wants buckets aligned by local time, the TIMESTAMPTZ input should be cast to TIMESTAMP (such a cast converts the value to local time) before being passed to time_bucket (see example below). Note that along daylight savings time boundaries the amount of data aggregated into a bucket after such a cast is irregular: for example if the bucket_ is 2 hours, the number of UTC hours bucketed by local time on daylight savings time boundaries can be either 3 hours or 1 hour.

Required Arguments

Optional Arguments

For Integer Time Inputs

Required Arguments

Optional Arguments

Sample Usage

Simple 5-minute averaging:

SELECT time_bucket('5 minutes', time) AS five_min, avg(cpu)
FROM metrics
GROUP BY five_min
ORDER BY five_min DESC LIMIT 10;

To report the middle of the bucket, instead of the left edge:

SELECT time_bucket('5 minutes', time) + '2.5 minutes'
  AS five_min, avg(cpu)
FROM metrics
GROUP BY five_min
ORDER BY five_min DESC LIMIT 10;

For rounding, move the alignment so that the middle of the bucket is at the 5 minute mark (and report the middle of the bucket):

SELECT time_bucket('5 minutes', time, '-2.5 minutes') + '2.5 minutes'
  AS five_min, avg(cpu)
FROM metrics
GROUP BY five_min
ORDER BY five_min DESC LIMIT 10;

To shift the alignment of the buckets you can use the origin parameter (passed as a timestamp, timestamptz, or date type). In this example, we shift the start of the week to a Sunday (the default is a Monday).

SELECT time_bucket('1 week', timetz, TIMESTAMPTZ '2017-12-31')
  AS one_week, avg(cpu)
FROM metrics
GROUP BY one_week
WHERE time > TIMESTAMPTZ '2017-12-01' AND time < TIMESTAMPTZ '2018-01-03'
ORDER BY one_week DESC LIMIT 10;

The value of the origin parameter we used in this example was2017-12-31, a Sunday within the period being analyzed. However, the origin provided to the function can be before, during, or after the data being analyzed. All buckets are calculated relative to this origin. So, in this example, any Sunday could have been used. Note that becausetime < TIMESTAMPTZ '2018-01-03' in this example, the last bucket would have only 4 days of data.

Bucketing a TIMESTAMPTZ at local time instead of UTC(see note above):

SELECT time_bucket(INTERVAL '2 hours', timetz::TIMESTAMP)
  AS five_min, avg(cpu)
FROM metrics
GROUP BY five_min
ORDER BY five_min DESC LIMIT 10;

Note that the above cast to TIMESTAMP converts the time to local time according to the server’s timezone setting.

WARNING:For users upgrading from a version before 1.0.0, please note that the default origin was moved from 2000-01-01 (Saturday) to 2000-01-03 (Monday) between versions 0.12.1 and 1.0.0. This change was made to make time_bucket compliant with the ISO standard for Monday as the start of a week. This should only affect multi-day calls to time_bucket. The old behavior can be reproduced by passing 2000-01-01 as the origin parameter to time_bucket.

time_bucket_gapfill() Community

Thetime_bucket_gapfill function works similar totime_bucket but also activates gap filling for the interval betweenstart andfinish. It can only be used with an aggregation query. Values outside ofstart andfinish will pass through but no gap filling will be done outside of the specified range.

Starting with version 1.3.0,start andfinish are optional arguments and will be inferred from the WHERE clause if not supplied as arguments.

TIP:We recommend using a WHERE clause whenever possible (instead of juststart andfinish arguments), as start and finish arguments will not filter input rows. Thus without a WHERE clause, this will lead TimescaleDB’s planner to select all data and not perform constraint exclusion to exclude chunks from further processing, which would be less performant.

Thetime_bucket_gapfill must be a top-level expression in a query or subquery, as shown in the above examples. You cannot, for example, do something likeround(time_bucket_gapfill(...)) or cast the result of the gapfill call (unless as a subquery where the outer query does the type cast).

Required Arguments

Optional Arguments

Note that explicitly providedstart andstop or derived from WHERE clause values need to be simple expressions. Such expressions should be evaluated to constants at the query planning. For example, simple expressions can contain constants or call tonow(), but cannot reference to columns of a table.

For Integer Time Inputs

Required Arguments

Optional Arguments

Starting with version 1.3.0start andfinish are optional arguments and will be inferred from the WHERE clause if not supplied as arguments.

Sample Usage

Get the metric value every day over the last 7 days:

SELECT
  time_bucket_gapfill('1 day', time) AS day,
  device_id,
  avg(value) AS value
FROM metrics
WHERE time > now() - INTERVAL '1 week' AND time < now()
GROUP BY day, device_id
ORDER BY day;
           day | device_id | value
------------------------+-----------+-------
 2019-01-10 01:00:00+01 | 1 |
 2019-01-11 01:00:00+01 | 1 | 5.0
 2019-01-12 01:00:00+01 | 1 |
 2019-01-13 01:00:00+01 | 1 | 7.0
 2019-01-14 01:00:00+01 | 1 |
 2019-01-15 01:00:00+01 | 1 | 8.0
 2019-01-16 01:00:00+01 | 1 | 9.0
(7 row)

Get the metric value every day over the last 7 days carrying forward the previous seen value if none is available in an interval:

SELECT
  time_bucket_gapfill('1 day', time) AS day,
  device_id,
  avg(value) AS value,
  locf(avg(value))
FROM metrics
WHERE time > now() - INTERVAL '1 week' AND time < now()
GROUP BY day, device_id
ORDER BY day;
           day | device_id | value | locf
------------------------+-----------+-------+------
 2019-01-10 01:00:00+01 | 1 | |
 2019-01-11 01:00:00+01 | 1 | 5.0 | 5.0
 2019-01-12 01:00:00+01 | 1 | | 5.0
 2019-01-13 01:00:00+01 | 1 | 7.0 | 7.0
 2019-01-14 01:00:00+01 | 1 | | 7.0
 2019-01-15 01:00:00+01 | 1 | 8.0 | 8.0
 2019-01-16 01:00:00+01 | 1 | 9.0 | 9.0

Get the metric value every day over the last 7 days interpolating missing values:

SELECT
  time_bucket_gapfill('5 minutes', time) AS day,
  device_id,
  avg(value) AS value,
  interpolate(avg(value))
FROM metrics
WHERE time > now() - INTERVAL '1 week' AND time < now()
GROUP BY day, device_id
ORDER BY day;
           day | device_id | value | interpolate
------------------------+-----------+-------+-------------
 2019-01-10 01:00:00+01 | 1 | |
 2019-01-11 01:00:00+01 | 1 | 5.0 | 5.0
 2019-01-12 01:00:00+01 | 1 | | 6.0
 2019-01-13 01:00:00+01 | 1 | 7.0 | 7.0
 2019-01-14 01:00:00+01 | 1 | | 7.5
 2019-01-15 01:00:00+01 | 1 | 8.0 | 8.0
 2019-01-16 01:00:00+01 | 1 | 9.0 | 9.0

Utilities/Statistics

timescaledb_information.data_nodes

Get information on data nodes. This function is specific to running TimescaleDB in a multi-node setup.

Available Columns

Sample Usage

Get metadata related to data nodes.

SELECT * FROM timescaledb_information.data_nodes;
 node_name | owner | options 
--------------+------------+--------------------------------
 dn1 | postgres | {host=localhost,port=15431,dbname=test} 
 dn2 | postgres | {host=localhost,port=15432,dbname=test} 
(2 rows)