TDengine Rust Connector

taos is the official Rust connector for TDengine. Rust developers can develop applications to access the TDengine instance data.

taos provides two ways to establish connections. One is the Native Connection, which connects to TDengine instances via the TDengine client driver (taosc). The other is the WebSocket connection, which connects to TDengine instances via the WebSocket interface provided by taosAdapter. You can specify a connection type with Cargo features. By default, both types are supported. The Websocket connection can be used on any platform. The native connection can be used on any platform that the TDengine Client supports.

The source code for the Rust connectors is located on GitHub.

Supported platforms

Native connections are supported on the same platforms as the TDengine client driver. Websocket connections are supported on all platforms that can run Go.

Version support

Please refer to version support list

The Rust Connector is still under rapid development and is not guaranteed to be backward compatible before 1.0. We recommend using TDengine version 3.0 or higher to avoid known issues.

Installation

Pre-installation preparation

• Install the Rust development toolchain
• If using the native connection, please install the TDengine client driver. Please refer to install client driver

Add taos dependency

Depending on the connection method, add the taos dependency in your Rust project as follows:

• Support Both
• Websocket only
• native connection only

In cargo.toml, add taos:

[dependencies]
# use default feature
taos = "*"

In cargo.toml, add taos and enable the ws feature:

[dependencies]
taos = { version = "*", default-features = false, features = ["ws"] }

In cargo.toml, add taos and enable the native feature:

[dependencies]
taos = { version = "*", default-features = false, features = ["native"] }

Establishing a connection

TaosBuilder creates a connection constructor through the DSN connection description string.

let builder = TaosBuilder::from_dsn("taos://")?;

You can now use this object to create the connection.

let conn = builder.build()?;

The connection object can create more than one.

let conn1 = builder.build()?;
let conn2 = builder.build()?;

The structure of the DSN description string is as follows:

<driver>[+<protocol>]://[[<username>:<password>@]<host>:<port>][/<database>][?<p1>=<v1>[&<p2>=<v2>]]
|------|------------|---|-----------|-----------|------|------|------------|-----------------------|
|driver|   protocol |   | username  | password  | host | port |  database  |  params               |

The parameters are described as follows:

• driver: Specify a driver name so that the connector can choose which method to use to establish the connection. Supported driver names are as follows:
  • taos: Table names use the TDengine connector driver.
  • tmq: Use the TMQ to subscribe to data.
  • http/ws: Use Websocket to establish connections.
  • https/wss: Use Websocket to establish connections, and enable SSL/TLS.
• protocol: Specify which connection method to use. For example, taos+ws://localhost:6041 uses Websocket to establish connections.
• username/password: Username and password used to create connections.
• host/port: Specifies the server and port to establish a connection. If you do not specify a hostname or port, native connections default to localhost:6030 and Websocket connections default to localhost:6041.
• database: Specify the default database to connect to. It’s optional.
• params：Optional parameters.

A sample DSN description string is as follows:

taos+ws://localhost:6041/test

This indicates that the Websocket connection method is used on port 6041 to connect to the server localhost and use the database test by default.

You can create DSNs to connect to servers in your environment.

use taos::*;
// use native protocol.
let builder = TaosBuilder::from_dsn("taos://localhost:6030")?;
let conn1 = builder.build();
//  use websocket protocol.
let conn2 = TaosBuilder::from_dsn("taos+ws://localhost:6041")?;

After the connection is established, you can perform operations on your database.

async fn demo(taos: &Taos, db: &str) -> Result<(), Error> {
    // prepare database
    taos.exec_many([
        format!("DROP DATABASE IF EXISTS `{db}`"),
        format!("CREATE DATABASE `{db}`"),
        format!("USE `{db}`"),
    ])
    .await?;
    let inserted = taos.exec_many([
        // create super table
        "CREATE TABLE `meters` (`ts` TIMESTAMP, `current` FLOAT, `voltage` INT, `phase` FLOAT) \
         TAGS (`groupid` INT, `location` BINARY(24))",
        // create child table
        "CREATE TABLE `d0` USING `meters` TAGS(0, 'California.LosAngles')",
        // insert into child table
        "INSERT INTO `d0` values(now - 10s, 10, 116, 0.32)",
        // insert with NULL values
        "INSERT INTO `d0` values(now - 8s, NULL, NULL, NULL)",
        // insert and automatically create table with tags if not exists
        "INSERT INTO `d1` USING `meters` TAGS(1, 'California.SanFrancisco') values(now - 9s, 10.1, 119, 0.33)",
        // insert many records in a single sql
        "INSERT INTO `d1` values (now-8s, 10, 120, 0.33) (now - 6s, 10, 119, 0.34) (now - 4s, 11.2, 118, 0.322)",
    ]).await?;
    assert_eq!(inserted, 6);
    let mut result = taos.query("select * from `meters`").await?;
    for field in result.fields() {
        println!("got field: {}", field.name());
    }
    let values = result.
}

There are two ways to query data: Using built-in types or the serde deserialization framework.

    // Query option 1, use rows stream.
    let mut rows = result.rows();
    while let Some(row) = rows.try_next().await? {
        for (name, value) in row {
            println!("got value of {}: {}", name, value);
        }
    }
    // Query options 2, use deserialization with serde.
    #[derive(Debug, serde::Deserialize)]
    #[allow(dead_code)]
    struct Record {
        // deserialize timestamp to chrono::DateTime<Local>
        ts: DateTime<Local>,
        // float to f32
        current: Option<f32>,
        // int to i32
        voltage: Option<i32>,
        phase: Option<f32>,
        groupid: i32,
        // binary/varchar to String
        location: String,
    }
    let records: Vec<Record> = taos
        .query("select * from `meters`")
        .await?
        .deserialize()
        .try_collect()
        .await?;
    dbg!(records);
    Ok(())

Usage examples

Write data

SQL Write

use taos::*;
#[tokio::main]
async fn main() -> anyhow::Result<()> {
    let dsn = "ws://";
    let taos = TaosBuilder::from_dsn(dsn)?.build()?;
    taos.exec_many([
        "DROP DATABASE IF EXISTS power",
        "CREATE DATABASE power",
        "USE power",
        "CREATE STABLE power.meters (ts TIMESTAMP, current FLOAT, voltage INT, phase FLOAT) TAGS (location BINARY(64), groupId INT)"
    ]).await?;
    let inserted = taos.exec("INSERT INTO 
    power.d1001 USING power.meters TAGS('California.SanFrancisco', 2)
     VALUES ('2018-10-03 14:38:05.000', 10.30000, 219, 0.31000) 
     ('2018-10-03 14:38:15.000', 12.60000, 218, 0.33000) ('2018-10-03 14:38:16.800', 12.30000, 221, 0.31000)
    power.d1002 USING power.meters TAGS('California.SanFrancisco', 3)
     VALUES ('2018-10-03 14:38:16.650', 10.30000, 218, 0.25000)
    power.d1003 USING power.meters TAGS('California.LosAngeles', 2) 
     VALUES ('2018-10-03 14:38:05.500', 11.80000, 221, 0.28000) ('2018-10-03 14:38:16.600', 13.40000, 223, 0.29000)
    power.d1004 USING power.meters TAGS('California.LosAngeles', 3) 
     VALUES ('2018-10-03 14:38:05.000', 10.80000, 223, 0.29000) ('2018-10-03 14:38:06.500', 11.50000, 221, 0.35000)").await?;
    assert_eq!(inserted, 8);
    Ok(())
}

view source code

STMT Write

use taos::*;
#[tokio::main]
async fn main() -> anyhow::Result<()> {
    let taos = TaosBuilder::from_dsn("taos://")?.build()?;
    taos.create_database("power").await?;
    taos.use_database("power").await?;
    taos.exec("CREATE STABLE IF NOT EXISTS meters (ts TIMESTAMP, current FLOAT, voltage INT, phase FLOAT) TAGS (location BINARY(64), groupId INT)").await?;
    let mut stmt = Stmt::init(&taos)?;
    stmt.prepare("INSERT INTO ? USING meters TAGS(?, ?) VALUES(?, ?, ?, ?)")?;
    // bind table name and tags
    stmt.set_tbname_tags(
        "d1001",
        &[
            Value::VarChar("California.SanFransico".into()),
            Value::Int(2),
        ],
    )?;
    // bind values.
    let values = vec![
        ColumnView::from_millis_timestamp(vec![1648432611249]),
        ColumnView::from_floats(vec![10.3]),
        ColumnView::from_ints(vec![219]),
        ColumnView::from_floats(vec![0.31]),
    ];
    stmt.bind(&values)?;
    // bind one more row
    let values2 = vec![
        ColumnView::from_millis_timestamp(vec![1648432611749]),
        ColumnView::from_floats(vec![12.6]),
        ColumnView::from_ints(vec![218]),
        ColumnView::from_floats(vec![0.33]),
    ];
    stmt.bind(&values2)?;
    stmt.add_batch()?;
    // execute.
    let rows = stmt.execute()?;
    assert_eq!(rows, 2);
    Ok(())
}

view source code

Query data

use taos::sync::*;
fn main() -> anyhow::Result<()> {
    let taos = TaosBuilder::from_dsn("ws:///power")?.build()?;
    let mut result = taos.query("SELECT ts, current FROM meters LIMIT 2")?;
    // print column names
    let meta = result.fields();
    println!("{}", meta.iter().map(|field| field.name()).join("\t"));
    // print rows
    let rows = result.rows();
    for row in rows {
        let row = row?;
        for (_name, value) in row {
            print!("{}\t", value);
        }
        println!();
    }
    Ok(())
}
// output(suppose you are in +8 timezone):
// ts      current
// 2018-10-03T14:38:05+08:00       10.3
// 2018-10-03T14:38:15+08:00       12.6

view source code

API Reference

Connector Constructor

You create a connector constructor by using a DSN.

let cfg = TaosBuilder::default().build()?;

You use the builder object to create multiple connections.

let conn: Taos = cfg.build();

Connection pooling

In complex applications, we recommend enabling connection pools. taos implements connection pools based on r2d2.

As follows, a connection pool with default parameters can be generated.

let pool = TaosBuilder::from_dsn(dsn)?.pool()?;

You can set the same connection pool parameters using the connection pool’s constructor.

let dsn = "taos://localhost:6030";
let opts = PoolBuilder::new()
    .max_size(5000) // max connections
    .max_lifetime(Some(Duration::from_secs(60 * 60))) // lifetime of each connection
    .min_idle(Some(1000)) // minimal idle connections
    .connection_timeout(Duration::from_secs(2));
let pool = TaosBuilder::from_dsn(dsn)?.with_pool_builder(opts)?;

In the application code, use pool.get()? to get a connection object Taos.

let taos = pool.get()?;

Connectors

The Taos object provides an API to perform operations on multiple databases.

1. exec: Execute some non-query SQL statements, such as CREATE, ALTER, INSERT, etc.

   let affected_rows = taos.exec("INSERT INTO tb1 VALUES(now, NULL)").await?;

2. exec_many: Run multiple SQL statements simultaneously or in order.

   taos.exec_many([
       "CREATE DATABASE test",
       "USE test",
       "CREATE TABLE `tb1` (`ts` TIMESTAMP, `val` INT)",
   ]).await?;

3. query: Run a query statement and return a [ResultSet] object.

   let mut q = taos.query("select * from log.logs").await?;

   The [ResultSet] object stores query result data and the names, types, and lengths of returned columns

   You can obtain column information by using [.fields()].

   let cols = q.fields();
   for col in cols {
       println!("name: {}, type: {:?} , bytes: {}", col.name(), col.ty(), col.bytes());
   }

   It fetches data line by line.

   let mut rows = result.rows();
   let mut nrows = 0;
   while let Some(row) = rows.try_next().await? {
       for (col, (name, value)) in row.enumerate() {
           println!(
               "[{}] got value in col {} (named `{:>8}`): {}",
               nrows, col, name, value
           );
       }
       nrows += 1;
   }

   Or use the serde deserialization framework.

   #[derive(Debug, Deserialize)]
   struct Record {
       // deserialize timestamp to chrono::DateTime<Local>
       ts: DateTime<Local>,
       // float to f32
       current: Option<f32>,
       // int to i32
       voltage: Option<i32>,
       phase: Option<f32>,
       groupid: i32,
       // binary/varchar to String
       location: String,
   }
   let records: Vec<Record> = taos
       .query("select * from `meters`")
       .await?
       .deserialize()
       .try_collect()
       .await?;

Note that Rust asynchronous functions and an asynchronous runtime are required.

Taos provides Rust methods for some SQL statements to reduce the number of format!s.

• .describe(table: &str): Executes DESCRIBE and returns a Rust data structure.
• .create_database(database: &str): Executes the CREATE DATABASE statement.
• .use_database(database: &str): Executes the USE statement.

In addition, this structure is also the entry point for [Parameter Binding](#Parameter Binding Interface) and [Line Protocol Interface](#Line Protocol Interface). Please refer to the specific API descriptions for usage.

Bind Interface

Similar to the C interface, Rust provides the bind interface’s wrapping. First, the Taos object creates a parameter binding object Stmt for an SQL statement.

let mut stmt = Stmt::init(&taos).await?;
stmt.prepare("INSERT INTO ? USING meters TAGS(?, ?) VALUES(?, ?, ?, ?)")?;

The bind object provides a set of interfaces for implementing parameter binding.

.set_tbname(name)

To bind table names.

let mut stmt = taos.stmt("insert into ? values(? ,?)")?;
stmt.set_tbname("d0")?;

.set_tags(&[tag])

Bind sub-table table names and tag values when the SQL statement uses a super table.

let mut stmt = taos.stmt("insert into ? using stb0 tags(?) values(? ,?)")?;
stmt.set_tbname("d0")?;
stmt.set_tags(&[Value::VarChar("taos".to_string())])?;

.bind(&[column])

Bind value types. Use the [ColumnView] structure to create and bind the required types.

let params = vec![
    ColumnView::from_millis_timestamp(vec![164000000000]),
    ColumnView::from_bools(vec![true]),
    ColumnView::from_tiny_ints(vec![i8::MAX]),
    ColumnView::from_small_ints(vec![i16::MAX]),
    ColumnView::from_ints(vec![i32::MAX]),
    ColumnView::from_big_ints(vec![i64::MAX]),
    ColumnView::from_unsigned_tiny_ints(vec![u8::MAX]),
    ColumnView::from_unsigned_small_ints(vec![u16::MAX]),
    ColumnView::from_unsigned_ints(vec![u32::MAX]),
    ColumnView::from_unsigned_big_ints(vec![u64::MAX]),
    ColumnView::from_floats(vec![f32::MAX]),
    ColumnView::from_doubles(vec![f64::MAX]),
    ColumnView::from_varchar(vec!["ABC"]),
    ColumnView::from_nchar(vec!["涛思数据"]),
];
let rows = stmt.bind(&params)?.add_batch()?.execute()?;

.execute()

Execute SQL. Stmt objects can be reused, re-binded, and executed after execution. Before execution, ensure that all data has been added to the queue with .add_batch.

stmt.execute()?;
// next bind cycle.
//stmt.set_tbname()?;
//stmt.bind()?;
//stmt.execute()?;

For a working example, see GitHub.

Subscriptions

TDengine starts subscriptions through TMQ.

You create a TMQ connector by using a DSN.

let tmq = TmqBuilder::from_dsn("taos://localhost:6030/?group.id=test")?;

Create a consumer:

let mut consumer = tmq.build()?;

A single consumer can subscribe to one or more topics.

consumer.subscribe(["tmq_meters"]).await?;

The TMQ is of futures::Stream type. You can use the corresponding API to consume each message in the queue and then use .commit to mark them as consumed.

{
    let mut stream = consumer.stream();
    while let Some((offset, message)) = stream.try_next().await? {
        // get information from offset
        // the topic
        let topic = offset.topic();
        // the vgroup id, like partition id in kafka.
        let vgroup_id = offset.vgroup_id();
        println!("* in vgroup id {vgroup_id} of topic {topic}\n");
        if let Some(data) = message.into_data() {
            while let Some(block) = data.fetch_raw_block().await? {
                // one block for one table, get table name if needed
                let name = block.table_name();
                let records: Vec<Record> = block.deserialize().try_collect()?;
                println!(
                    "** table: {}, got {} records: {:#?}\n",
                    name.unwrap(),
                    records.len(),
                    records
                );
            }
        }
        consumer.commit(offset).await?;
    }
}

Unsubscribe:

consumer.unsubscribe().await;

The following parameters can be configured for the TMQ DSN. Only group.id is mandatory.

• group.id: Within a consumer group, load balancing is implemented by consuming messages on an at-least-once basis.
• client.id: Subscriber client ID.
• auto.offset.reset: Initial point of subscription. earliest subscribes from the beginning, and latest subscribes from the newest message. The default is earliest. Note: This parameter is set per consumer group.
• enable.auto.commit: Automatically commits. This can be enabled when data consistency is not essential.
• auto.commit.interval.ms: Interval for automatic commits.

For more information, see GitHub sample file.

For information about other structure APIs, see the Rust documentation.