StatsD writing

StatsD is a simple daemon for aggregating application metrics, which has evolved rapidly in recent years into a unified protocol for collecting application performance metrics.

You can write StatsD data to TDengine by simply modifying the configuration file of StatsD with the domain name (or IP address) of the server running taosAdapter and the corresponding port. It can take full advantage of TDengine’s efficient storage query performance and clustering capabilities for time-series data.

Prerequisites

To write StatsD data to TDengine requires the following preparations.

  • The TDengine cluster has been deployed and is working properly
  • taosAdapter is installed and running properly. Please refer to the taosAdapter manual for details.
  • StatsD has been installed. To install StatsD, please refer to official documentation

Configuration steps

Configuring taosAdapter

To configure taosAdapter to receive StatsD data.

  • Enable the configuration item in the taosAdapter configuration file (default location /etc/taos/taosadapter.toml)
  1. ...
  2. [statsd]
  3. enable = true
  4. port = 6044
  5. db = "statsd"
  6. user = "root"
  7. password = "taosdata"
  8. worker = 10
  9. gatherInterval = "5s"
  10. protocol = "udp"
  11. maxTCPConnections = 250
  12. tcpKeepAlive = false
  13. allowPendingMessages = 50000
  14. deleteCounters = true
  15. deleteGauges = true
  16. deleteSets = true
  17. deleteTimings = true
  18. ...

The default database name written by taosAdapter is statsd. To specify a different name, you can also modify the taosAdapter configuration file db entry. user and password fill in the actual TDengine configuration values. After changing the configuration file, you need to restart the taosAdapter.

  • You can also enable taosAdapter to receive StatsD data by using the taosAdapter command-line parameters or setting environment variables.

Configuring StatsD

To use StatsD, you need to download its source code. Please refer to the example file exampleConfig.js in the root directory of the source download to modify the configuration file. In <taosAdpater’s host>, please fill in the domain name or IP address of the server running taosAdapter, and <port for StatsD>, please fill in the port where taosAdapter receives StatsD data (default is 6044).

  1. backends section add ". /backends/repeater"
  2. Add { host:'<taosAdapter's host>', port: <port for StatsD>} to repeater section

Example configuration file.

  1. {
  2. port: 8125
  3. , backends: [". /backends/repeater"]
  4. , repeater: [{ host: '127.0.0.1', port: 6044}]
  5. }

Start StatsD after adding the following (assuming the config file is modified to config.js)

  1. node stats.js config.js &

Verification method

Start StatsD:

  1. $ node stats.js config.js &
  2. [1] 8546
  3. $ 20 Apr 09:54:41 - [8546] reading config file: exampleConfig.js
  4. 20 Apr 09:54:41 - server is up INFO

Using the utility software nc to write data for test:

  1. $ echo "foo:1|c" | nc -u -w0 127.0.0.1 8125

Use the TDengine CLI to verify that StatsD data is written to TDengine and can read out correctly.

  1. Welcome to the TDengine shell from Linux, Client Version:2.4.0.0
  2. Copyright (c) 2020 by TAOS Data, Inc. All rights reserved.
  4. taos> show databases;
  5.               name              |      created_time       |   ntables   |   vgroups   | replica | quorum |  days  |           keep           |  cache(MB)  |   blocks    |   minrows   |   maxrows   | wallevel |    fsync    | comp | cachelast | precision | update |   status   |
  6. ====================================================================================================================================================================================================================================================================================
  7.  log                            | 2022-04-20 07:19:50.260 |          11 |           1 |       1 |      1 |     10 | 3650                     |          16 |           6 |         100 |        4096 |        1 |        3000 |    2 |         0 | ms        |      0 | ready      |
  8.  statsd                         | 2022-04-20 09:54:51.220 |           1 |           1 |       1 |      1 |     10 | 3650                     |          16 |           6 |         100 |        4096 |        1 |        3000 |    2 |         0 | ns        |      2 | ready      |
  9. Query OK, 2 row(s) in set (0.003142s)
  11. taos> use statsd;
  12. Database changed.
  14. taos> show stables;
  15.               name              |      created_time       | columns |  tags  |   tables    |
  16. ============================================================================================
  17.  foo                            | 2022-04-20 09:54:51.234 |       2 |      1 |           1 |
  18. Query OK, 1 row(s) in set (0.002161s)
  20. taos> select * from foo;
  21.               ts               |         value         |         metric_type          |
  22. =======================================================================================
  23.  2022-04-20 09:54:51.219614235 |                     1 | counter                      |
  24. Query OK, 1 row(s) in set (0.004179s)
  26. taos>