Security

How to secure your rqlite deployment

rqlite can be secured in various way, and with different levels of control.

SQLite security

SQLite has some documentation on security, which is worth reviewing. Much of it can be applied to rqlite, though implementing some of the practices would need you to recompile rqlite.

File system security

You should control access to the data directory that each rqlite node uses. There is no reason for any user to directly access this directory. File-level security is also very important if you decide to use TLS Certificates and Keys with rqlite (see later).

You are also responsible for securing access to the SQLite database file if you change its path via rqlited command-line flags. Unless your application requires it, there is no reason for any client to directly access the SQLite file, and writing to the SQLite file may cause rqlite to fail.

Network security

Each rqlite node listens on 2 TCP ports – one for the HTTP API, and the other for inter-node (Raft consensus) communications. Only the HTTP API port need be reachable from outside the cluster.

So, if possible, configure the network such that the Raft port on each node is only accessible from other nodes in the cluster. Alternatively run the Raft connections on a physically, or logically, different network from the network the HTTP API is connected to. There is no need for the Raft port to be accessible by rqlite clients, which only need to use the HTTP API.

If the IP addresses (or subnets) of rqlite clients is also known, it may also be possible to limit access to the HTTP API from those addresses only.

AWS EC2 Security Groups, for example, support all this functionality. So if running rqlite in the AWS EC2 cloud you can implement this level of security at the network level.

HTTPS API

rqlite supports secure access via HTTPS, using Transport Layer Security (TLS). Using TLS ensures that all communication between clients and a cluster is encrypted. The HTTPS API also supports Mutual TLS. If mutual TLS is enabled only clients that present a trusted certificate can access rqlite.

To configure HTTPS, you set the following command-line options when launching rqlite:

  1.   -http-ca-cert string
  2.       Path to X.509 CA certificate for HTTPS.
  3.       If not set, then the host systems CA certificate(s) will be used.
  4.       This certificate must be set however if using mutual TLS.
  5.   -http-cert string
  6.       Path to HTTPS X.509 certificate
  7.   -http-key string
  8.       Path to corresponding HTTPS X.509 private key
  9.   -http-no-verify
  10.       Skip verification of HTTPS certificates when joining a cluster.
  11.       Mostly used for testing.
  12.   -http-verify-client
  13.       Enable mutual TLS for HTTPS. Mutual TLS is disabled by default.

Node-to-node encryption

rqlite supports encryption of all inter-node traffic. TLS is again used, and mutual TLS is also supported so you can restrict nodes to only accept inter-node connections from other nodes that present a valid certificate. To use TLS each node must be supplied with the relevant SSL certificate and corresponding private key, in X.509 format. Note that every node in a cluster must operate with inter-node encryption enabled, or none at all.

  1.   -node-ca-cert string
  2.       Path to X.509 CA certificate for node-to-node encryption.
  3.       If not set, then the host systems CA certificate(s) will be used
  4.       when verifying server certificates. This certificate is required
  5.       for verifying client certificates.
  6.   -node-cert string
  7.       Path to X.509 certificate for node-to-node communication
  8.   -node-key string
  9.       Path to X.509 private key for node-to-node communicate
  10.   -node-no-verify
  11.       Skip verification of any node-node certificates in each direction.
  12.       Mostly used for testing.
  13.   -node-verify-client
  14.       Enable mutual TLS for node-to-node communication.
  15.       Mutual TLS is disabled by default.

Configuring Usernames and Passwords

The HTTP API supports Basic Auth. Each rqlite node can be passed a JSON-formatted configuration file, which configures valid usernames and associated passwords for that node. In otherwords if you want every node in a cluster to accept identical credentials for a given user, you must ensure the configuration file for every node contains the same information for that user. But the configuration does not need to be identical under every node.

The 7.x series supported bcrypted password hashes, as well as plaintext passwords. The use of bcrypted hashes was flawed in the 7.x series, and support has been removed for bcrypted hashes in the 8.x series. If you are running the 7.x series you should replace any use of bcrypted hashes with the actual passwords (and secure the Credentials configuration).

User-level permissions

rqlite, via the configuration file, also supports user-level permissions. Each user can be granted one or more of the following permissions:

  • all: user can perform all operations on a node.
  • execute: user may access the execute endpoint at /db/execute.
  • query: user may access the query endpoint at /db/query.
  • load: user may load an SQLite dump file into a node via the /db/load or /boot endpoints.
  • backup: user may retrieve a backup via the endpoint /db/backup.
  • snapshot: user may initiate a Raft Snapshot via the endpoint /snapshot.
  • status: user can retrieve node status and Go runtime information.
  • ready: user can retrieve node readiness via /readyz
  • join: user can join a cluster. In practice only a node joins a cluster, so it’s the joining node that must supply the credentials.
  • join-read-only: user can join a cluster, but only as a read-only node.
  • remove: user can remove a node from a cluster. If a node performs an auto-remove on shutdown, then the -join-as user must have this permission.

Note that for a user to be able to access the Unified Endpoint, they must have both execute and query permissions.

Example configuration file

An example configuration file is shown below.

  1. [
  2.   {
  3.     "username": "bob",
  4.     "password": "secret1",
  5.     "perms": ["all"]
  6.   },
  7.   {
  8.     "username": "mary",
  9.     "password": "secret2",
  10.     "perms": ["query", "backup", "join"]
  11.   },
  12.   {
  13.     "username": "*",
  14.     "perms": ["status", "ready", "join-read-only"]
  15.   }
  16. ]

This configuration file sets authentication for three usernames, bob, mary, and *. It sets a password for the first two.

This configuration also sets permissions for all users. bob has permission to perform all operations, but mary can query the cluster, as well as backup and join the cluster. * is a special username, which indicates that all users – even anonymous users (requests without any BasicAuth information) – have permission to check the cluster status and readiness. All users can also join as a read-only node. This can be useful if you wish to leave certain operations open to all accesses.

Secure cluster example

Starting a node with HTTPS enabled, node-to-node encryption, mutual TLS disabled, and with the above configuration file. It is assumed the HTTPS X.509 certificate and key are at the paths server.crt and key.pem respectively, and the node-to-node certificate and key are at node.crt and node-key.pem

  1. rqlited -auth config.json -http-cert server.crt -http-key key.pem \
  2. -node-cert node.crt -node-key node-key.pem ~/node.1

Bringing up a second node on the same host, joining it to the first node, using bob’s credentials.

  1. rqlited -auth config.json -http-addr localhost:4003 -http-cert server.crt \
  2. -http-key key.pem -raft-addr :4004 -join localhost:4002 -join-as bob
  3. -node-cert node.crt -node-key node-key.pem ~/node.2

Querying the node, as user mary.

  1. curl -G 'https://mary:secret2@localhost:4001/db/query?pretty&timings' \
  2. --data-urlencode 'q=SELECT * FROM foo'

How can I generate certificates and keys?

There are a lot of resources available on the web explaining how to do so. One popular tool for creating and signing certificates and keys is OpenSSL.

Last modified August 26, 2024: Update _index.md (c2eb9d0)