Build from source

How to build rqlite from source

rqlite can be compiled for a wide variety of operating systems and platforms.

Building rqlite

Building rqlite requires Go 1.18 or later. gvm is a great tool for installing and managing your versions of Go.

One goal of rqlite is to keep the build process as simple as possible, to aid development and debugging. Download, build, and run rqlite like so (tested on 64-bit Ubuntu 20.04, macOS, and Windows):

  1. mkdir rqlite # Or any directory of your choice.
  2. cd rqlite/
  3. export GOPATH=$PWD
  4. mkdir -p src/github.com/rqlite
  5. cd src/github.com/rqlite
  6. git clone https://github.com/rqlite/rqlite.git
  7. cd rqlite
  8. go install ./...
  9. $GOPATH/bin/rqlited ~/node.1

This starts a rqlite server listening on localhost, port 4001. This single node automatically becomes the Leader.

To rebuild and run, perhaps after making some changes to the source, do something like the following:

  1. cd $GOPATH/src/github.com/rqlite/rqlite
  2. go install ./...
  3. $GOPATH/bin/rqlited ~/node.1

Linking behavior

Note that the above commands build a version of rqlited that dynamically links libc. When officially released, rqlited for certain CPU architectures will statically link all code, including libc. The SQLite source code is always part of rqlite however, and you never need any SQLite library (often named libsqlite3) on your host machine.

Raspberry Pi

The process outlined above will work for Linux, OSX, and Windows. For Raspberry Pi, check out this GitHub issue.

Protobuf code generation

This step is not necessary unless you are making changes to protobuf definitions.

Ensure you have the required tools installed, and that GOPATH is set.

  1. go install google.golang.org/protobuf/cmd/protoc-gen-go
  2. export GOBIN=$GOPATH/bin
  3. export PATH=$PATH:$GOBIN
  4. export DEST_DIR=$GOPATH/src
  5. export SRC_DIR=$GOPATH/src/github.com/rqlite/rqlite
  6. export PROTO_PATH=$GOPATH/src/github.com/rqlite/rqlite
  7. for f in command/command.proto cluster/message.proto; do
  8.   protoc -I=$SRC_DIR --proto_path=$PROTO_PATH --go_out=$DEST_DIR $SRC_DIR/$f
  9. done

Speeding up the build process

It can be rather slow to rebuild rqlite, due to the repeated compilation of the SQLite source code. You can compile and install the SQLite libary once, so subsequent builds are much faster. To do so, execute the following commands:

  1. cd $GOPATH/src/github.com/rqlite/rqlite
  2. go install github.com/rqlite/go-sqlite3

Cloning a fork

If you wish to work with fork of rqlite, your own fork for example, you must still follow the directory structure above. But instead of cloning the main repo, instead clone your fork. You must fork the project if you want to contribute upstream.

Follow the steps below to work with a fork:

  1. export GOPATH=$HOME/rqlite
  2. mkdir -p $GOPATH/src/github.com/rqlite
  3. cd $GOPATH/src/github.com/rqlite
  4. git clone git@github.com:<your Github username>/rqlite

Retaining the directory structure $GOPATH/src/github.com/rqlite is necessary so that Go imports work correctly.

Testing

Be sure to run the unit test suite before opening a pull request. An example test run is shown below.

  1. $ cd $GOPATH/src/github.com/rqlite/rqlite
  2. $ go test ./...
  3. ?       github.com/rqlite/rqlite       [no test files]
  4. ok      github.com/rqlite/rqlite/auth  0.001s
  5. ?       github.com/rqlite/rqlite/cmd/rqlite    [no test files]
  6. ?       github.com/rqlite/rqlite/cmd/rqlited   [no test files]
  7. ok      github.com/rqlite/rqlite/db    0.769s
  8. ok      github.com/rqlite/rqlite/http  0.006s
  9. ok      github.com/rqlite/rqlite/store 6.117s
  10. ok      github.com/rqlite/rqlite/system_test   7.853s

Development philosophy

Clean commit histories

If you open a pull request, please ensure the commit history is clean. Squash the commits into logical blocks, perhaps a single commit if that makes sense. What you want to avoid is commits such as “WIP” and “fix test” in the history. This is so we keep history on master clean and straightforward.

Third-party libraries

Please avoid using libaries other than those available in the standard library, unless necessary. This requirement is relaxed somewhat for software other than rqlite node software itself. To understand why this approach is taken, check out this post.

Last modified November 23, 2023: Update _index.md (726cd32)