Build on Ubuntu/Debian

Instructions for building Vitess on your machine for testing and development purposes

If you run into issues or have questions, we recommend posting in our Slack channel, click the Slack icon in the top right to join. This is a very active community forum and a great place to interact with other users.

The following has been verified to work on Ubuntu 19.10 and Debian 10. If you are new to Vitess, it is recommended to start with the local install guide instead.

Install Dependencies

Install Go 1.17+

Download and install Golang 1.17. For example, at writing:

  1. curl -LO https://dl.google.com/go/go1.17.linux-amd64.tar.gz
  2. sudo tar -C /usr/local -xzf go1.17.linux-amd64.tar.gz

Make sure to add go to your bashrc:

  1. # Additions to ~/.bashrc file
  3. # Add go PATH
  4. export PATH=$PATH:/usr/local/go/bin
  6. # Add GOROOT
  7. export GOROOT=/usr/local/go/
  9. # Add GOPATH
  10. export GOPATH=/home/<user>/go

Packages from apt repos

Install dependencies required to build and run Vitess:

  1. # Ubuntu
  2. sudo apt-get install -y mysql-server mysql-client make unzip g++ etcd curl git wget
  4. # Debian
  5. sudo apt-get install -y default-mysql-server default-mysql-client make unzip g++ etcd curl wget

The services mysqld and etcd should be shutdown, since etcd will conflict with the etcd started in the examples, and mysqlctl will start its own copies of mysqld:

  1. sudo service mysql stop
  2. sudo service etcd stop
  3. sudo systemctl disable mysql
  4. sudo systemctl disable etcd

Notes:

  • We will be using etcd as the topology service. The command make tools can also install Zookeeper or Consul for you, which requires additional dependencies.
  • Vitess currently has some additional tests written in Python, but we will be skipping this step for simplicity.

Disable mysqld AppArmor Profile

The mysqld AppArmor profile will not allow Vitess to launch MySQL in any data directory by default. You will need to disable it:

  1. sudo ln -s /etc/apparmor.d/usr.sbin.mysqld /etc/apparmor.d/disable/
  2. sudo apparmor_parser -R /etc/apparmor.d/usr.sbin.mysqld

The following command should return an empty result:

  1. sudo aa-status | grep mysqld

Build Vitess

Navigate to the directory where you want to download the Vitess source code and clone the Vitess GitHub repo:

  1. cd ~
  2. git clone https://github.com/vitessio/vitess.git
  3. cd vitess

Set environment variables that Vitess will require. It is recommended to put these in your .bashrc:

  1. # Additions to ~/.bashrc file
  3. #VTDATAROOT
  4. export VTDATAROOT=/tmp/vtdataroot
  6. # Vitess binaries
  7. export PATH=~/vitess/bin:${PATH}

Build Vitess:

  1. make build

Testing your Binaries

The unit test requires that you first install the following packages:

  1. sudo apt-get install -y ant maven default-jdk zip

You can then install additional components from make tools. If your machine requires a proxy to access the Internet, you will need to set the usual environment variables (e.g. http_proxy, https_proxy, no_proxy) first:

  1. make tools
  2. make unit_test

In addition to running tests, you can try running the local example.

Common Build Issues

Key Already Exists

This error is because etcd was not cleaned up from the previous run of the example. You can manually fix this by running ./401_teardown.sh, removing vtdataroot and then starting again:

  1. Error:  105: Key already exists (/vitess/zone1) [6]
  2. Error:  105: Key already exists (/vitess/global) [6]

MySQL Fails to Initialize

This error is most likely the result of an AppArmor enforcing profile being present:

  1. 1027 18:28:23.462926   19486 mysqld.go:734] mysqld --initialize-insecure failed: /usr/sbin/mysqld: exit status 1, output: mysqld: [ERROR] Failed to open required defaults file: /home/morgo/vitess/vtdataroot/vt_0000000102/my.cnf
  2. mysqld: [ERROR] Fatal error in defaults handling. Program aborted!
  4. could not stat mysql error log (/home/morgo/vitess/vtdataroot/vt_0000000102/error.log): stat /home/morgo/vitess/vtdataroot/vt_0000000102/error.log: no such file or directory
  5. E1027 18:28:23.464117   19486 mysqlctl.go:254] failed init mysql: /usr/sbin/mysqld: exit status 1, output: mysqld: [ERROR] Failed to open required defaults file: /home/morgo/vitess/vtdataroot/vt_0000000102/my.cnf
  6. mysqld: [ERROR] Fatal error in defaults handling. Program aborted!
  7. E1027 18:28:23.464780   19483 mysqld.go:734] mysqld --initialize-insecure failed: /usr/sbin/mysqld: exit status 1, output: mysqld: [ERROR] Failed to open required defaults file: /home/morgo/vitess/vtdataroot/vt_0000000101/my.cnf
  8. mysqld: [ERROR] Fatal error in defaults handling. Program aborted!

The following command disables the AppArmor profile for mysqld:

  1. sudo ln -s /etc/apparmor.d/usr.sbin.mysqld /etc/apparmor.d/disable/
  2. sudo apparmor_parser -R /etc/apparmor.d/usr.sbin.mysqld

The following command should now return an empty result:

  1. sudo aa-status | grep mysqld

If this doesn’t work, you can try making sure all lurking processes are shutdown, and then restart the example again in the /tmp directory:

  1. for process in `pgrep -f '(vtdataroot|VTDATAROOT)'`; do 
  2.  kill -9 $process
  3. done;
  5. export VTDATAROOT=/tmp/vtdataroot
  6. ./101_initial_cluster.sh