Build on CentOS

Instructions for building Vitess on your machine for testing and development purposes

If you run into issues or have questions, we recommend posting in our Slack channel, click the Slack icon in the top right to join. This is a very active community forum and a great place to interact with other users.

The following has been verified to work on CentOS 7. If you are new to Vitess, it is recommended to start with the local install guide instead.

Install Dependencies

Install Go 1.12+

Download and install the latest version of Golang. For example, at writing:

  1. curl -O https://dl.google.com/go/go1.13.3.linux-amd64.tar.gz
  2. sudo tar -C /usr/local -xzf go1.13.3.linux-amd64.tar.gz

Make sure to add go to your bashrc:

  1. export PATH=$PATH:/usr/local/go/bin

Packages from CentOS repos

The MariaDB version included with CentOS 7 (5.5) is not supported by Vitess. First install the MySQL 5.7 repository from Oracle:

  1. sudo yum localinstall -y https://dev.mysql.com/get/mysql57-community-release-el7-9.noarch.rpm
  2. sudo yum install -y mysql-community-server

Install additional dependencies required to build and run Vitess:

  1. sudo yum install -y make unzip g++ etcd curl git wget

Notes:

  • Vitess currently has some tests written in Python, but this dependency can be avoided by running the tests in Docker (recommended).
  • We will be using etcd as the topology service. The bootstrap.sh script can also install Zookeeper or Consul for you, which requires additional dependencies.

Disable SELinux

SELinux will not allow Vitess to launch MySQL in any data directory by default. You will need to disable it:

  1. sudo setenforce 0

Install Docker

Running the testsuite requires that you install Docker. Should you decide to skip this step, you will still be able to compile and run Vitess.

Build Vitess

Navigate to the directory where you want to download the Vitess source code and clone the Vitess GitHub repo. After doing so, navigate to the src/vitess.io/vitess directory.

  1. mkdir -p ~/vitess
  2. cd ~/vitess
  3. git clone https://github.com/vitessio/vitess.git \
  4.     src/vitess.io/vitess
  5. cd src/vitess.io/vitess

Set environment variables that Vitess will require. It is recommended to put these in your .bashrc:

  1. # Additions to ~/.bashrc file
  3. # Add go PATH
  4. export PATH=$PATH:/usr/local/go/bin
  6. # Vitess
  7. export VTROOT=~/vitess
  8. export VTTOP=~/vitess/src/vitess.io/vitess
  9. export VTDATAROOT=~/vitess/vtdataroot
  10. export PATH=${VTROOT}/bin:${PATH}

Run bootstrap.sh script to download additional dependencies. If your machine requires a proxy to access the Internet, you will need to set the usual environment variables (e.g. http_proxy, https_proxy, no_proxy):

  1. BUILD_PYTHON=0 BUILD_JAVA=0 ./bootstrap.sh

Build Vitess:

  1. source ./dev.env
  2. make build

Testing your Binaries

Run the included local example:

  1. cd examples/local
  2. ./101_initial_cluster.sh

You should see the following:

  1. $ ./101_initial_cluster.sh 
  2. [morgo@localhost local]$ ./101_initial_cluster.sh
  3. enter etcd2 env
  4. add /vitess/global
  5. add /vitess/zone1
  6. add zone1 CellInfo
  7. etcd start done...
  8. enter etcd2 env
  9. Starting vtctld...
  10. Access vtctld web UI at http://localhost:15000
  11. Send commands with: vtctlclient -server localhost:15999 ...
  12. enter etcd2 env
  13. Starting MySQL for tablet zone1-0000000100...
  14. Starting MySQL for tablet zone1-0000000101...
  15. Starting MySQL for tablet zone1-0000000102...
  16. Starting vttablet for zone1-0000000100...
  17. Access tablet zone1-0000000100 at http://localhost:15100/debug/status
  18. Starting vttablet for zone1-0000000101...
  19. Access tablet zone1-0000000101 at http://localhost:15101/debug/status
  20. Starting vttablet for zone1-0000000102...
  21. Access tablet zone1-0000000102 at http://localhost:15102/debug/status
  22. Waiting for tablets to be listening...
  23. Tablets up!
  24. W1028 09:30:40.634534    7333 main.go:64] W1028 15:30:40.630786 reparent.go:182] master-elect tablet zone1-0000000100 is not the shard master, proceeding anyway as -force was used
  25. W1028 09:30:40.634792    7333 main.go:64] W1028 15:30:40.631267 reparent.go:188] master-elect tablet zone1-0000000100 is not a master in the shard, proceeding anyway as -force was used
  26. New VSchema object:
  27. {
  28.   "tables": {
  29.     "corder": {
  31.     },
  32.     "customer": {
  34.     },
  35.     "product": {
  37.     }
  38.   }
  39. }
  40. If this is not what you expected, check the input data (as JSON parsing will skip unexpected fields).
  41. enter etcd2 env
  42. Waiting for vtgate to be up...
  43. vtgate is up!
  44. Access vtgate at http://localhost:15001/debug/status

You can continue the remaining parts of this example by following the local get started guide.

Full testsuite

To run the testsuite in Docker:

  1. make docker_test flavor=mysql57

Running the full suite currently takes 2+ hours to complete.

Common Build Issues

Key Already Exists

This error is because etcd was not cleaned up from the previous run of the example. You can manually fix this by running ./401_teardown.sh and then start again:

  1. Error:  105: Key already exists (/vitess/zone1) [6]
  2. Error:  105: Key already exists (/vitess/global) [6]

MySQL Fails to Initialize

This error is most likely the result of SELinux enabled:

  1. 1027 18:28:23.462926   19486 mysqld.go:734] mysqld --initialize-insecure failed: /usr/sbin/mysqld: exit status 1, output: mysqld: [ERROR] Failed to open required defaults file: /home/morgo/vitess/vtdataroot/vt_0000000102/my.cnf
  2. mysqld: [ERROR] Fatal error in defaults handling. Program aborted!
  4. could not stat mysql error log (/home/morgo/vitess/vtdataroot/vt_0000000102/error.log): stat /home/morgo/vitess/vtdataroot/vt_0000000102/error.log: no such file or directory
  5. E1027 18:28:23.464117   19486 mysqlctl.go:254] failed init mysql: /usr/sbin/mysqld: exit status 1, output: mysqld: [ERROR] Failed to open required defaults file: /home/morgo/vitess/vtdataroot/vt_0000000102/my.cnf
  6. mysqld: [ERROR] Fatal error in defaults handling. Program aborted!
  7. E1027 18:28:23.464780   19483 mysqld.go:734] mysqld --initialize-insecure failed: /usr/sbin/mysqld: exit status 1, output: mysqld: [ERROR] Failed to open required defaults file: /home/morgo/vitess/vtdataroot/vt_0000000101/my.cnf
  8. mysqld: [ERROR] Fatal error in defaults handling. Program aborted!

Python Errors

The end-to-end test suite currently requires Python 2.7. We are working on removing this dependency, but in the mean time you can run tests from within Docker. The MySQL 5.7 container provided includes the required dependencies:

  1. make docker_test flavor=mysql57

No .installed_version file

This error indicates that you have not put the required vitess environment variables in your .bashrc file:

  1. enter etcd2 env
  2. cat: /dist/etcd/.installed_version: No such file or directory

Make sure the following variables are defined:

  1. export VTROOT=~/vitess
  2. export VTTOP=~/vitess/src/vitess.io/vitess
  3. export VTDATAROOT=~/vitess/vtdataroot
  4. export PATH=${VTROOT}/bin:${PATH}

Cannot create dir /etcd

This indicates that the environment variable VTDATAROOT is not defined, and you have not put the required vitess environment variables in your .bashrc file:

  1. ./101_initial_cluster.sh
  2. enter etcd2 env
  3. mkdir: cannot create directory ‘/etcd’: Permission denied

Make sure the following variables are defined:

  1. export VTROOT=~/vitess
  2. export VTTOP=~/vitess/src/vitess.io/vitess
  3. export VTDATAROOT=~/vitess/vtdataroot
  4. export PATH=${VTROOT}/bin:${PATH}