Build TensorFlow from Source Code using Docker

To contribute to TensorFlow, we need to build TensorFlow from source code. The official guide is great. However, it interleaves the native building process and that using Docker and makes it confusing because packages needed by the former are not by the latter. Also, we found some useful tricks to start Docker containers in practices that are not in the official guide. Hence this document.

Build the Pip Package in TensorFlow Development Container

On either Mac or Linux, or any other OS, we don’t have to install and configure the building tools; instead, we can use a Docker image where all tools have been installed and properly configured.

• Get the Docker image containing all the building tools:

docker pull tensorflow/tensorflow:latest-devel

• Then, let’s get the source code. On any OS, please install git using the native package manager. For example, on Ubuntu, please

sudo apt-get install git

or, on Mac,

brew install git

Then, use the git just installed, let’s clone tensorflow source code:

git clone https://github.com/tensorflow/tensorflow
cd tensorflow

By default, we will be on the master branch. Feel free to do you change in your feature branches, or switch to a release branch, for example:

git checkout v1.11.0
git checkout -b v1.11.0

• Then, let us start a Docker container running the tensorflow/tensorflow:latest-devel image:

docker run --rm -it \
   -w /tensorflow \
   -v $PWD:/tensorflow \
   -v $HOME/.cache:/root/.cache \
   -e "HOST_PERMS=$(id -u):$(id -g)" \
   tensorflow/tensorflow:latest-devel \
   /bin/bash

• -w /tensorflow brings us to the /tensorflow directory in the container once after we start it.
• -v $PWD:/tensorflow maps the current directory, which is the just cloned TensorFlow source directory on the host, to /tensorflow in the container.
• -v $HOME/.cache:/root/.cache maps the Bazel temporary directory on the host into the container, so the intermediate files generated by Bazel running in the container are actually saved on the host. This allows us to interrupt the container during the build and restart it later to resume the building.
• e "HOST_PERMS=$(id -u):$(id -g)" passes the user identification on the host into the container as an environment variable. We can reset the mode of files generated in the container to this user identity.
  • From now on, we will be working in the container. Let us first configure the building:

./configure

Usually, I would simply choose all the default options by hitting enter all the way down.

• Build the pip package builder:

bazel build --config=opt //tensorflow/tools/pip_package:build_pip_package

• Build the pip package and save it into /tensorflow:

./bazel-bin/tensorflow/tools/pip_package/build_pip_package /tensorflow

• Change the generated wheel file into the usual mode as on the host:

chown $HOST_PERMS /mnt/tensorflow-*.whl

Install the Pip Pacakge in TensorFlow Development Container

• Let’s now try the new pip package. First, we need to uninstall the current tensorflow pip package, then we install the newly built one:

pip uninstall tensorflow
pip install /tensorflow/tensorflow-*.whl

• Now, we can verify if the new package works. First, we need to switch to a directory out from /tensorflow, so we don’t import from the source directory:

cd /tmp  # other directories also work

then, we can try import the newly installed tensorflow package and verify it:

python
>>> import tensorflow
>>> print(tensorflow.__version__)

• Now, let us quit from Python and from the Docker container. We should see the tensorflow-*.whl file on the host in the current directory.

Install the Pip Package in a Clean Python Package

After we quit from the development container, we should see the wheel file in the TensorFlow source directory on the host. Now, we can start a Python container and install the Pip package in it.

• Start the Python container

docker run --rm -it -v $PWD:/tensorflow python:2.7 bash

• Install the pip pacakge in the container

pip install /tensorflow/tensorflow*.whl

• Try TensorFlow by starting Python in the container

python
>>> import tensorflow as tf
>>> print(tf.__version__)