How to build Tensorflow-GPU from source

By Michael Yee

In this blog, I will be describing the step by step instructions to install TensorFlow-GPU version 1.5.0-rc1 into a Python 3.5 Virtualenv environment on Ubuntu 16.04 (64bit).

Step 1: Pre-installation

a) Verify the system has a CUDA-capable GPU

$ lspci | grep -i nvidia

If you do not see any settings, update the PCI hardware database that Linux maintains by entering update-pciids (generally found in /sbin) at the command line and rerun the previous lspci command.

If your graphics card is from NVIDIA and it is listed in, your GPU is CUDA-capable.

NOTE: Write down the Compute Capability value for later use in section: Installing TensorFlow from sources - Configure the installation

b) Verify your NVIDIA graphics driver is 384.81 or newer

$ nvidia-smi

You should see the driver version in the top of the output similar to the following:

| NVIDIA-SMI 390.12                 Driver Version: 390.12                |
| GPU  Name        Persistence-M| Bus-Id   Disp.A | Volatile Uncorr. ECC | 
| Fan  Temp  Perf  Pwr:Usage/Cap|    Memory-Usage | GPU-Util  Compute M. |
|   0  GeForce GTX 108...  Off  | 000:01:00.0  On |                  N/A |
|  0%   30C    P8    17W / 275W |    522MiB / 11175MiB |      0% Default |

To install up-to-date NVIDIA graphics drivers, go to NVIDIA download drivers and follow the instructions

c) Verify your version of Linux is supported

$ uname -m && cat /etc/*release

You should see output similar to the following:

VERSION="16.04.3 LTS (Xenial Xerus)"
PRETTY_NAME="Ubuntu 16.04.3 LTS"

The first and most important line containing x86_64, indicates you are running on a 64-bit system which is supported by CUDA. The CUDA Development Tools are only supported on some specific distributions of Linux. These are listed in the CUDA Toolkit release notes.

d) Update all the Ubuntu things!

$ sudo apt-get update && sudo apt-get upgrade && sudo apt-get dist-upgrade

e) Verify your kernel version is supported and the correct development packages are installed

$ uname -r

You should see output similar to the following:


NOTE: NVIDIA recommend a kernel version of 4.4, otherwise the NVIDIA driver may not start.  

The kernel headers and development packages for the currently running kernel can be installed with:

$ sudo apt-get install linux-headers-$(uname -r)

f) Install dependencies (you need this things to build from source)

$ sudo apt-get install build-essential cmake python3-dev python3-numpy python3-pip python3-wheel unzip

libcupti (NVIDIA CUDA Profiler Tools Interface development files)

$ sudo apt-get install libcupti-dev
$ echo 'export LD_LIBRARY_PATH=/usr/local/cuda/extras/CUPTI/lib64:$LD_LIBRARY_PATH' >> ~/.bashrc

Java JDK (Bazel requires Java JDK 8 or later)

$ sudo apt-get install openjdk-8-jdk
$ echo "deb [arch=amd64] stable jdk1.8" | sudo tee /etc/apt/sources.list.d/bazel.list

Bazel (Bazel is an open-source build and test tool)

$ curl | sudo apt-key add -
$ sudo apt-get update && sudo apt-get install bazel
$ sudo apt-get upgrade bazel

Step Two: CUDA

Download CUDA

$ wget

Install CUDA

a) Install repository meta-data

$ sudo dpkg -i cuda-repo-ubuntu1604_9.1.85-1_amd64.deb

b) Installing the CUDA public GPG key using network repo:

$ sudo apt-key adv --fetch-keys

c) Update the Apt repository cache

$ sudo apt-get update

d) Install CUDA

$ sudo apt-get install cuda

CUDA environment set-up

a) The PATH variable needs to include /usr/local/cuda-9.1/bin and the LD_LIBRARY_PATH variable needs to contain /usr/local/cuda-9.1/lib64

Edit ~/.bashrc and save the following to the end of the file:

export PATH=/usr/local/cuda-9.1/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/usr/local/cuda-9.1/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}

b) Restart system

Step Three: cuDNN

Download cuDNN

In order to download the required cuDNN files, ensure you are registered for the NVIDIA Developer Program.

Go to: NVIDIA cuDNN home page, login and download the following files:

CuDNN Files

Install cuDNN

a) Navigate to the directory containing cuDNN Debian files

b) Install the runtime library

$ sudo dpkg -i libcudnn7_7.0.3.11-1+cuda9.0_amd64.deb

c) Install the developer library

$ sudo dpkg -i libcudnn7-dev_7.0.3.11-1+cuda9.0_amd64.deb

d) Install the code samples and the cuDNN Library User Guide

$ sudo dpkg -i libcudnn7-doc_7.0.3.11-1+cuda9.0_amd64.deb

Verify the installation

To verify that cuDNN is installed and is running properly, compile the mnistCUDNN sample located in the /usr/src/cudnn_samples_v7 directory in the debian file.

a) Copy the cuDNN sample to a writable path and go to that folder

$ cp -r /usr/src/cudnn_samples_v7/ $HOME
$ cd $HOME/cudnn_samples_v7/mnistCUDNN

b) Compile the mnistCUDNN sample

$ make clean && make

c) Run the mnistCUDNN sample

$ ./mnistCUDNN

If cuDNN is properly installed and running, you will see a message similar to the following:

Test passed!

NOTE: cuDNN v7 can coexist with previous versions of cuDNN, such as v5 or v6

Installing TensorFlow from sources

Download and unzip TensorFlow

a) Create and navigate to a directory to containing TensorFlow files

b) Download TensorFlow

$ wget

c) Unzip TensorFlow

$ unzip

d) Navigate into the unzipped TensorFlow folder

$ cd tensorflow-1.5.0-rc1 

Configure the installation

The root of the source tree contains a bash script named configure. This script asks you to identify the pathname of all relevant TensorFlow dependencies and specify other build configuration options such as compiler flags. You must run this script prior to creating the pip package and installing TensorFlow.

The following is an example execution of the configure script:

    $ ./configure

    You have bazel 0.9.0 installed.
    Please specify the location of python. [Default is /home/michael/python3.5-venv/bin/python]: /home/michael/python3.5-venv/bin/python
    Traceback (most recent call last):
      File "<string>", line 1, in <module>
    AttributeError: module 'site' has no attribute 'getsitepackages'
    Found possible Python library paths:
    Please input the desired Python library path to use.  Default is [/home/michael/python3.5-venv/lib/python3.5/site-packages]

    Do you wish to build TensorFlow with jemalloc as malloc support? [Y/n]: Y
    jemalloc as malloc support will be enabled for TensorFlow.

    Do you wish to build TensorFlow with Google Cloud Platform support? [Y/n]: Y
    Google Cloud Platform support will be enabled for TensorFlow.

    Do you wish to build TensorFlow with Hadoop File System support? [Y/n]: Y
    Hadoop File System support will be enabled for TensorFlow.

    Do you wish to build TensorFlow with Amazon S3 File System support? [Y/n]: Y
    Amazon S3 File System support will be enabled for TensorFlow.

    Do you wish to build TensorFlow with XLA JIT support? [y/N]: N
    No XLA JIT support will be enabled for TensorFlow.

    Do you wish to build TensorFlow with GDR support? [y/N]: N
    No GDR support will be enabled for TensorFlow.

    Do you wish to build TensorFlow with VERBS support? [y/N]: N
    No VERBS support will be enabled for TensorFlow.

    Do you wish to build TensorFlow with OpenCL SYCL support? [y/N]: N
    No OpenCL SYCL support will be enabled for TensorFlow.

    Do you wish to build TensorFlow with CUDA support? [y/N]: Y
    CUDA support will be enabled for TensorFlow.

    Please specify the CUDA SDK version you want to use, e.g. 7.0. [Leave empty to default to CUDA 9.0]: 9.1

    Please specify the location where CUDA 9.1 toolkit is installed. Refer to for more details. [Default is /usr/local/cuda]: /usr/local/cuda

    Please specify the cuDNN version you want to use. [Leave empty to default to cuDNN 7.0]: 7.0.5

    Please specify the location where cuDNN 7.0.5 library is installed. Refer to for more details. [Default is /usr/local/cuda]:/usr/lib/x86_64-linux-gnu

    Please specify a list of comma-separated Cuda compute capabilities you want to build with.
    You can find the compute capability of your device at:
    Please note that each additional compute capability significantly increases your build time and binary size. [Default is: 6.1]6.1

    Do you want to use clang as CUDA compiler? [y/N]: N
    nvcc will be used as CUDA compiler.

    Please specify which gcc should be used by nvcc as the host compiler. [Default is /usr/bin/gcc]: /usr/bin/gcc

    Do you wish to build TensorFlow with MPI support? [y/N]: N
    No MPI support will be enabled for TensorFlow.

    Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified [Default is -march=native]: -march=native

    Add "--config=mkl" to your bazel command to build with MKL support.
    Please note that MKL on MacOS or windows is still not supported.
    If you would like to use a local MKL instead of downloading, please set the environment variable "TF_MKL_ROOT" every time before build.

    Would you like to interactively configure ./WORKSPACE for Android builds? [y/N]: N
    Not configuring the WORKSPACE for Android builds.

    Configuration finished

Build the pip package

To build a pip package for TensorFlow-GPU, invoke the following command:

    $ bazel build --config=opt --config=cuda --incompatible_load_argument_is_label=false //tensorflow/tools/pip_package:build_pip_package

    NOTE: This process will take about an hour, so open up a beer and play your favourite video game for a bit

The bazel build command builds a script named build_pip_package. Running this script as follows will build a .whl file within the tensorflow_pkg folder:

$ bazel-bin/tensorflow/tools/pip_package/build_pip_package tensorflow_pkg

Install the pip package

a) Create and activate a virtual environment

b) Invoke pip install to install that pip package

$ sudo pip install tensorflow_pkg/tensorflow*

Validate your installation

a) Invoke python:

$ python

b) Enter the following short program inside the python interactive shell:

    import tensorflow as tf
    hello = tf.constant('Hello, TensorFlow!')
    sess = tf.Session()

If the system outputs the following, then you are ready to begin writing TensorFlow programs:

Hello, TensorFlow!


For more information, please visit the following links:

a) CUDA toolkit documentation

b) Deep learning SDK documentation

c) Installing TensorFlow from sources

Written on January 22, 2018