Building Your Own GPU-Powered Machine Learning Server Part 2. Ubuntu and Nvidia

Why Ubuntu and Nvidia?

Ubuntu – You need a linux distro to run nvidia-docker in order to build and launch GPU based docker images like Keras and Tensorflow-GPU.  At the time of this writing, OSx and Windows are NOT supported by nvidia-docker.

Nvidia – Nvidia’s CUDA and cuDNN are fundamental in many Neural Net libraries like TensorFlow-GPU and PyTorch.

Docker* – Docker is awesome.  I can’t say enough about it.  There are MANY options out there for hosting virtual development environments, but docker is a new breed.  The way it hooks into the host kernel and the availability of docker images on dockerhub make it perfect for Data Science.

If you have read or completed part 1 of this series, you have a modern Intel Motherboard/CPU combo, at least 8 gigs of ram and most importantly a solid GPU then you’re set for the following instructions.  You’ll also need the following:

  • A computer to create your Ubuntu installer USB
  • A 1GB USB stick
    • If you are having trouble with the install below, try a different USB stick or a different USB port.
  • An internet connection

Step 1 – Downloading Ubuntu

Xenial 16.0.4 Desktop (or Server) .iso using UEFI

Head over to and download either server or desktop version 16.0.4 64 bit.

Step 2 – Create Bootable UEFI USB stick

UEFI is quickly replacing outdated BIOS based systems.  I won’t get into the details here, but its better.  If you have a newer machine with a newer video card, you are going to want to install Ubuntu in UEFI mode, NOT legacy or BIOS mode.  For more info on UEFI check this link.

Format your USB stick to Fat32 with GPT (MBR may work as well ).

“Make the boot drive with for example mkusb, Disks, the new Ubuntu Startup Disk Creator (version 0.3.2) or the Win32 Disk Imager. These tools clone the iso file and are likely to succeed. Some ‘grub and ISO’ systems work in UEFI mode, others work only in BIOS mode. Syslinux does not work in UEFI mode.” – Ubuntu

Step 3 – System Settings Before Install

Getting into your motherboard’s UEFI system settings is usually done by spamming the DEL key.   Check the following settings in the BOOT menu:

  • Either disable CSM or make sure your boot devices are set to UEFI first.
    • CSM makes older BIOS reliant hardware and OSes work on newer UEFI systems.   Cases where you might need CSM:
      • Your system has a 32 bit OS installed
      • You system has an MBR OS installation
      • You are using hardware that does not have UEFI support – (older graphics cards or controllers)
      • Your system will usually turn CSM on on its own if it encounters hardware incompatibilities.
  • If the installation below initially fails, set Intel Speed Step to disabled. – (this will be in CPU SETTINGS, not BOOT menu)
  • Set Secure Boot to Other OS even if you have Windows installed.

Step 4 – Installing Ubuntu

We have our Fat32 formatted Ubuntu.iso USB stick and our System Settings are dialed in.

  • As a precautionary measure, unplug all but the drive you intend to install Ubuntu on.
    • Ubuntu’s Partition Creation Screen can be a bit confusing and the last thing you want to do is format the wrong drive.
  • Plug in the stick and boot your system and get ready to spam your boot selection screen key.
    • This is usually f8 or f12 on Asus motherboards
    • If you can’t find your boot selection key you can usually find the manual boot selection override in the Boot Menu of your UEFI System Settings.
  • IMPORTANT – When you see the boot selection screen, choose the UEFI Boot Method of your Ubuntu Installer.

  • If you did not disable CSM, you will see both a UEFI and standard USB Stick boot option.  Use the UEFI version.
    • Operating systems create boot tables in whatever method they were installed with.  If you launch an OS installer in UEFI, it will install with a GPT  and an EFI partition.  If you install in Non-UEFI mode, it will use an MBR partition table and you will need either CSM or a legacy bios system to boot.  This applies to Windows as well.  There is no indication in the installers to tell you which version you are installing, they look completely identical.
  • Pick your language and click install Ubuntu.
  • Download updates while installing (This will only work if Ubuntu installer recognized your Network Adapter)
  • Proceed and select “Do Something Else”

  • The next screen can be confusing if you didn’t unplug your other drives.
    • Side-note:  If you would like to install on the same drive as an existing Windows installation, check out this link.
  • Your partition table may differ to the example below.

  • Understanding the partitions: (click this link for detailed explanations)
    • Swapfile – An area of the Physical Disk reserved for use as memory if you run out of Physical Memory.
      • I only recommend this if you have less that 16GB of RAM.
    • Root – An area of the Physical Disk reserved for your kernel, boot files, system files, command-line utilities, libraries, system-wide configuration files and logs.
    •  Home – An area of the Physical Disk for your user’s files: documents, images, music and videos. It’s much more like Users folder in Windows.
    • Tmp – optional.  Stores temporary files up to the max size of the partition.  If you don’t create this partition a tmp folder will be made.
    • Unless you are 100% sure you want to delete a partition DON’T TOUCH THE OTHER PARTITIONS!
  • Click on your blank disk (either new or formatted) and click “New Partition Table”
  • You should see now see “/free space”
  • Select “/free space” and click the “+”.

  • Create a Swapfile area like so: If you have 16GB of RAM or more,  you can skip this step.

  • Make it between 512MB and 1024MB.  Keep in mind, this is not free RAM.  This is an area to store stuff when RAM gets used up.
  • Create the Root partition like so.  20GB should be enough but you can make this larger if you’d like.

  • Create the Home partition with the rest of the free space.  I recommend Ext4 journaling file system with SSDs.
    • Take extra care to select the proper mount point.
  • When all is said and done, you should have something that looks like this.

  • Click install now.
  • Set username and password and you are home free.

Step 5 – Downloading Nvidia Drivers, CUDA and Patch

  • Open Firefox and head over to and create an account.
  • Once complete, head to this link and download the CUDA Toolkit 9.2 sh script using the following selections. (save it in tmp folder or tmp partition)

  • Download both the base installer and patch:

Step 6 – Preparing For Nvidia Kernel Install

  • Now things get a bit tricky.
    • If you installed Ubuntu Desktop, it comes with a GUI, and a GUI needs a graphics kernel.  Ubuntu handles that by installing a kernel called “Nouveau” which handles most Nvidia cards.  We’re going to need to get rid of this in order to install the Nvidia Drivers and Cuda Toolkit.  We’re also going to have to shut down the GUI and do this from command line to free up the kernels we are removing*.
  • Hit ControlAltF1.
  • You should now be in a TTY bash shell command line.
  • We need to block the Nouveau Kernel and we do that like so:
    • Create a file at /etc/modprobe.d/blacklist-nouveau.conf with the following contents:
      blacklist nouveau
      options nouveau modeset=0
    • Regenerate the kernel initramfs:

    • $ sudo update-initramfs -u
    • Restart
  • When you arrive back at the Ubuntu login screen, your screen resolution and color depth may look strange.  This is good.
  • Hit Control-Alt-F1 again.
  • Shutdown the GUI by entering:
    • sudo service lightdm stop
    • sudo init 3

Step 7 – Installing Nvidia Kernel and CUDA Toolkit 9.2

  • Start the installer by entering:
    • cd /directory/where/cuda/downloads/are
    • sudo sh
    • Follow prompts and complete installation of drivers, toolkit and optionally, samples
  • Repeat the above process for the Patch file:
    • cd /directory/where/cuda/downloads/are
    • sudo sh
  • Hopefully, everything worked up to this point.  We’re not done yet.  We need to add the new package and kernel paths to our bash profile.  Copy and run the code snippets below.
    • echo 'export PATH=/usr/local/cuda-9.2/bin${PATH:+:${PATH}}' >> ~/.bashrc
    • echo 'export LD_LIBRARY_PATH=/usr/local/cuda-9.2/lib64 ${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}' >> ~/.bashrc
  • Lets reboot and test.  On login screen, resolution should be solid again.
  • Once you reboot, run the following commands in a terminal window:
    • nvidia-smi
    • nvcc --version
  • Depending on your GPU, you should be greeted with something similar to the following.

    nvcc: NVIDIA (R) Cuda compiler driver
    Copyright (c) 2005-2018 NVIDIA Corporation
    Built on Wed_Apr_11_23:16:29_CDT_2018
    Cuda compilation tools, release 9.2, V9.2.88

  • Congratulations! You have Ubuntu 16.0.4 Running with the following configuration:
    • 64Bit
    • UEFI
    • Nvidia 369.26 Kernel
    • CUDA 9.2.88 Kernel

Leave a Reply

Your email address will not be published. Required fields are marked *