Building your own GPU-Powered Machine Learning Server Part 3 – Nvidia-Docker and Dockerfile

Why Docker?

Docker is what I learned to use early on in my Dat-Sci journey.  I’ve compared it to conda and virtualenv (apples to oranges) and find the advantages worth the learning curve.

  • Use QUALITY pre-made images
  • Completely isolate your images and volumes from your local environment
  • Get a conda equipped, scikit notebook running in 5 min flat.
  • DockerHub
  • Add SQL or Mongo servers with one docker command

Why Build a New Image?

Unless I suck at googling (which I’m pretty sure I don’t), you’d be hard-pressed to find a solid python Dat-Sci image with all of the bells and whistles of the Jupyter Notebooks image plus CUDA and cuDNN with Python3.6.  So I figured this is a good opportunity to learn and build one.

My requirements:

  • Python 3.6 (I built a bunch of libraries that use f string formatting which I didn’t know is new in 3.6)
  • Jupyter Notebooks
  • Common Dat-Sci python libraries preinstalled
  • Nvidia CUDA and cuDNN support
  • TensorFlow-GPU
  • Keras

So after a ton of trial and error, I’ve completed testing my Dockerfile and am uploading my docker image to DockerHub (as I write this).  Be warned, it’s close to 6GB.


Step 1 – Installing Docker

Credit – DigitalOcean’s Guide
  • Add docker GPG key for official docker repository
  • curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -
  • Add docker repo to apt-get
  • sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"
  • Update apt-get
  • sudo apt-get update
  • Make sure to install from docker repo
  • apt-cache policy docker-ce
  • Install docker
  • sudo apt-get install -y docker-ce
  • Check docker install
  • sudo systemctl status docker
  • Add docker to User group (or else you’ll have to type sudo for every docker command)
  • sudo usermod -aG docker ${USER}
  • Refresh users and groups
  • su - ${USER}
  • Test your docker install by running:
  • docker run hello-world
  • “Do a little dance, make a little love”

Step 2 – Installing nvidia-docker

Docker containers are hardware agnostic, meaning that they natively can’t use anything outside of base system hardware (CPU, RAM, HDs), so in order for your docker containers to make use of your fancy pants video card, you are going to need nvidia-docker, a docker library/wrapper written by Nvidia to allow containers to utilize your GPU.

If you’ve followed along with everything we’ve done in Step 2 of this series, the install should be straightforward.

  • Add nvidia-docker GPG key
  • curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | \
      sudo apt-key add -
  • Add nvidia-docker repo
  • curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | \
      sudo tee /etc/apt/sources.list.d/nvidia-docker.list
  • Update apt-get
  • sudo apt-get update
  • Install nvidia-docker
  • sudo apt-get install -y nvidia-docker2
  • Restart docker daemon
  • sudo pkill -SIGHUP dockerd
  • Test nvidia-docker (be aware, the nvidia-cuda image is 1.2 gigs)
  • docker run --runtime=nvidia --rm nvidia cuda nvidia-smi
  • Do another dance

Step 3 – Run a CUDA Based Container

IMPORTANT:  I struggled for almost a day getting my containers to run nvidia-smi properly because I foolishly forgot to start them using “nvidia-docker run…”.

Explore some GPU based images on dockerhub OR use the image I meticulously crafted by running this command:

nvidia-docker run -it -p 8888:8888 -p 6006:6006 dimak415/cudaconda

Do all the dances and make all the love.

Leave a Reply

Your email address will not be published. Required fields are marked *