How to use CUDA with Docker in Ubuntu

Configure your docker app to access that juicy CUDA GPU power.

Assuming you already have your Nvidia drivers and CUDA drivers installed.

I made a guide to help you through the process here.

Installing CUDA with Docker on Ubuntu: A Comprehensive Guide

Docker has emerged as a powerful tool for encapsulating and deploying applications within isolated environments. When combined with CUDA, it opens the door to seamless GPU-accelerated computing for various tasks, from machine learning to scientific simulations.

In this guide, we’ll walk you through the steps to set up CUDA using Docker on Ubuntu, providing a streamlined approach to harnessing the computational prowess of NVIDIA GPUs.

Whether you’re a developer, researcher, or enthusiast, this guide will equip you with the knowledge to efficiently tap into the world of GPU-accelerated computing through Docker and CUDA.

1. Install docker:

   Use the package manager to install docker

   sudo apt update
   sudo apt install docker.io
   

2. Install nvidia-container-toolkit-base:

   Use apt to install the specific package from Nvidia to handle containers

   sudo apt install nvidia-container-toolkit-base
   

3. Generate CDI spec:

   This should include the NVIDIA Container Toolkit CLI (nvidia-ctk) and the version can be confirmed by running:

   sudo nvidia-ctk cdi generate --output=/etc/cdi/nvidia.yaml
   

4. Setup NVIDIA Container Toolkit:

   Setup the package repository and the GPG key:

   curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg   
   curl -s -L https://nvidia.github.io/libnvidia-container/ubuntu22.04/libnvidia-container.list | sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
   

5. Install nvidia-container-toolkit:

   Install the nvidia-container-toolkit package (and dependencies) after updating the package listing:

   sudo apt update
   sudo apt install nvidia-container-toolkit
   

6. Configure Docker to use Nvidia:

   Configure the Docker daemon to recognize the NVIDIA Container Runtime:

   sudo nvidia-ctk runtime configure --runtime=docker  
   sudo systemctl restart docker
   

7. Reboot now  Reboot the computer

8. Verify everything went well:

   At this point, a working setup can be tested by running a base CUDA container:

   sudo docker run --rm --runtime=nvidia --gpus all nvidia/cuda:11.6.2-base-ubuntu20.04 nvidia-smi
   

   Which should output something like this:

   Unable to find image 'nvidia/cuda:11.6.2-base-ubuntu20.04' locally
   11.6.2-base-ubuntu20.04: Pulling from nvidia/cuda
   56e0351b9876: Pull complete 
   0e353182dfa4: Pull complete 
   63add13c711b: Pull complete 
   1210b79751b0: Pull complete 
   eb1e2ff09225: Pull complete 
   Digest: sha256:4b0c83c0f2e66dc97b52f28c7acf94c1461bfa746d56a6f63c0fef5035590429
   Status: Downloaded newer image for nvidia/cuda:11.6.2-base-ubuntu20.04
   Tue Aug 22 16:19:31 2023       
   +---------------------------------------------------------------------------------------+
   | NVIDIA-SMI 535.86.10              Driver Version: 535.86.10    CUDA Version: 12.2     |
   |-----------------------------------------+----------------------+----------------------+
   | GPU  Name                 Persistence-M | Bus-Id        Disp.A | Volatile Uncorr. ECC |
   | Fan  Temp   Perf          Pwr:Usage/Cap |         Memory-Usage | GPU-Util  Compute M. |
   |                                         |                      |               MIG M. |
   |=========================================+======================+======================|
   |   0  NVIDIA GeForce RTX 2070        On  | 00000000:01:00.0  On |                  N/A |
   | N/A   49C    P8              12W / 115W |    989MiB /  8192MiB |     19%      Default |
   |                                         |                      |                  N/A |
   +-----------------------------------------+----------------------+----------------------+
   
   +---------------------------------------------------------------------------------------+
   | Processes:                                                                            |
   |  GPU   GI   CI        PID   Type   Process name                            GPU Memory |
   |        ID   ID                                                             Usage      |
   |=======================================================================================|
   +---------------------------------------------------------------------------------------+
   

Now you’re ready to go!