NVIDIA Driver and CUDA Installation on Ubuntu Linux¶

In this article

Installing NVIDIA Drivers

Installing CUDA

Installing NVIDIA modules for Docker

One-Click Installation of Drivers and CUDA 12

This instructional guide details the procedure for installing NVIDIA graphics card drivers and CUDA on the subsequent operating systems: Ubuntu 22.04, Ubuntu 24.04.

Attention

For proper operation of Tesla series graphics cards (e.g., NVIDIA Tesla T4), ensure that the server's BIOS has the parameter 'above 4G decoding' or 'large/64bit BARs' or 'Above 4G MMIO BIOS assignment' enabled.

Installing NVIDIA Drivers¶

Update the system:

sudo apt update && sudo apt full-upgrade -y

For RTX4xxx series, A100, and H100 on Ubuntu 22.04, you need to update the kernel version. You can also update the kernel version for older graphics cards:
```
sudo apt install linux-generic-hwe-22.04
```

Install the ubuntu-drivers-common package and determine the recommended driver to install:

sudo apt install ubuntu-drivers-common
ubuntu-drivers devices

You should get an output similar to this:

user@48567:~$ ubuntu-drivers devices
== /sys/devices/pci0000:00/0000:00:02.5/0000:07:00.0 == 
modalias  : pci:v000010DEd000024B0sv000010DEsd000014ADbc03sc00i00
vendor    : NVIDIA Corporation
model     : GA104GL [RTX A4000]
driver    : nvidia-driver-550 - third-party non-free recommended
driver    : nvidia-driver-550-open - third-party non-free
driver    : nvidia-driver-470-server - distro non-free
driver    : nvidia-driver-535-server - distro non-free
driver    : nvidia-driver-470 - distro non-free
driver    : nvidia-driver-545-open - distro non-free
driver    : nvidia-driver-550 - third-party non-free recommended

Install the recommended driver:
```
sudo apt install nvidia-driver-550
```
Reboot the system.

Verification of driver installation on graphics card:

nvidia-smi

Output similar to that expected:

user@48567:~$ nvidia-smi
Fri May 10 15:58:17 2024
+-----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.54.15              Driver Version: 550.54.15      CUDA Version: 12.4     |
|-----------------------------------------+------------------------+----------------------+
| GPU  Name                 Persistence-M | Bus-Id          Disp.A | Volatile Uncorr. ECC |
| Fan  Temp   Perf          Pwr:Usage/Cap |           Memory-Usage | GPU-Util  Compute M. |
|                                         |                        |               MIG M. |
|=========================================+========================+======================|
|   0  NVIDIA RTX A4000               Off |   00000000:07:00.0 Off |                  Off |
| 41%   31C    P8             15W /  140W |       3MiB /  16376MiB |      0%      Default |
|                                         |                        |                  N/A |
+-----------------------------------------+------------------------+----------------------+

+-----------------------------------------------------------------------------------------+
| Processes:                                                                              |
|  GPU   GI   CI        PID   Type   Process name                              GPU Memory |
|        ID   ID                                                               Usage      |
|=========================================================================================|
|  No running processes found                                                             |
+-----------------------------------------------------------------------------------------+

Note

The most current guide for installing NVIDIA drivers under Ubuntu can be found at this link.

Installing CUDA¶

Installing CUDA 11.8¶

Install the gcc compiler, which is required to build CUDA.
```
sudo apt install gcc
```

Download and install CUDA (for Ubuntu 22.04 only).

sudo wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-ubuntu2204.pin
sudo mv cuda-ubuntu2204.pin /etc/apt/preferences.d/cuda-repository-pin-600
sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/3bf863cc.pub
sudo add-apt-repository "deb https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/ /"
sudo apt update && sudo apt upgrade -y
sudo apt install cuda-11-8 -y

Set environment variables for your frameworks and applications to detect CUDA in your .bashrc file:

 echo 'export PATH=/usr/local/cuda-11.8/bin:$PATH' >> ~/.bashrc
 echo 'export LD_LIBRARY_PATH=/usr/local/cuda-11.8/lib64:$LD_LIBRARY_PATH' >> ~/.bashrc
 source ~/.bashrc

!!! warning "Attention" You must run these commands for all users who need to use CUDA.

Reboot your system.
Verify the CUDA installation:

nvcc -V

After a successful installation, you should see an output similar to this:

nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2021 NVIDIA Corporation
Built on Thu_Nov_18_09:45:30_PST_2021
Cuda compilation tools, release 11.5, V11.5.119
Build cuda_11.5.r11.5/compiler.30672275_0

Installing CUDA 12 (recommended)¶

Install the compiler gcc, which is necessary for building CUDA:
```
sudo apt install gcc
```

Download and install CUDA. For Ubuntu 24.04, replace ubuntu2204 with ubuntu2404 in the wget path:

wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb
sudo apt update
sudo apt install cuda -y

Set environment variables for CUDA in your ~/.bashrc file:

echo 'export PATH="/sbin:/bin:/usr/sbin:/usr/bin:${PATH}:/usr/local/cuda/bin"' >> ~/.bashrc
echo 'export LD_LIBRARY_PATH=/usr/local/cuda/lib64\${LD_LIBRARY_PATH:+:\${LD_LIBRARY_PATH}}' >> ~/.bashrc
source ~/.bashrc

Attention

You must execute these commands for all users who need to utilize CUDA.

Verify the installation:

nvcc -V

Upon proper installation, you should receive an output similar to this:

user@48567:~$ nvcc -V
nvcc: NVIDIA (R) CUDA compiler driver
Copyright 2005-2024 NVIDIA Corporation
Built on Thu Mar 28 02:18:24 PDT 2024
CUDA compilation tools, release 12.4, V12.4.131
Build cuda_12.4.r12.4/compilier.34097967_0

Installing NVIDIA modules for Docker¶

If you're using Docker containers, don't forget to install the nvidia-docker2 package:

```bash
sudo apt install -y nvidia-docker2
sudo systemctl restart docker
```

One-Click Installation of Drivers and CUDA 12¶

You can use this script for automatic installation of drivers and CUDA 12:

#!/bin/bash

# Update and upgrade the system using apt
sudo apt update
sudo apt upgrade -y

#Check Ubuntu 22.04 and update kernel
lsb_release=$(lsb_release -a | grep "22.04")
if [[ -n "$lsb_release" ]]; then

    # Check if there's a video card with Nvidia (10de) H100 model (23xx)
    lspci_output=$(lspci -nnk | awk '/\[10de:23[0-9a-f]{2}\]/ {print $0}')
    if [[ -n "$lspci_output" ]]; then
        echo "H100 detected"
        # If yes install the necessary kernel package
        sudo apt install -y linux-generic-hwe-22.04
    fi

    # Check if there's a video card with Nvidia (10de) A100 model (20xx)
    lspci_output=$(lspci -nnk | awk '/\[10de:20[0-9a-f]{2}\]/ {print $0}')
    if [[ -n "$lspci_output" ]]; then
        echo "A100 detected"
        # If yes install the necessary kernel package
        sudo apt install -y linux-generic-hwe-22.04
    fi
fi

# Install Ubuntu drivers common package
sudo apt install ubuntu-drivers-common -y

recommended_driver=$(ubuntu-drivers devices | grep 'nvidia' | cut -d ',' -f 1 | grep 'recommended')
package_name=$(echo $recommended_driver | awk '{print $3}')
sudo apt install $package_name -y

# Install GCC compiler for CUDA install
sudo apt install gcc -y

# Get the release version of Ubuntu
RELEASE_VERSION=$(lsb_release -rs | sed 's/\([0-9]\+\)\.\([0-9]\+\)/\1\2/')

# Download and install CUDA package for Ubuntu
wget https://developer.download.nvidia.com/compute/cuda/repos/ubuntu${RELEASE_VERSION}/x86_64/cuda-keyring_1.1-1_all.deb
sudo dpkg -i cuda-keyring_1.1-1_all.deb

# Update and upgrade the system again to ensure all packages are installed correctly
sudo apt update
sudo apt install cuda -y

# Add PATH and LD_LIBRARY_PATH environment variables for CUDA in .bashrc file
echo 'export PATH="/sbin:/bin:/usr/sbin:/usr/bin:${PATH}:/usr/local/cuda/bin"' >> ~/.bashrc
echo 'export LD_LIBRARY_PATH=/usr/local/cuda/lib64\${LD_LIBRARY_PATH:+:\${LD_LIBRARY_PATH}}' >> ~/.bashrc
source ~/.bashrc

#Installing Docker binding for Nvidia

if command -v docker &> /dev/null; then
  echo "Docker is installed."
  sudo apt install -y nvidia-docker2
  sudo systemctl restart docker
else
  echo "Docker is not installed."
fi

#Reboot the system for enable kernel modules
reboot