> ## Content Index > Fetch the complete content index at: https://corelab.tech/llms.txt > Use this file to discover other available public pages before exploring further. # NVIDIA Toolkit Setup on Linux - URL: https://corelab.tech/nvidiatoolkit/ - Published: 2025-08-11T14:19:31.000Z - Updated: 2026-10-31T16:06:51.000Z - Author: Core Lab Joe - Tags: Linux, Docker, NVIDIA, #2025, #table-of-content ### **Introduction / why?** Now I'm actually a bit of an underdog (AMD) fanboy, but in the end a real tech/nerd uses the best tool available, to do the job. [NVIDIA](https://corelab.tech/llmgpu/) is wearing the King's crown for GPU accelerated functions like video [transcoding](https://corelab.tech/transcoding-architecture-nvenc-vs-quicksync/) and [hosting private local AI models with Ollama](https://corelab.tech/private-chatgpt-local-llm/). This guide will walk you through installing the NVIDIA Toolkit, drivers, and setting up your [docker](https://corelab.tech/setupcompose/) runtime to make use of it for GPU acceleration of any dockers you so chose, such as [Plex / Jellyfin](https://corelab.tech/plexvsjellyfin/)! ## **Prerequisites** - [A supported NVIDIA GPU with a min compute capability of 3.0](https://developer.nvidia.com/cuda-gpus?ref=corelab.tech) - NVIDIA driver already installed & compatible with [CUDA Toolkit version](https://docs.nvidia.com/deploy/cuda-compatibility/?ref=corelab.tech) - This guide is for Debian or Ubuntu or their derivitaves like PopOS! - Instructions are for headless Linux but will work w/GUI if you run from CLI. ## **Part 1 - Linux Driver install (Apt way)** 💡 You may have to disable secure boot from BIOS - google that on your specific motherboard! If you don't want to do that you have to go through the process of installing nvidia mok key. Installing NVIDIA drivers is a piece of cake on Debian based systems. We can install right from apt repository or, from direct download. I recommend the apt repo b/c it's simpler and will update for you when a new version is released by NVIDIA. First, update your apt repository if you have not done so yet. Use your fav text editor like nano to edit source.list found here: ```bash **Backup** your existing sources.lists! cp /etc/apt/sources.list /etc/apt/sources.list.bak nano /etc/apt/sources.list **For Debian 12** - Replace yours with: deb https://deb.debian.org/debian bullseye main contrib non-free deb https://deb.debian.org/debian bullseye-updates main contrib non-free deb https://deb.debian.org/debian-security bullseye-security main contrib non-free ``` For Debian 13 Trixie: ```bash deb https://deb.debian.org/debian trixie main contrib non-free non-free-firmware deb https://deb.debian.org/debian trixie-updates main contrib non-free non-free-firmware deb https://deb.debian.org/debian-security trixie-security main contrib non-free non-free-firmware ``` Once you've done that, you're /etc/apt/sources.list should look something like: ```bash deb http://deb.debian.org/debian/ bookworm main contrib non-free non-free-firmware deb http://security.debian.org/debian-security bookworm-security main contrib non-free non-free-firmware deb http://deb.debian.org/debian/ bookworm-updates main contrib non-free non-free-firmware ``` ### Now, install the new drivers! ```bash # Do an apt update with your new sources and upgrade everything, so there's nothing waiting or in the way. sudo apt update && sudo apt upgrade # Last upgrade step, moves you from a minor upgrade to a major, if there's any. sudo apt full-upgrade # Cleanup! sudo apt --purge autoremove sudo reboot # Search apt for NVIDIA drivers if you want. sudo apt search nvidia-driver # At the time of writing the latest driver is 575.57.08 # Install headless server drivers. sudo apt install nvidia-headless-575-server # Install NVIDIA Utils (gives us nvidia-smi!) sudo apt install nvidia-utils-575-server # Search apt for libnvidia-encode. Needed for things like transcoding. sudo apt search nvidia-encode # Install libnvidia-encode package. sudo apt install libnvidia-encode-575-server ``` Code for installing drivers After installing all the packages and rebooting (if needed), check if everything is working as it should. Type: ```bash nvidia-smi ``` ```bash Mon Aug 11 09:14:35 2025 +-----------------------------------------------------------------------------------------+ | NVIDIA-SMI 570.124.04 Driver Version: 570.124.04 CUDA Version: 12.8 | |-----------------------------------------+------------------------+----------------------+ | GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC | | Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. | | | | MIG M. | |=========================================+========================+======================| | 0 NVIDIA GeForce RTX 3060 Off | 00000000:2E:00.0 Off | N/A | | 39% 44C P0 45W / 170W | 405MiB / 12288MiB | 0% Default | | | | N/A | +-----------------------------------------+------------------------+----------------------+ +-----------------------------------------------------------------------------------------+ | Processes: | | GPU GI CI PID Type Process name GPU Memory | | ID ID Usage | |=========================================================================================| | 0 N/A N/A 1580836 C frigate.detector.tensorrt 266MiB | | 0 N/A N/A 3645798 C /usr/lib/ffmpeg/7.0/bin/ffmpeg 125MiB | +-----------------------------------------------------------------------------------------+ ``` What NVIDIA-SMI looks like post driver install. --- ## Part 2 - Installing NVIDIA container toolkit Now that we're here, we've got 1 more package repo to add/fixup. Copy this whole code block and paste it into your CLI. ```bash # Add the gpgpkey and repository curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey | sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg && curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list | sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' | sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list ``` If you're already root, remove the sudos in this code block! If that took, you should get a blank prompt but **no errors**. ### Install the Toolkit, finally ```bash # Update and install NVIDIA Container Toolkit sudo apt-get update sudo apt-get install -y nvidia-container-toolkit # Configure NVIDIA Container Toolkit sudo nvidia-ctk runtime configure --runtime=docker sudo systemctl restart docker ``` --- ## Step 3 - Testing / Verification Now check that everything is running properly, test it out and ensure everything is running properly. We have 2 ways to do this. One is just checking docker itself and another is more intensive, downloading and running a test docker from NVIDIA and actually having it utilize the GPU. ### Docker Runtime check. ```bash docker info (sudo docker info if you didn't setup a user properly!) ``` You should see something like this, look for the 'Runtimes:' line (Bolded). You're looking to see it says nvidia in there. #### Output of command docker info Client: Docker Engine - CommunityVersion: 28.3.3Context: defaultDebug Mode: falsePlugins:buildx: Docker Buildx (Docker Inc.)Version: v0.26.1Path: /usr/libexec/docker/cli-plugins/docker-buildxcompose: Docker Compose (Docker Inc.)Version: v2.39.1Path: /usr/libexec/docker/cli-plugins/docker-composeServer:Containers: 55Running: 54Paused: 0Stopped: 1Images: 52Server Version: 28.3.3Storage Driver: overlay2Backing Filesystem: extfsSupports d\_type: trueUsing metacopy: falseNative Overlay Diff: trueuserxattr: falseLogging Driver: json-fileCgroup Driver: systemdCgroup Version: 2Plugins:Volume: localNetwork: bridge host ipvlan macvlan null overlayLog: awslogs fluentd gcplogs gelf journald json-file local splunk syslogCDI spec directories:/etc/cdi/var/run/cdiSwarm: inactiveRuntimes: io.containerd.runc.v2 nvidia runcDefault Runtime: runcInit Binary: docker-initcontainerd version: 05044ec0a9a75232cad458027ca83437aae3f4darunc version: v1.2.5-0-g59923efinit version: de40ad0Security Options:apparmorseccompProfile: builtincgroupnsKernel Version: 6.12.30+bpo-amd64Operating System: Debian GNU/Linux 12 (bookworm)OSType: linuxArchitecture: x86\_64CPUs: 16Total Memory: 62.73GiBName: codexID: efa2c03f-60b2-4948-a862-303f7e7b5273Docker Root Dir: /var/lib/dockerDebug Mode: falseExperimental: falseInsecure Registries:::1/128127.0.0.0/8Live Restore Enabled: false ### Running a docker with GPU acceleration This is as easy as typing: ```bash docker run --gpus all nvidia/cuda:13.0.0-base-ubuntu24.04 nvidia-smi ``` If that doesn't work due to drivers / type of GPU you have let's try and older one: ```yaml docker run --gpus all nvidia/cuda:11.5.2-base-ubuntu20.04 nvidia-smi ``` 💡 This is running a mini Ubuntu instance with nvidia-smi on it, so when you're paassing that command to the docker, it's telling you that it does in fact have access to the NVIDIA GPU and the drivers! Congrats! ### Cleanup Time: Warning this is a "Deep clean!" ```bash docker system prune -a Hit "y" so it removes any extra images/docker cruft from us playing. ``` WARNING: It removes --- Now you're ready to utilize that [GPU](https://corelab.tech/llmgpu/) to accelerate everything and anything you'd like to do in your [self-hosted lab](https://corelab.tech/ultimate-media-server-guide/), like using it for [Plex](https://corelab.tech/plexoptimization/), or [deploying a private local LLM stack in Docker](https://corelab.tech/private-chatgpt-local-llm/) right at home! --- ## ❓ Frequently Asked Questions (FAQ) ### Do I need to disable secure boot to install NVIDIA drivers on Linux? Possibly, depending on your motherboard. If you don't want to disable secure boot in BIOS, you can instead go through the process of installing and signing an NVIDIA MOK (Machine Owner Key) so the kernel will trust the driver modules. ### Should I install NVIDIA drivers from apt or a direct download? The apt repository method is recommended for Debian-based systems since it's simpler to set up and will automatically update when NVIDIA releases new versions, rather than requiring you to manually track and reinstall a downloaded driver package. ### How do I check if Docker is actually using the NVIDIA GPU? Run `docker info` and look for `nvidia` listed on the Runtimes line, confirming the NVIDIA Container Toolkit registered correctly. For a full functional test, run `docker run --gpus all nvidia/cuda:-base-ubuntu24.04 nvidia-smi` and confirm it reports your GPU from inside the container. ### What if the NVIDIA CUDA test container doesn't work? Try an older CUDA base image tag, such as `nvidia/cuda:11.5.2-base-ubuntu20.04`, since a very new CUDA container image can be incompatible with an older installed driver version. ### Can I use the NVIDIA Container Toolkit for Plex or Jellyfin transcoding? Yes. Once the toolkit is installed and Docker's runtime is configured for NVIDIA, any container you run, including Plex or Jellyfin, can be given GPU access for hardware-accelerated transcoding the same way the guide's test container demonstrates.