Installation — DeepStream documentation (original) (raw)

NVIDIA® DeepStream Software Development Kit (SDK) is an accelerated AI framework to build intelligent video analytics (IVA) pipelines. DeepStream runs on NVIDIA® T4, NVIDIA® Hopper, NVIDIA® Ampere, NVIDIA® ADA and platforms such as NVIDIA® Jetson AGX Orin™, NVIDIA® Jetson Orin™ NX, NVIDIA® Jetson Orin™ Nano. For dGPU platforms Enterprise GPUs are highly recommended for deployments that are expected to run 24x7. Gaming GPUs are not designed to perform in such type of environments.

Jetson Setup#

This section explains how to prepare a Jetson device before installing the DeepStream SDK.

Note

Steps to install DeepStream SDK locally, assume that ~/.local/bin/ has been added to the ~/.bashrc or ~/.profile

Install Jetson SDK components#

Download NVIDIA SDK Manager from https://developer.nvidia.com/embedded/jetpack. You will use this to install JetPack 6.1 GA (corresponding to L4T 36.4 release)

Prerequisites#

Note

Sometimes while running gstreamer pipeline or sample apps, user can encounter error : GLib (gthread-posix.c): Unexpected error from C library during 'pthread_setspecific': Invalid argument. Aborting.The issue is caused because of a bug in glib 2.0-2.72 version which comes with ubuntu22.04 by default. The issue is addressed in glib2.76 and its installation is required to fix the issue (GNOME/glib).

Migrate glib to newer version#

In order to migrate to newer glib version (e.g. 2.76.6) follow below steps:

  1. Prerequisites: Install below packages:
    pip3 install meson
    pip3 install ninja

Note

On baremetal, use sudo pip3 install meson and sudo pip3 install ninja for root permissions

  1. Compilation and installation steps:
    $ git clone https://github.com/GNOME/glib.git
    $ cd glib
    $ git checkout

e.g. 2.76.6

$ meson build --prefix=/usr
$ ninja -C build/
$ cd build/
$ ninja install

Note

On baremetal, use sudo ninja install to install with root permissions

  1. Check and confirm the newly installed glib version:

pkg-config --modversion glib-2.0

Install Dependencies#

Install prerequisite packages#

Enter the following commands to install the prerequisite packages:

$ sudo apt install
libssl3
libssl-dev
libgstreamer1.0-0
gstreamer1.0-tools
gstreamer1.0-plugins-good
gstreamer1.0-plugins-bad
gstreamer1.0-plugins-ugly
gstreamer1.0-libav
libgstreamer-plugins-base1.0-dev
libgstrtspserver-1.0-0
libjansson4
libyaml-cpp-dev

Note

Sometimes with RTSP streams the application gets stuck on reaching EOS. This is because of an issue in rtpjitterbuffer component. To fix this issue, a script “update_rtpmanager.sh” at /opt/nvidia/deepstream/deepstream/ has been provided with required details to update gstrtpmanager library. The script should be executed once above mentioned packages are installed as prerequisite.

Install librdkafka (to enable Kafka protocol adaptor for message broker)#

  1. Clone the librdkafka repository from GitHub:
    $ git clone https://github.com/confluentinc/librdkafka.git
  2. Configure and build the library:
    $ cd librdkafka
    $ git checkout tags/v2.2.0
    $ ./configure --enable-ssl
    $ make
    $ sudo make install
  3. Copy the generated libraries to the deepstream directory:
    $ sudo mkdir -p /opt/nvidia/deepstream/deepstream/lib
    $ sudo cp /usr/local/lib/librdkafka* /opt/nvidia/deepstream/deepstream/lib
    $ sudo ldconfig

Install latest NVIDIA BSP packages#

Installation of JetPack 6.1 GA will ensure that latest NVIDIA BSP packages are installed.

Install the DeepStream SDK#

dGPU Setup for Ubuntu#

This section explains how to prepare an Ubuntu x86_64 system with NVIDIA dGPU devices before installing the DeepStream SDK.

Note

Steps to install DeepStream SDK locally, assume that ~/.local/bin/ has been added to the ~/.bashrc or ~/.profile

Note

This document uses the term dGPU (“discrete GPU”) to refer to NVIDIA GPU expansion card products such as NVIDIA Tesla® T4, NVIDIA® Hopper, NVIDIA® Ampere, NVIDIA® ADA, NVIDIA GeForce® RTX 2080, NVIDIA GeForce® RTX 3080, NVIDIA GeForce® RTX 4080 and GeForce®/NVIDIA RTX/QUADRO. This version of DeepStream SDK runs on GPUs supported by NVIDIA driver 535.183.06 for Data Center GPUs and 560.35.03 for RTX GPUs and NVIDIA TensorRT™ 10.3.0.26 and later versions.

Prerequisites#

You must install the following components:

Remove all previous DeepStream installations#

Enter the following commands to remove all previous DeepStream 3.0 or prior installations:

$ sudo rm -rf /usr/local/deepstream /usr/lib/x86_64-linux-gnu/gstreamer-1.0/libgstnv* /usr/bin/deepstream* /usr/lib/x86_64-linux-gnu/gstreamer-1.0/libnvdsgst* /usr/lib/x86_64-linux-gnu/gstreamer-1.0/deepstream* /opt/nvidia/deepstream/deepstream* $ sudo rm -rf /usr/lib/x86_64-linux-gnu/libv41/plugins/libcuvidv4l2_plugin.so

To remove DeepStream 4.0 or later installations:

  1. Open the uninstall.sh file in /opt/nvidia/deepstream/deepstream/
  2. Run the following script as sudo ./uninstall.sh

Note

Sometimes while running gstreamer pipeline or sample apps, user can encounter error : GLib (gthread-posix.c): Unexpected error from C library during 'pthread_setspecific': Invalid argument. Aborting.The issue is caused because of a bug in glib 2.0-2.72 version which comes with ubuntu22.04 by default. The issue is addressed in glib2.76 and its installation is required to fix the issue (GNOME/glib).

Migrate glib to newer version#

In order to migrate to newer glib version (e.g. 2.76.6) follow below steps:

  1. Prerequisites: Install below packages:
    pip3 install meson
    pip3 install ninja

Note

On baremetal x86, use sudo pip3 install meson and sudo pip3 install ninja for root permissions

  1. Compilation and installation steps:
    $ git clone https://github.com/GNOME/glib.git
    $ cd glib
    $ git checkout

e.g. 2.76.6

$ meson build --prefix=/usr
$ ninja -C build/
$ cd build/
$ ninja install

Note

On baremetal x86, use sudo ninja install to install with root permissions

  1. Check and confirm the newly installed glib version:

pkg-config --modversion glib-2.0

Install Dependencies#

Install prerequisite packages#

Enter the following commands to install the necessary packages before installing the DeepStream SDK:

$ sudo apt install
libssl3
libssl-dev
libgles2-mesa-dev
libgstreamer1.0-0
gstreamer1.0-tools
gstreamer1.0-plugins-good
gstreamer1.0-plugins-bad
gstreamer1.0-plugins-ugly
gstreamer1.0-libav
libgstreamer-plugins-base1.0-dev
libgstrtspserver-1.0-0
libjansson4
libyaml-cpp-dev
libjsoncpp-dev
protobuf-compiler
gcc
make
git
python3

Note

Sometimes with RTSP streams the application gets stuck on reaching EOS. This is because of an issue in rtpjitterbuffer component. To fix this issue,a script “update_rtpmanager.sh” at /opt/nvidia/deepstream/deepstream/ has been provided with required details to update gstrtpmanager library. The script should be executed once above mentioned packages are installed as prerequisite.

Install CUDA Toolkit 12.6#

Run the following commands (reference, https://developer.nvidia.com/cuda-downloads):

$ sudo apt-key adv --fetch-keys https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/3bf863cc.pub $ sudo add-apt-repository "deb https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64/ /" $ sudo apt-get update $ sudo apt-get install cuda-toolkit-12-6

Note

If you observe following errors while CUDA installation, refer to https://developer.nvidia.com/blog/updating-the-cuda-linux-gpg-repository-key/.

W: GPG error: https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64 InRelease: The following signatures couldn't be verified because the public key is not available: NO_PUBKEY A4B469963BF863CC E: The repository 'https://developer.download.nvidia.com/compute/cuda/repos/ubuntu2204/x86_64 InRelease' is no longer signed.

Install NVIDIA driver 535.183.06 (for Data Center GPUs) and 560.35.03 (for RTX GPUs)#

Note

Ensure gdm, lightdm or Xorg service is stopped while installing nvidia driver

Use command : sudo service gdm stop sudo service lightdm stop sudo pkill -9 Xorg

Install TensorRT 10.3.0.26#

Run the following command to install TensorRT 10.3.0.26:

version="10.3.0.26-1+cuda12.5" sudo apt-get install libnvinfer-dev=${version} libnvinfer-dispatch-dev=${version} libnvinfer-dispatch10=${version} libnvinfer-headers-dev=${version} libnvinfer-headers-plugin-dev=${version} libnvinfer-lean-dev=${version} libnvinfer-lean10=${version} libnvinfer-plugin-dev=${version} libnvinfer-plugin10=${version} libnvinfer-vc-plugin-dev=${version} libnvinfer-vc-plugin10=${version} libnvinfer10=${version} libnvonnxparsers-dev=${version} libnvonnxparsers10=${version} tensorrt-dev=${version}

Note

Install librdkafka (to enable Kafka protocol adaptor for message broker)#

  1. Clone the librdkafka repository from GitHub:
    $ git clone https://github.com/confluentinc/librdkafka.git
  2. Configure and build the library:
    $ cd librdkafka
    $ git checkout tags/v2.2.0
    $ ./configure --enable-ssl
    $ make
    $ sudo make install
  3. Copy the generated libraries to the deepstream directory:
    $ sudo mkdir -p /opt/nvidia/deepstream/deepstream/lib
    $ sudo cp /usr/local/lib/librdkafka* /opt/nvidia/deepstream/deepstream/lib
    $ sudo ldconfig

Install the DeepStream SDK#

IGX/dGPU on ARM Setup for Ubuntu (Alpha)#

This section explains how to prepare a NVIDIA IGX system with NVIDIA dGPU devices before installing the DeepStream SDK.

Note

This document uses the term dGPU (“discrete GPU”) to refer to NVIDIA GPU expansion card products such as NVIDIA RTX A6000 and NVIDIA RTX 6000 Ada. This version of DeepStream SDK has been validated with NVIDIA driver 535.183.01 and NVIDIA TensorRT™ 10.3.0.26.

You must install the following components:

Run ARM SBSA docker on IGX/dGPU#

  1. Pull the DeepStream Triton Inference Server docker
    docker pull nvcr.io/nvidia/deepstream:7.1-triton-arm-sbsa
  2. Start the docker
    sudo docker run -it --rm --runtime=nvidia --network=host -e NVIDIA_DRIVER_CAPABILITIES=compute,utility,video,graphics --gpus all --privileged -e DISPLAY=:0 -v /tmp/.X11-unix:/tmp/.X11-unix -v /etc/X11:/etc/X11 nvcr.io/nvidia/deepstream:7.1-triton-arm-sbsa

Known Limitation with Video Subsystem and Workaround#

DeepStream 7.1 supports Cuda-12.6 in the compute stack and also in the docker container by default. Data Center GPUs are currently only supported by driver 535.183.06, which comes with Cuda-12.2 driver by default.

Even though CUDA supports forward compatibility with newer runtime versions like Cuda-12.6, other components such as Cuda-GL Interop do not support forward compatibility which are required for display sink to work. Hence, for other components such as Cuda-GL Interop to work, Cuda-12.2 toolkit also must be installed alongside Cuda-12.6 runtime inside the docker container.

For users who need to use/enable display output, following steps can be utilized inside the docker to implement this workaround:

  1. Start the docker as shown in the step above.
  2. Install cuda-toolkit-12-2. Please follow these instructions
  3. PLEASE NOTE: From the Cuda-12 installation instructions mentioned in link above, please replace : sudo apt-get -y install cuda with sudo apt-get -y install cuda-toolkit-12-2
  4. Change default CUDA version to point to Cuda-12.2 inside the docker using update alternatives: update-alternatives --set cuda /usr/local/cuda-12.2
  5. To check which version of CUDA is currently in use inside the docker, run : update-alternatives --display cuda

dGPU on ARM Setup for Ubuntu (Alpha)#

This section explains how to prepare an Ubuntu aarch64 system with NVIDIA dGPU devices before installing the DeepStream SDK.

Note

This document uses the term dGPU (“discrete GPU”) to refer to NVIDIA GPU expansion card products such as NVIDIA Tesla T4, NVIDIA Hopper, NVIDIA Ampere, NVIDIA® ADA, NVIDIA GeForce GTX 1080, NVIDIA GeForce RTX 2080, NVIDIA GeForce RTX 3080, NVIDIA GeForce RTX 4080 and GeForce/NVIDIA RTX/QUADRO. This version of DeepStream SDK runs on GPUs supported by NVIDIA driver 535.183.06 (for Data Center GPUs) and 560.35.03 (for RTX GPUs) and NVIDIA TensorRT™ 10.3.0.26.

You must install the following components:

Install NVIDIA driver 535.183.06 (for Data Center GPUs) and 560.35.03 (for RTX GPUs)#

Note

Ensure gdm, lightdm or Xorg service is stopped while installing Nvidia driver

Use command : sudo service gdm stop sudo service lightdm stop sudo pkill -9 Xorg

Run dGPU on ARM Docker (SBSA)#

  1. Pull the DeepStream Triton Inference Server docker
    docker pull nvcr.io/nvidia/deepstream:7.1-triton-arm-sbsa
  2. Start the docker
    sudo docker run -it --rm --runtime=nvidia --network=host -e NVIDIA_DRIVER_CAPABILITIES=compute,utility,video,graphics --gpus all --privileged -e DISPLAY=:0 -v /tmp/.X11-unix:/tmp/.X11-unix -v /etc/X11:/etc/X11 nvcr.io/nvidia/deepstream:7.1-triton-arm-sbsa

Known Limitation with Video Subsystem and Workaround#

DeepStream 7.1 supports Cuda-12.6 in the compute stack and also in the docker container by default. Data Center GPUs are currently only supported by driver 535.183.06, which comes with Cuda-12.2 driver by default.

Even though CUDA supports forward compatibility with newer runtime versions like Cuda-12.6, other components such as Cuda-GL Interop do not support forward compatibility which are required for display sink to work. Hence, for other components such as Cuda-GL Interop to work, Cuda-12.2 toolkit also must be installed along with the default Cuda-12.6 runtime inside the docker container.

For users who need to use/enable display output, following steps can be utilized inside the docker to implement this workaround:

  1. Start the docker as shown in the step above.
  2. Install cuda-toolkit-12-2. Please follow these instructions
  3. PLEASE NOTE: From the Cuda-12 installation instructions mentioned in link above, please replace : sudo apt-get -y install cuda with sudo apt-get -y install cuda-toolkit-12-2
  4. Change default CUDA version to point to Cuda-12.2 inside the docker using update alternatives: update-alternatives --set cuda /usr/local/cuda-12.2
  5. To check which version of CUDA is currently in use inside the docker, run : update-alternatives --display cuda

Note

Similar limitation is also present for dGPU driver 550, which comes with Cuda-12.4 by default and Cuda-12.6 based docker containers which need to use/enable display.

For users who need to use/enable display on systems using dGPU driver 550, please enable cuda-toolkit-12-4 in addition to Cuda-12.6 inside the container by following similar steps as above.

Replace step no. 2 with cuda-toolkit-12-4 installation instructions found here

dGPU Setup for RedHat Enterprise Linux (RHEL)#

DeepStream for RHEL is not supported in this release.

Platform and OS Compatibility#

The following table provides information about platform and operating system compatibility in the current and earlier versions of DeepStream.

Note

By default, OpenCV has been deprecated. However, OpenCV can be enabled in plugins such as nvinfer (nvdsinfer) and dsexample (gst-dsexample) by setting WITH_OPENCV=1 in the Makefile of these components. Please refer component README for more instructions.

NA: “Not Applicable”

DeepStream 7.1 comes with Alpha support on ARM SBSA platform. This support is currently enabled by SBSA triton docker. SBSA category currently supports IGX/dGPU, GH100 and GH200.