Registry — DeepStream documentation (original) (raw)

The Registry is an integral part of the Graph Composer ecosystem and is responsible for providing unified interface between the extensions and tools. Registry service can be accessed using registry CLI:

• Registry Command Line Interface

In the backend, Registry communicates with Nvidia Cloud repository and Local workspace for extension management while it uses local database as cache of extension metadata.

Repository Manager#

Repository manager is responsible to communicating with NVIDIA Cloud repository or local workspace based on operation or arguments to operation.

NVIDIA Cloud Repository#

Extensions from Nvidia for public access are published in Nvidia Cloud repository. External users can access these extensions using registry repo sync -n ngc-public command. This command downloads all published extensions metadata to Cache on local disk.

Local Workspace#

Local workspace is provided for Extension Development Workflow. Registry stores current development version of extension in local workspace. Default workspace path used by Registry is /var/tmp/gxf/.cache/gxf_workspace. It stores only one version so it will overwrite whenever user updates version of extension and adds it to registry.

Note

/var/tmp/gxf/` is considered as the registry root on local disk which is configurable using the GXF_REGISTRY_ROOT environment variable.

Cache#

Registry maintains metadata for all extensions synchronized from Nvidia Cloud repository or added locally by Extension Development Workflow. It allows quick access to extension metadata on query from tools. Users can clean cache or refresh cache using Registry Command Line Interface commands. Users has to sync repositories again after cleaning cache to query extension information from registry.

Registry maintains two types of cache. The extension metadata contains extension interfaces from all the extensions that were synced along with their corresponding variants which have been uploaded to NGC. This cache is populated by the registry by registry repo sync command. The extension variants are cached by the registry graph install command. The variant archive containing the extension library along with any other headers, source or data files are cached the first time they are downloaded from a NGC repo and reused for subsequent registry graph install operations.

Note

/var/tmp/gxf/ is considered as the registry root which is configurable using the GXF_REGISTRY_ROOT environment variable. Optionally, registry cache --set command can be used to set a specific path for the registry cache.

Note

If Composer is being lauched from a docker, the registry cache from the host can be accessed in the docker by mounting the registry cache path.

Sample Command for docker launch: :: docker run -it –rm –net=host –runtime nvidia -e DISPLAY=$DISPLAY -v /tmp/.X11-unix/:/tmp/.X11-unix -v /var/tmp/gxf/.cache/gxf_registry/:/var/tmp/gxf/.cache/gxf_registry/ –privileged :

Extension Registration#

Extension registration adds a new extension to the registry’s cache and default repository. register_extension macro can be used to register a new extension with the registry.

Mandatory fields:

• name - Name of the extension registration target. It must follow register_<extension-name>_ext naming convention.
• extension – Extension target complied using the graph_cc_extension(…) rule
• uuid – Universally unique identifier used to register the extension. Format should follow the standard uuid convention
• version – Extension version. A string that specifies the version of the extension in the following format MAJOR.MINOR.PATCH. Extension developers are expected to follow the semantic versioning concept where a new update in version should follow these guidelines:

  • MAJOR version update when there are API-breaking changes
  • MINOR version update when features are added in backward compatible manner
  • PATCH version update when bugs are fixed in backward compatible manner

• license – Extension license. A string naming the type of license associated with the extension. For example – MIT, Apache-2.0, BSD.
• license_file – Path to a text file which has the complete licensing terms and specifications.

Optional fields:

• url – A link to the web address of the extension project
• repository – A link to the web address of the extension source code
• labels – A list of strings which can be used to categorize the extension in NGC. For example, [“nvidia”, “gpu”, “nvgraph”]
• priority – A value between 0-100 used to indicate the priority of the extension to be used during deployment.
• target – Target configuration for the extension variant. A dictionary containing the following keys - arch, os and distribution. If the target info is not specified, it will be automatically deduced based on the bazel build configs used. Accepted values:

  • arch – x86_64, aarch64, sbsa
  • os – linux
  • distribution – ubuntu_22.04

• local_dependencies – A list of registration targets of the dependent repositories. Extensions specified here will be registered prior to the current extension. For example:
  local_dependencies = ["//gxf/sample_extension:register_sample_ext"]
• compute_dependencies – A list of compute dependencies needed to build the extension library. The required versions of these compute dependencies will be deduced automatically using the bazel build configs used. Accepted values - cuda, cudnn, tensorrt, deepstream, vpi. Example usage:
  compute_dependencies = ["cuda", "tensorrt", "deepstream"]
• headers – A list of strings containing the paths to the header files of the extension.
• binaries – A list of strings containing the paths to any optional binary files the extension needs during deployment.
• data – A list of strings containing the paths to any optional data files the extension needs during deployment.

Once register_extension macro has been invoked, a manifest file would be auto generated and the registry CLI tool would be used to run extn add command along with the corresponding arguments. This rule further adds this extension to the registry’s local cache and updates the default repository with the extension library, manifest, metadata, and any of the optional files that were specified during registration. The registration process also generates an output file which contains metadata regarding the extension’s interface. More specifically, it contains information regarding various components and the parameters used in those components, header files, dependencies, labels, author, description etc.

Here are the guidelines for registering an extension:

• x86_64 version of an extension must be registered first before any cross complied (aarch64) variant of the extension is registered. Since the cross complied extension cannot be queried for the extension interface (types of components and parameters) the registry considers the x86_64 variant of the extension as the reference for its interface. In case the cache and/or default repository has been cleaned since the extension was previously registered, the x86_64 variant of the extension must be registered again to repopulate the cache and default repository.
• If the extension getting registered depends on other extensions in the source code then it must be listed as a dependency in the “dependencies” field. Specifically, the registration target name of the dependent extension must be listed as the dependency so that registry can query the dependent extension’s metadata for its uuid, name and version.
• If the same extension variant is registered twice, the previously registered variant in the default repository is removed and updated with the contents in the new manifest. Similarly, registering any of the x86_64 variants will update the extension interface in registry cache and hence it is important that extensions interface remains the same across all the variants of the same extension and version.
• Registering an extension and its variants is only supported on linux x86_64 platforms.

Installing graphs for deployment#

The registry can also be used to deploy graphs locally or containerize the graph using container builder. To execute a graph using gxe, the registry provides a functionality to prepare a manifest and a corresponding archive package containing all the extensions that are required.

An example manifest for the sample graph above is shown here:

An example archive for the sample graph above is shown here:

To install a graph:

• Create a graph using the nvgraph composer and save it to the local filesystem.
• Use the graph install command in the CLI tool along with the target platform configuration that decides the variant of the extensions that should be used to for deployment. Registry will auto select the best matching variant for each extension based on the target platform.
• The output of the graph install command is a gxe manifest along with an archive which contains all the extension libraries and files that were packaged with the extension variant.
• There’s an option to unpack the deployment package in a local directory which can be used to run the graph locally. Use registry CLI tool with graph install --help to view all the arguments that are supported.
• All the extensions used in the graph must be present in the registry’s local cache to install the graph successfully. The version of the extension used in the graph must also match the versions of the extension found in the registry cache.
• Suppose an extension A is listed as a dependency for extension B and extension C. If a graph is created using extensions B and extension C, it needs to be ensured that both B and C are dependent on the same version of extension A and that version of extension A should be synchronized with the registry to install the graph. Using multiple versions of the same extension in a graph is not supported and is not advisable to create such dependencies.
• The registry performs version management while installing graphs to ensure the latest versions of extensions are being deployed. Suppose a graph was created using version 1.0.0 of extension A, while there was a newer version of extension 1.1.0 available during graph install stage. It is ensured only the latest minor.patch version corresponding to the same major of the extension is added in the archive.