NeMo Microservices | NVIDIA NGC (original) (raw)

Containers

Sorry, your browser does not support inline SVG.

Helm Charts

Sorry, your browser does not support inline SVG.

Models

Sorry, your browser does not support inline SVG.

Resources

Sorry, your browser does not support inline SVG.

Overview

NVIDIA NeMo microservices provide a modular platform for building and deploying AI workflows across on-premises or cloud Kubernetes environments. These microservices leverage proprietary data and enable continuous optimization of AI applications through a powerful data flywheel architecture.

Core Microservices

• NVIDIA NeMo Customizer: Facilitates the fine-tuning of large language models (LLMs) using supervised and parameter-efficient fine-tuning techniques.
• NVIDIA NeMo Evaluator: Provides comprehensive evaluation capabilities for LLMs, supporting academic benchmarks, custom automated evaluations, and LLM-as-a-Judge approaches.
• NVIDIA NeMo Guardrails: Adds safety checks and content moderation to LLM endpoints, protecting against hallucinations, harmful content, and security vulnerabilities.

Platform Component Microservices

• NVIDIA NeMo Data Store: Serves as the default file storage solution for the NeMo microservices platform, exposing APIs compatible with the Hugging Face Hub client (HfApi).
• NVIDIA NeMo Entity Store: Provides tools to manage and organize general entities such as namespaces, projects, datasets, and models.
• NVIDIA NeMo Deployment Management: Provides an API to deploy NIM for LLMs on a Kubernetes cluster and manage them through the NIM Operator microservice.
• NVIDIA NeMo NIM Proxy: Provides a unified endpoint that you can use to access all deployed NIM for LLMs for inference tasks.
• NVIDIA NeMo Operator: Manages custom resource definitions (CRDs) for NeMo Customizer fine-tuning jobs.
• DGX Cloud Admission Controller: Enables multi-node training requirements for NeMo Customizer jobs through a mutating admission webhook.

For detailed information regarding the NeMo microservices, visit the NeMo microservices documentation.

Note: Use, distribution or deployment of these microservices in production requires an NVIDIA AI Enterprise License.

GOVERNING TERMS

The software and materials are governed by the NVIDIA Software License Agreement and the Product-Specific Terms for NVIDIA AI Products.