GitHub - NVIDIA/NeMo-Agent-Toolkit: The NVIDIA NeMo Agent toolkit is an open-source library for efficiently connecting and optimizing teams of AI agents. (original) (raw)

NVIDIA NeMo Agent Toolkit

NVIDIA NeMo Agent Toolkit

NVIDIA NeMo Agent toolkit is a flexible, lightweight, and unifying library that allows you to easily connect existing enterprise agents to data sources and tools across any framework.

Note

NeMo Agent toolkit was previously known as the Agent Intelligence (AIQ) toolkit, and AgentIQ. The library was renamed to better reflect the purpose of the toolkit and to align with the NVIDIA NeMo family of products. The core technologies, performance and roadmap remain unchanged and the API is fully compatible with previous releases.

The rename is still in progress and references to the previous name may still be found in the codebase and documentation.

Key Features

With NeMo Agent toolkit, you can move quickly, experiment freely, and ensure reliability across all your agent-driven projects.

Component Overview

The following diagram illustrates the key components of NeMo Agent toolkit and how they interact. It provides a high-level view of the architecture, including agents, plugins, workflows, and user interfaces. Use this as a reference to understand how to integrate and extend NeMo Agent toolkit in your projects.

NeMo Agent toolkit Components Diagram

Get Started

Prerequisites

Before you begin using NeMo Agent toolkit, ensure that you meet the following software prerequisites.

Install From Source

  1. Clone the NeMo Agent toolkit repository to your local machine.
    git clone git@github.com:NVIDIA/NeMo-Agent-Toolkit.git aiqtoolkit
    cd aiqtoolkit
  2. Initialize, fetch, and update submodules in the Git repository.
    git submodule update --init --recursive
  3. Fetch the data sets by downloading the LFS files.
    git lfs install
    git lfs fetch
    git lfs pull
  4. Create a Python environment.
    uv venv --seed .venv
    source .venv/bin/activate
    Make sure the environment is built with Python version 3.11 or 3.12. If you have multiple Python versions installed, you can specify the desired version using the --python flag. For example, to use Python 3.11:
    uv venv --seed .venv --python 3.11
    You can replace --python 3.11 with any other Python version (3.11 or 3.12) that you have installed.
  5. Install the NeMo Agent toolkit library. To install the NeMo Agent toolkit library along with all of the optional dependencies. Including developer tools (--all-groups) and all of the dependencies needed for profiling and plugins (--all-extras) in the source repository, run the following:
    uv sync --all-groups --all-extras
    Alternatively to install just the core NeMo Agent toolkit without any plugins, run the following:
    At this point individual plugins, which are located under the packages directory, can be installed with the following command uv pip install -e '.[<plugin_name>]'. For example, to install the langchain plugin, run the following:
    uv pip install -e '.[langchain]'

    [!NOTE] Many of the example workflows require plugins, and following the documented steps in one of these examples will in turn install the necessary plugins. For example following the steps in the examples/simple/README.md guide will install the aiqtoolkit-langchain plugin if you haven't already done so.
    In addition to plugins, there are optional dependencies needed for profiling. To install these dependencies, run the following:
    uv pip install -e '.[profiling]'

  6. Verify the installation using the NeMo Agent toolkit CLI
    This should output the NeMo Agent toolkit version which is currently installed.

Hello World Example

  1. Ensure you have set the NVIDIA_API_KEY environment variable to allow the example to use NVIDIA NIMs. An API key can be obtained by visiting build.nvidia.com and creating an account.
    export NVIDIA_API_KEY=
  2. Create the NeMo Agent toolkit workflow configuration file. This file will define the agents, tools, and workflows that will be used in the example. Save the following as workflow.yaml:
    functions:

    Add a tool to search wikipedia

    wikipedia_search:
    _type: wiki_search
    max_results: 2
    llms:

    Tell NeMo Agent toolkit which LLM to use for the agent

    nim_llm:
    _type: nim
    model_name: meta/llama-3.1-70b-instruct
    temperature: 0.0
    workflow:

    Use an agent that 'reasons' and 'acts'

    _type: react_agent

    Give it access to our wikipedia search tool

    tool_names: [wikipedia_search]

    Tell it which LLM to use

    llm_name: nim_llm

    Make it verbose

    verbose: true

    Retry parsing errors because LLMs are non-deterministic

    retry_parsing_errors: true

    Retry up to 3 times

    max_retries: 3
  3. Run the Hello World example using the aiq CLI and the workflow.yaml file.
    aiq run --config_file workflow.yaml --input "List five subspecies of Aardvarks"
    This will run the workflow and output the results to the console.
    Workflow Result:
    ['Here are five subspecies of Aardvarks:\n\n1. Orycteropus afer afer (Southern aardvark)\n2. O. a. adametzi Grote, 1921 (Western aardvark)\n3. O. a. aethiopicus Sundevall, 1843\n4. O. a. angolensis Zukowsky & Haltenorth, 1957\n5. O. a. erikssoni Lönnberg, 1906']

Feedback

We would love to hear from you! Please file an issue on GitHub if you have any feedback or feature requests.

Acknowledgements

We would like to thank the following open source projects that made NeMo Agent toolkit possible: