(original) (raw)

> For the complete documentation index, see [llms.txt](https://docs.flowiseai.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.flowiseai.com/integrations/langchain/chat-models/nvidia-nim.md). # NVIDIA NIM ## Local ### Important Note on Running NIM with Flowise If an existing NIM instance is already running (e.g., via NVIDIA’s ChatRTX), starting another instance through Flowise **without checking for an existing endpoint** may cause conflicts. This issue occurs when multiple `podman run` commands are executed on the same NIM, leading to failures. For support, refer to: * [**NVIDIA Developer Forums**](https://forums.developer.nvidia.com/) – For technical issues and questions. * [**NVIDIA Developer Discord**](https://discord.gg/nvidiadeveloper) – For community engagement and [announcements](https://discord.com/channels/1019361803752456192/1340013505834647572). ### Prerequisite 1. Setup [NVIDIA NIM locally with WSL2](https://docs.nvidia.com/nim/wsl2/1.0.0/getting-started.html). ### Flowise 1. **Chat Models** > Drag the **Chat NVIDIA NIM** node > Click **Setup NIM Locally**.

2. If NIM is already installed, click **Next**. Otherwise, click **Download** to start the installer.

3. Select a model image to download.

4. Once selected, click **Next** to proceed with the download.

5. **Downloading Image** – Duration depends on internet speed.

6. Learn more about [Relax Memory Constraints](https://docs.nvidia.com/nim/large-language-models/1.7.0/configuration.html#environment-variables).\\ The **Host Port** is the port for the container to map to the local machine.

7. **Starting the container...**

*Note: If you already have a container running with the selected model, Flowise will ask you if you want to reuse the running container. You can choose to reuse the running container or start a new one with a different port.*

8. **Save the chatflow** 9. [🎉](https://emojipedia.org/party-popper/) **Voila!** Your **Chat NVIDIA NIM** node is now ready to use in Flowise!

## Cloud ### Prerequisite 1. Log in or sign up to [NVIDIA](https://build.nvidia.com/). 2. From the top navigation bar, click NIM:

3. Search for the model you would like to use. To download it locally, we will be using Docker:

4. Follow the instructions from the Docker setup. You must first get an API Key to pull the Docker image:

### Flowise 1. **Chat Models** > drag **Chat NVIDIA NIM** node

2. If you are using NVIDIA hosted endpoint, you must have your API key. **Connect Credential** > click **Create New.** However if you are using local setup, this is optional. 3. Put in the model name and voila [🎉](https://emojipedia.org/party-popper/), your **Chat NVIDIA NIM node** is now ready to be used in Flowise!

### Resources * [NVIDIA LLM Getting Started](https://docs.nvidia.com/nim/large-language-models/latest/getting-started.html) * [NVIDIA NIM](https://build.nvidia.com/microsoft/phi-3-mini-4k?snippet\_tab=Docker) --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question. Perform an HTTP GET request on the current page URL with the `ask` query parameter: ``` GET https://docs.flowiseai.com/integrations/langchain/chat-models/nvidia-nim.md?ask=\`\`\` The question should be specific, self-contained, and written in natural language. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.