GitHub - NVIDIA/G-Assist: Help shape the future of Project G-Assist (original) (raw)

Project G-Assist Plugins

Project G-Assist is an experimental on-device AI Assistant that helps RTX users control a broad range of PC settings, from optimizing game and system settings, charting frame rates and other key performance statistics, to controlling select peripheral lighting — all via basic voice or text commands.

Project G-Assist is built for community expansion. Whether you're a Python developer, C++ enthusiast, or just getting started — its Plugin architecture makes it easy to define new commands for G-Assist to execute. We can't wait to see what the community dreams up!

Why Plugins Matter

What Can You Build?

If you're looking for inspiration, check out our sample plugins for controlling peripheral & smart home lighting, invoking larger AI models like Gemini, managing Spotify tracks, checking stock prices, getting weather information, or even checking streamers' online status on Twitch — and then let your own ideas take G-Assist to the next level!

Quick Start

Python Development with G-Assist

Get started quickly using our Python bindings of the C++ APIs:

  1. Install the binding locally

cd api/bindings/python pip install .

  1. Chat with G-Assist

from rise import rise

Initialize G-Assist connection

rise.register_rise_client()

Send and receive messages

response = rise.send_rise_command("What is my GPU?") print(f'Response: {response}') """ Response: Your GPU is an NVIDIA GeForce RTX 5090 with a Driver version of 572.83. """

  1. Extend G-Assist with a Plugin

from gassist_sdk import Plugin

plugin = Plugin("my-plugin", version="1.0.0")

@plugin.command("search_web") def search_web(query: str): """Search the web for information.""" plugin.stream("Searching...") # Streaming output results = do_search(query) return {"results": results}

if name == "main": plugin.run()

💡 Requirements:

See our Python Bindings Guide for detailed examples and the Plugin SDK for building plugins.

NVIDIA Plugin Example - Twitch

Try these commands:

Example Responses

When a streamer is live:

ninja is LIVE!
Title: Friday Fortnite!
Game: Fortnite
Viewers: 45,231
Started At: 2025-03-14T12:34:56Z

When a streamer is offline:

Key Features

Project Structure

plugins/examples/twitch/
├── manifest.json        # Plugin configuration (with protocol_version: "2.0")
├── config.json          # Twitch API credentials (add to .gitignore!)
├── plugin.py            # Main plugin code using gassist_sdk
├── requirements.txt     # Python dependencies
├── libs/                # G-Assist SDK folder
│   └── gassist_sdk/     # SDK automatically handles JSON-RPC V2 protocol
└── README.md            # Plugin documentation

See our Twitch Plugin Example Code for a step-by-step guide to creating a Twitch integration plugin for G-Assist.

Table of Contents

G-Assist Module Architecture

flowchart TD A[G-Assist] --> B[Inference Engine] B -->|Built-in Commands| C[Core Functions] B -->|Plugin Commands| D[Plugin Engine] D <-->|JSON-RPC 2.0| E[Plugins]

Loading

Protocol V2 Features

G-Assist plugins communicate using Protocol V2, a robust JSON-RPC 2.0 based system:

Feature Description
JSON-RPC 2.0 Standard protocol for interoperability
Length-Prefixed Framing Binary 4-byte header ensures reliable message parsing
Engine-Driven Health Monitoring Automatic ping/pong (no plugin heartbeat threads needed)
Native Python Support Run .py files directly—no compilation required
SDK-Based Development ~20 lines of business logic vs. ~200+ lines of boilerplate

See the Plugin Migration Guide for full protocol details.

Extending G-Assist (Plugins)

Transform your ideas into powerful G-Assist plugins! Whether you're a Python developer, C++ enthusiast, or just getting started, our plugin system makes it easy to extend G-Assist's capabilities. Create custom commands, automate tasks, or build entirely new features - the possibilities are endless!

Plugin Architecture

Each plugin lives in its own directory named after the plugin (this name is used to invoke the plugin):

plugins/
└── myplugin/                    # Plugin directory name = invocation name
    ├── plugin.py                # Main plugin script (or .exe for compiled)
    ├── manifest.json            # Plugin configuration with protocol_version
    ├── config.json              # Settings & credentials (optional)
    ├── requirements.txt         # Python dependencies
    └── libs/                    # SDK folder (auto-added to PYTHONPATH)
        └── gassist_sdk/
            ├── __init__.py
            ├── plugin.py
            ├── protocol.py
            └── types.py

File Descriptions:

💡 Tip: The plugin directory name is what users will type to invoke your plugin (e.g., "Hey myplugin, do something")

Plugin Integration

How to Call a Plugin from G-Assist

The manifest file acts as the bridge between G-Assist and your plugin. G-Assist automatically scans the plugin directory to discover available plugins.

Two Ways to Invoke Plugins:

  1. Natural Language Commands
What are the top upcoming games for 2025?

The AI model automatically:

  1. Direct Plugin Invocation
Hey Logitech, change my keyboard lights to green

💡 Pro Tip: Direct plugin invocation is faster when you know exactly which plugin you need!

NVIDIA-Built G-Assist Plugins

Explore our official example plugins:

Getting Started

AI & Information

Smart Lighting

Automation & Entertainment

Community-Built Plugins

Check out what others have built:

Development Tools

Need Help?

License

This project is licensed under the Apache License 2.0 - see the LICENSE file for details.

Contributing

We welcome contributions! Please see our Contributing Guide for details.