Using your own API keys in GitHub Models - GitHub Docs (original) (raw)

Learn how to integrate your preferred custom models with GitHub Models by using your own LLM API keys.

Who can use this feature?

Organization owners can add custom models to GitHub Models for their organization

Note

The ability to bring your own keys (BYOK) to use custom models with GitHub Models for organizations on GitHub is in public preview and subject to change.

Model support is currently limited to OpenAI and AzureAI.

You can bring your own API keys (BYOK) to GitHub Models, and enable teams to use your preferred large language model (LLM) providers across tools like Prompts, Playground, and Models in Actions. For more information about GitHub Models, see About GitHub Models.

To learn about billing and pricing, see GitHub Models billing.

Why bring your own API keys?

As an organization owner, you may have specific requirements for governance, data security, and compliance. By setting up your own API keys, you can:

• Governance and compliance: Choose LLM providers that comply with your organization's policies and regulatory requirements.
• Cost management: Align with your existing payment methods, contracts, credits, or negotiated rates, and avoid usage overages.
• Visibility and control: Manage which models your team can access, and monitor usage through your provider's existing dashboards and billing.
• Flexibility: Support custom or specialized models that your organization already uses.

Setting up your API keys to add custom models

Important

We highly recommend adhering to the principle of least privilege by assigning only the minimum necessary scopes to your API keys.

You must first add the relevant API keys for the organization. After that, your can specify or enable the custom models you wish to make available to users.

Adding API keys

1. In the upper-right corner of GitHub, click your profile picture, then click Organizations.
2. Next to the organization, click Settings.
3. In the "Code, planning, and automation" section of the sidebar, click Models dropdown. Then click Custom models. The "Custom Models" page displays the API keys and custom models added to the organization.
4. Click Add API key.
5. In the "Add the API key" dialog, provide details about your key. Name and API key are compulsory fields.
6. Click Save.

Enabling custom models

1. In the upper-right corner of GitHub, click your profile picture, then click Organizations.
2. Next to the organization, click Settings.
3. In the "Code, planning, and automation" section of the sidebar, click the Models dropdown. Then click Development.
4. Under "Models permissions," select All publishers to enable models added by API keys.
   • If this option isn't available, you need to allow the use of the model in the organization. See Managing your team's model usage.
5. Optionally, select Only select models to create a custom list of enabled or disabled models. This allows you to control which models are available to your organization.

Creating an enabled or disabled models list

To control which models are available to your organization, create an enabled or disabled models list as follows:

1. In the upper-right corner of GitHub, click your profile picture, then click Organizations.
2. Next to the organization, click Settings.
3. In the "Code, planning, and automation" section of the sidebar, click the Models dropdown. Then click Development.
4. Under "Models permissions," select Only select models, then select Enabled list from the dropdown to create a list of allowed models.
5. Optionally, select Disabled list from the dropdown if you want to create a list of blocked models.
6. Above the table of models, click Add models or publishers.
7. In the "Select models and publishers to allow" dialog, select the models and publishers you want to include in your list.
8. Click Update enabled list to save your changes.

Next steps

Now that you've enabled your custom models in GitHub Models, learn how to:

• Experiment with your custom model in the playground. See Experimenting with AI models in the playground.
• Store prompts on GitHub so that you can iterate to fine-tune your prompts, and share them with stakeholders. See Storing prompts in GitHub repositories.
• Launch your AI application. See Going to production.