Galadriel | liteLLM (original) (raw)

LiteLLM supports all models on Galadriel.

from litellm import completion
import os

os.environ['GALADRIEL_API_KEY'] = ""
response = completion(
    model="galadriel/llama3.1", 
    messages=[
       {"role": "user", "content": "hello from litellm"}
   ],
)
print(response)

from litellm import completion
import os

os.environ['GALADRIEL_API_KEY'] = ""
response = completion(
    model="galadriel/llama3.1", 
    messages=[
       {"role": "user", "content": "hello from litellm"}
   ],
    stream=True
)

for chunk in response:
    print(chunk)

We support ALL Galadriel AI models, just set galadriel/ as a prefix when sending completion requests

We support both the complete model name and the simplified name match.

You can specify the model name either with the full name or with a simplified version e.g. llama3.1:70b