Skip to main content

GradientAI

https://digitalocean.com/products/gradientai

LiteLLM provides native support for GradientAI models. To use a GradientAI model, specify it as gradient_ai/<model-name> in your LiteLLM requests.

API Key & Endpoint​

Set your credentials and endpoint as environment variables:

import os
os.environ['GRADIENT_AI_API_KEY'] = "your-api-key"
os.environ['GRADIENT_AI_AGENT_ENDPOINT'] = "https://api.gradient_ai.com/api/v1/chat" # default endpoint

Sample Usage​

from litellm import completion
import os

os.environ['GRADIENT_AI_API_KEY'] = "your-api-key"
response = completion(
model="gradient_ai/model-name",
messages=[
{"role": "user", "content": "Hello, how are you?"}
],
)
print(response.choices[0].message.content)

Streaming Example​

from litellm import completion
import os

os.environ['GRADIENT_AI_API_KEY'] = "your-api-key"
response = completion(
model="gradient_ai/model-name",
messages=[
{"role": "user", "content": "Write a story about a robot learning to love"}
],
stream=True,
)

for chunk in response:
print(chunk.choices[0].delta.content or "", end="")

Supported Parameters​

ParameterTypeDescription
temperaturefloatControls randomness (0.0-2.0)
top_pfloatNucleus sampling parameter (0.0-1.0)
max_tokensintMaximum tokens to generate
max_completion_tokensintAlternative to max_tokens
streamboolWhether to stream the response
kintTop results to return from knowledge bases
retrieval_methodstringRetrieval strategy (rewrite/step_back/sub_queries/none)
frequency_penaltyfloatPenalizes repeated tokens (-2.0 to 2.0)
presence_penaltyfloatPenalizes tokens based on presence (-2.0 to 2.0)
stopstring/listSequences to stop generation
kb_filtersList[Dict]Filters for knowledge base retrieval
instruction_overridestringOverride agent's default instruction
include_retrieval_infoboolInclude document retrieval metadata
include_guardrails_infoboolInclude guardrail trigger metadata
provide_citationsboolInclude citations in response

For more details, see DigitalOcean GradientAI documentation.