Skip to main content

OpenAI (Text Completion)

LiteLLM supports OpenAI text completion models

Required API Keys

import os 
os.environ["OPENAI_API_KEY"] = "your-api-key"

Usage

import os 
from litellm import completion

os.environ["OPENAI_API_KEY"] = "your-api-key"

# openai call
response = completion(
model = "gpt-3.5-turbo-instruct",
messages=[{ "content": "Hello, how are you?","role": "user"}]
)

Usage - LiteLLM Proxy Server

Here's how to call OpenAI models with the LiteLLM Proxy Server

1. Save key in your environment

export OPENAI_API_KEY=""

2. Start the proxy

model_list:
- model_name: gpt-3.5-turbo
litellm_params:
model: openai/gpt-3.5-turbo # The `openai/` prefix will call openai.chat.completions.create
api_key: os.environ/OPENAI_API_KEY
- model_name: gpt-3.5-turbo-instruct
litellm_params:
model: text-completion-openai/gpt-3.5-turbo-instruct # The `text-completion-openai/` prefix will call openai.completions.create
api_key: os.environ/OPENAI_API_KEY

Use this to add all openai models with one API Key. WARNING: This will not do any load balancing This means requests to gpt-4, gpt-3.5-turbo , gpt-4-turbo-preview will all go through this route

model_list:
- model_name: "*" # all requests where model not in your config go to this deployment
litellm_params:
model: openai/* # set `openai/` to use the openai route
api_key: os.environ/OPENAI_API_KEY
$ litellm --model gpt-3.5-turbo-instruct

# Server running on http://0.0.0.0:4000

3. Test it

curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--data ' {
"model": "gpt-3.5-turbo-instruct",
"messages": [
{
"role": "user",
"content": "what llm are you"
}
]
}
'
import openai
client = openai.OpenAI(
api_key="anything",
base_url="http://0.0.0.0:4000"
)

# request sent to model set on litellm proxy, `litellm --model`
response = client.chat.completions.create(model="gpt-3.5-turbo-instruct", messages = [
{
"role": "user",
"content": "this is a test request, write a short poem"
}
])

print(response)

from langchain.chat_models import ChatOpenAI
from langchain.prompts.chat import (
ChatPromptTemplate,
HumanMessagePromptTemplate,
SystemMessagePromptTemplate,
)
from langchain.schema import HumanMessage, SystemMessage

chat = ChatOpenAI(
openai_api_base="http://0.0.0.0:4000", # set openai_api_base to the LiteLLM Proxy
model = "gpt-3.5-turbo-instruct",
temperature=0.1
)

messages = [
SystemMessage(
content="You are a helpful assistant that im using to make a test request to."
),
HumanMessage(
content="test from litellm. tell me why it's amazing in 1 sentence"
),
]
response = chat(messages)

print(response)

OpenAI Text Completion Models / Instruct Models

Model NameFunction Call
gpt-3.5-turbo-instructresponse = completion(model="gpt-3.5-turbo-instruct", messages=messages)
gpt-3.5-turbo-instruct-0914response = completion(model="gpt-3.5-turbo-instruct-0914", messages=messages)
text-davinci-003response = completion(model="text-davinci-003", messages=messages)
ada-001response = completion(model="ada-001", messages=messages)
curie-001response = completion(model="curie-001", messages=messages)
babbage-001response = completion(model="babbage-001", messages=messages)
babbage-002response = completion(model="babbage-002", messages=messages)
davinci-002response = completion(model="davinci-002", messages=messages)