Skip to main content

Oracle Cloud Infrastructure (OCI)

LiteLLM supports the following models for OCI on-demand GenAI API.

Check the OCI Models List to see if the model is available for your region.

  • meta.llama-4-maverick-17b-128e-instruct-fp8

  • meta.llama-4-scout-17b-16e-instruct

  • meta.llama-3.3-70b-instruct

  • meta.llama-3.2-90b-vision-instruct

  • meta.llama-3.1-405b-instruct

  • xai.grok-4

  • xai.grok-3

  • xai.grok-3-fast

  • xai.grok-3-mini

  • xai.grok-3-mini-fast

Authentication​

LiteLLM uses OCI signing key authentication. Follow the official Oracle tutorial to create a signing key and obtain the following parameters:

  • user
  • fingerprint
  • tenancy
  • region
  • key_file

Usage​

Input the parameters obtained from the OCI signing key creation process into the completion function.

import os
from litellm import completion

messages = [{"role": "user", "content": "Hey! how's it going?"}]
response = completion(
    model="oci/xai.grok-4",
    messages=messages,
    oci_region=<your_oci_region>,
    oci_user=<your_oci_user>,
    oci_fingerprint=<your_oci_fingerprint>,
    oci_tenancy=<your_oci_tenancy>,
    oci_key=<string_with_content_of_oci_key>,
    oci_compartment_id=<oci_compartment_id>,
)
print(response)

Usage - Streaming​

Just set stream=True when calling completion.

import os
from litellm import completion

messages = [{"role": "user", "content": "Hey! how's it going?"}]
response = completion(
    model="oci/xai.grok-4",
    messages=messages,
    stream=True,
    oci_region=<your_oci_region>,
    oci_user=<your_oci_user>,
    oci_fingerprint=<your_oci_fingerprint>,
    oci_tenancy=<your_oci_tenancy>,
    oci_key=<string_with_content_of_oci_key>,
    oci_compartment_id=<oci_compartment_id>,
)
for chunk in response:
    print(chunk["choices"][0]["delta"]["content"])  # same as openai format