Skip to main content

Snowflake

PropertyDetails
DescriptionThe Snowflake Cortex LLM REST API lets you access the COMPLETE function via HTTP POST requests
Provider Route on LiteLLMsnowflake/
Link to Provider DocSnowflake ↗
Base URLhttps://{account-id}.snowflakecomputing.com/api/v2/cortex/inference:complete/
Supported OpenAI Endpoints/chat/completions, /completions

Currently, Snowflake's REST API does not have an endpoint for snowflake-arctic-embed embedding models. If you want to use these embedding models with Litellm, you can call them through our Hugging Face provider.

Find the Arctic Embed models here on Hugging Face.

Supported OpenAI Parameters​

    "temperature",
    "max_tokens",
    "top_p",
    "response_format"

API KEYS​

Snowflake does have API keys. Instead, you access the Snowflake API with your JWT token and account identifier.

import os 
os.environ["SNOWFLAKE_JWT"] = "YOUR JWT"
os.environ["SNOWFLAKE_ACCOUNT_ID"] = "YOUR ACCOUNT IDENTIFIER"

Usage​

from litellm import completion

## set ENV variables
os.environ["SNOWFLAKE_JWT"] = "YOUR JWT"
os.environ["SNOWFLAKE_ACCOUNT_ID"] = "YOUR ACCOUNT IDENTIFIER"

# Snowflake call
response = completion(
    model="snowflake/mistral-7b", 
    messages = [{ "content": "Hello, how are you?","role": "user"}]
)

Usage with LiteLLM Proxy​

1. Required env variables​

export SNOWFLAKE_JWT=""
export SNOWFLAKE_ACCOUNT_ID = ""

2. Start the proxy~​

model_list:
  - model_name: mistral-7b
    litellm_params:
        model: snowflake/mistral-7b
        api_key: YOUR_API_KEY
        api_base: https://YOUR-ACCOUNT-ID.snowflakecomputing.com/api/v2/cortex/inference:complete

litellm --config /path/to/config.yaml

3. Test it​

curl --location 'http://0.0.0.0:4000/chat/completions' \
--header 'Content-Type: application/json' \
--data ' {
      "model": "snowflake/mistral-7b",
      "messages": [
        {
          "role": "user",
          "content": "Hello, how are you?"
        }
      ]
    }
'