LiteLLM Model Configuration

Overview

This guide explains how to add and configure AI models in LiteLLM Proxy for AI/Run CodeMie. LiteLLM acts as a unified gateway to multiple LLM providers.

See the official LiteLLM documentation for supported providers and models.

LiteLLM Proxy Config Structure

LiteLLM Proxy uses a config.yaml file to define model configurations and other settings.

note

In AI/Run CodeMie, this configuration is managed through Helm chart values that generate the underlying LiteLLM config.

The config.yaml file contains five main sections:

model_list: Array of model routing configurations
litellm_settings: Module-level LiteLLM settings
general_settings: Global proxy settings (authentication, alerting, etc.)
router_settings: Load balancing and routing behavior
credential_list: Authentication credentials for different providers

Example: Config Structure

model_list:
  - model_name: fake-model-endpoint
    litellm_params:
      model: fake-model-name
    model_info:
      id: fake-model-unique-id
      base_model: fake-base-model-id
      label: "Fake Model Name for Testing"

litellm_settings:
  # ... additional configuration fields

general_settings:
  master_key: sk-1234

router_settings:
  # ... additional configuration fields

credential_list:
  - credential_name: default_aws_credential
    aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID
    aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY

Model Configuration Structure

Each model entry in the model_list array consists of three main sections:

model_name

The model name that users specify when making API calls to LiteLLM

Example: Basic Configuration

model_list:
  - model_name: claude-sonnet-4-5-20250929
    # ... additional configuration fields

  - model_name: claude-sonnet-4-6
  # ... additional configuration fields

Multiple entries can share the same model_name for load balancing

Example: Load Balancing Configuration

model_list:
  - model_name: claude-sonnet-4-5-20250929
    model_info:
      id: claude-sonnet-4-5-20250929-unique-id-0
    # ... additional configuration fields

  - model_name: claude-sonnet-4-5-20250929
    model_info:
      id: claude-sonnet-4-5-20250929-unique-id-1
    # ... additional configuration fields

litellm_params

model: Backend provider model identifier

Example: Model Identifiers

model_list:
  - model_name: claude-sonnet-4-5-20250929
    litellm_params:
      model: bedrock/us.anthropic.claude-sonnet-4-5-20250929-v1:0
    # ... additional configuration fields

  - model_name: gpt-5-2025-08-07
    litellm_params:
      model: azure/gpt-5-2025-08-07
    # ... additional configuration fields

  - model_name: gemini-3.1-pro
    litellm_params:
      model: vertex_ai/gemini-3.1-pro-preview
    # ... additional configuration fields

litellm_credential_name: Reference to authentication credentials configured in secrets

Example: Credentials Configuration

model_list:
  - model_name: claude-sonnet-4-5-20250929
    litellm_params:
      model: bedrock/us.anthropic.claude-sonnet-4-5-20250929-v1:0
      litellm_credential_name: default_aws_bedrock_credential
    # ... additional configuration fields

  - model_name: gpt-5-2025-08-07
    litellm_params:
      model: azure/gpt-5-2025-08-07
      litellm_credential_name: default_azure_openai_credential
    # ... additional configuration fields

credential_list:
  - credential_name: default_aws_bedrock_credential
    credential_values:
      aws_access_key_id: os.environ/AWS_ACCESS_KEY_ID
      aws_secret_access_key: os.environ/AWS_SECRET_ACCESS_KEY

  - credential_name: default_azure_openai_credential
    credential_values:
      tenant_id: os.environ/AZURE_TENANT_ID
      client_id: os.environ/AZURE_CLIENT_ID
      client_secret: os.environ/AZURE_CLIENT_SECRET

model_info

id: Unique identifier for the specific model instance
base_model: Base model identifier used to retrieve defaults and pricing from LiteLLM models database.

info
Set base_model accurately – the same model across different providers or regions may have different costs and capabilities.
label: Human-readable display name shown in CodeMie UI
forbidden_for_web: (Optional) Set to true to hide this model from CodeMie UI

default_for_categories: Array of categories for default model selection.

Available categories:

Category	Description
`global`	Default model for all tasks
`code`	Default model for code tasks
`chat`	Default model for conversations

Required defaults

At least one chat model and one embedding model must have the global category assigned.

Examples: Default Models

# Note: When using load balancing with multiple entries of the same model_name,
# all entries must have the default_for_categories field

# Default chat model
- model_name: gpt-4.1
  model_info:
    default_for_categories: [global]
  # ... additional configuration fields

# Model without default_for_categories - not selected as default for any category
- model_name: gpt-5-2-2025-12-11
  # ... additional configuration fields

# Default code model
- model_name: claude-4-5-sonnet
  model_info:
    default_for_categories: [code]
  # ... additional configuration fields

# Model without default_for_categories - not selected as default for any category
- model_name: claude-sonnet-4-6
  # ... additional configuration fields

# Default embedding model
- model_name: codemie-text-embedding-ada-002
  model_info:
    default_for_categories: [global]
  # ... additional configuration fields

Model Configuration Examples

This guide provides tested and verified model configurations currently used in AI/Run CodeMie production. While not all steps for adding new models are covered (refer to the official LiteLLM documentation for comprehensive setup instructions), working examples from the production environment are shared and can be adapted for any deployment.

tip

All configuration examples below have been validated and are actively used in CodeMie. They can be used as templates when adding similar models to the environment.

Models Reference

Configuration examples for these models can be found in the provider-specific sections below.

AWS Bedrock Models

Model Name	Description
`claude-4-5-sonnet`	Claude 4.5 Sonnet
`claude-sonnet-4-6`	Claude Sonnet 4.6
`claude-opus-4-5-20251101`	Claude Opus 4.5
`claude-opus-4-6-20260205`	Claude Opus 4.6
`claude-haiku-4-5-20251001`	Claude Haiku 4.5
`amazon.titan-embed-text-v2:0`	Amazon Titan Embeddings

Azure OpenAI Models

Model Name	Description
`gpt-4.1`	GPT-4.1
`gpt-4.1-mini`	GPT-4.1 mini
`gpt-5-2025-08-07`	GPT-5
`gpt-5-mini-2025-08-07`	GPT-5 mini
`gpt-5-nano-2025-08-07`	GPT-5 nano
`gpt-5-1-codex-2025-11-13`	GPT-5.1 Codex
`gpt-5-2-2025-12-11`	GPT-5.2
`gpt-5.3-codex-2026-02-24`	GPT-5.3 Codex
`gpt-5.4-2026-03-05`	GPT-5.4
`o1`	o1
`o3-mini`	o3 mini
`o3-2025-04-16`	o3
`o4-mini-2025-04-16`	o4 mini
`codemie-text-embedding-ada-002`	Text Embedding Ada-002

Vertex AI Models

Model Name	Description
`claude-4-5-sonnet-vertex`	Claude 4.5 Sonnet
`gemini-3-flash`	Gemini 3 Flash
`gemini-3.1-pro`	Gemini 3.1 Pro
`text-embedding-005`	Text Embedding

GitHub Copilot Models

Model Name	Description
`github-copilot-gpt-5`	GPT-5
`github-copilot-gpt-5-mini`	GPT-5 Mini
`github-copilot-gpt-5-1`	GPT-5.1
`github-copilot-gpt-5-1-codex-max`	GPT-5.1 Codex Max
`github-copilot-gpt-5-2`	GPT-5.2
`github-copilot-claude-haiku-4.5`	Claude Haiku 4.5
`github-copilot-claude-sonnet-4.5`	Claude Sonnet 4.5
`github-copilot-claude-opus-4-5`	Claude Opus 4.5

AWS Bedrock Provider Examples

AWS Bedrock Region Configuration

To use a different AWS region, modify the aws_region_name parameter in the model's configuration.

model_list:
  - model_name: claude-sonnet-4-6
    litellm_params:
      aws_region_name: us-west-2

Claude Sonnet

Claude Sonnet 4.5

Claude 4.5 Sonnet

# US Region
- model_name: claude-4-5-sonnet
  litellm_params:
    model: bedrock/us.anthropic.claude-sonnet-4-5-20250929-v1:0
    litellm_credential_name: default_aws_bedrock_credential
    aws_region_name: us-west-2
  model_info:
    id: claude-4-5-sonnet-us-west-2
    base_model: us.anthropic.claude-sonnet-4-5-20250929-v1:0
    label: "Bedrock Claude 4.5 Sonnet"

# EU Region
- model_name: claude-4-5-sonnet
  litellm_params:
    model: bedrock/eu.anthropic.claude-sonnet-4-5-20250929-v1:0
    litellm_credential_name: default_aws_bedrock_credential
    aws_region_name: eu-central-1
  model_info:
    id: claude-4-5-sonnet-eu-central-1
    base_model: eu.anthropic.claude-sonnet-4-5-20250929-v1:0
    label: "Bedrock Claude 4.5 Sonnet"

# Global routing
- model_name: claude-4-5-sonnet
  litellm_params:
    model: bedrock/global.anthropic.claude-sonnet-4-5-20250929-v1:0
    litellm_credential_name: default_aws_bedrock_credential
    aws_region_name: us-west-2
  model_info:
    id: claude-4-5-sonnet-global-us-west-2
    base_model: global.anthropic.claude-sonnet-4-5-20250929-v1:0
    label: "Bedrock Claude 4.5 Sonnet"

Claude Sonnet 4.6

Claude Sonnet 4.6

# US Region
- model_name: claude-sonnet-4-6
  litellm_params:
    model: bedrock/us.anthropic.claude-sonnet-4-6
    litellm_credential_name: default_aws_bedrock_credential
    aws_region_name: us-east-1
  model_info:
    id: claude-sonnet-4-6-us-east-1
    base_model: us.anthropic.claude-sonnet-4-6
    label: "Bedrock Claude Sonnet 4.6"

Claude Haiku

Claude Haiku 4.5

Claude Haiku 4.5

# US Region
- model_name: claude-haiku-4-5-20251001
  litellm_params:
    model: bedrock/converse/us.anthropic.claude-haiku-4-5-20251001-v1:0
    litellm_credential_name: default_aws_bedrock_credential
    aws_region_name: us-west-2
  model_info:
    id: claude-4-5-haiku-us-west-2
    base_model: us.anthropic.claude-haiku-4-5-20251001-v1:0
    label: "Bedrock Claude Haiku 4.5"

# EU Region
- model_name: claude-haiku-4-5-20251001
  litellm_params:
    model: bedrock/converse/eu.anthropic.claude-haiku-4-5-20251001-v1:0
    litellm_credential_name: default_aws_bedrock_credential
    aws_region_name: eu-central-1
  model_info:
    id: claude-4-5-haiku-eu-central-1
    base_model: eu.anthropic.claude-haiku-4-5-20251001-v1:0
    label: "Bedrock Claude Haiku 4.5"

Claude Opus

Claude Opus 4.5

Claude Opus 4.5

# US Region
- model_name: claude-opus-4-5-20251101
  litellm_params:
    model: bedrock/us.anthropic.claude-opus-4-5-20251101-v1:0
    litellm_credential_name: default_aws_bedrock_credential
    aws_region_name: us-west-2
  model_info:
    id: claude-opus-4-5-20251101-us-west-2
    base_model: us.anthropic.claude-opus-4-5-20251101-v1:0
    label: "Bedrock Claude Opus 4.5"

# EU Region
- model_name: claude-opus-4-5-20251101
  litellm_params:
    model: bedrock/eu.anthropic.claude-opus-4-5-20251101-v1:0
    litellm_credential_name: default_aws_bedrock_credential
    aws_region_name: eu-central-1
  model_info:
    id: claude-opus-4-5-20251101-eu-central-1
    base_model: anthropic.claude-opus-4-5-20251101-v1:0
    label: "Bedrock Claude Opus 4.5"

Claude Opus 4.6

Claude Opus 4.6

# US Region
- model_name: claude-opus-4-6-20260205
  litellm_params:
    model: bedrock/us.anthropic.claude-opus-4-6-v1
    litellm_credential_name: default_aws_bedrock_credential
    aws_region_name: us-west-2
  model_info:
    id: claude-opus-4-6-20260205-us-west-2
    base_model: us.anthropic.claude-opus-4-6-v1
    label: "Bedrock Claude Opus 4.6"

# EU Region
- model_name: claude-opus-4-6-20260205
  litellm_params:
    model: bedrock/eu.anthropic.claude-opus-4-6-v1
    litellm_credential_name: default_aws_bedrock_credential
    aws_region_name: eu-central-1
  model_info:
    id: claude-opus-4-6-20260205-eu-central-1
    base_model: eu.anthropic.claude-opus-4-6-v1
    label: "Bedrock Claude Opus 4.6"

Amazon Titan

Amazon Titan

# US Region
- model_name: amazon.titan-embed-text-v2:0
  litellm_params:
    model: bedrock/amazon.titan-embed-text-v2:0
    litellm_credential_name: default_aws_bedrock_credential
    aws_region_name: us-west-2
  model_info:
    id: titan-us-west-2
    base_model: amazon.titan-embed-text-v2:0
    label: "Titan Embed Text v2.0"

# EU Region
- model_name: amazon.titan-embed-text-v2:0
  litellm_params:
    model: bedrock/amazon.titan-embed-text-v2:0
    litellm_credential_name: default_aws_bedrock_credential
    aws_region_name: eu-central-1
  model_info:
    id: titan-eu-central-1
    base_model: amazon.titan-embed-text-v2:0
    label: "Titan Embed Text v2.0"

Azure OpenAI Provider Examples

Azure OpenAI Region Configuration

For Azure, the region is determined by the endpoint URL in the api_base parameter.

model_list:
  - model_name: gpt-5-2-2025-12-11
    litellm_params:
      model: azure/gpt-5.2-2025-12-11
      api_base: https://your-resource.openai.azure.com/
      litellm_credential_name: default_azure_openai_credential

GPT-4.1 series

GPT-4.1

GPT-4.1

# US Region
- model_name: gpt-4.1
  litellm_params:
    model: azure/gpt-4.1-2025-04-14
    api_base: https://api-base-eastus-0.openai.azure.com/
    litellm_credential_name: default_azure_openai_credential
  model_info:
    id: gpt-4.1-eastus-0
    base_model: azure/gpt-4.1-2025-04-14
    label: "GPT-4.1 2025-04-14"

# EU Region
- model_name: gpt-4.1
  litellm_params:
    model: azure/gpt-4.1-2025-04-14
    api_base: https://api-base-eastus-0.openai.azure.com/
    litellm_credential_name: default_azure_openai_credential
  model_info:
    id: gpt-4.1-swedencentral-0
    base_model: azure/gpt-4.1-2025-04-14
    label: "GPT-4.1 2025-04-14"

GPT-4.1 Mini

GPT-4.1 Mini

- model_name: gpt-4.1-mini
  litellm_params:
    model: azure/gpt-4.1-mini-2025-04-14
    api_base: https://api-base-swedencentral-0.openai.azure.com/
    litellm_credential_name: default_azure_openai_credential
  model_info:
    id: gpt-4.1-mini-swedencentral-0
    base_model: azure/gpt-4.1-mini-2025-04-14
    label: "GPT-4.1 mini 2025-04-14"

GPT-5 series

GPT-5

GPT-5

# US Region
- model_name: gpt-5-2025-08-07
  litellm_params:
    model: azure/gpt-5-2025-08-07
    api_base: https://api-base-eastus-0.openai.azure.com/
    litellm_credential_name: default_azure_openai_credential
  model_info:
    id: gpt-5-2025-08-07-eastus-0
    base_model: azure/gpt-5-2025-08-07
    label: "GPT-5 2025-08-07"
    top_p: false

# EU Region
- model_name: gpt-5-2025-08-07
  litellm_params:
    model: azure/gpt-5-2025-08-07
    api_base: https://api-base-swedencentral-0.openai.azure.com/
    litellm_credential_name: default_azure_openai_credential
  model_info:
    id: gpt-5-2025-08-07-swedencentral-0
    base_model: azure/gpt-5-2025-08-07
    label: "GPT-5 2025-08-07"
    top_p: false

GPT-5 Mini

GPT-5 Mini

# US Region
- model_name: gpt-5-mini-2025-08-07
  litellm_params:
    model: azure/gpt-5-mini-2025-08-07
    api_base: https://api-base-eastus2-0.openai.azure.com/
    litellm_credential_name: default_azure_openai_credential
  model_info:
    id: gpt-5-mini-2025-08-07-eastus2-0
    base_model: azure/gpt-5-mini-2025-08-07
    label: "GPT-5 Mini 2025-08-07"
    top_p: false

# EU Region
- model_name: gpt-5-mini-2025-08-07
  litellm_params:
    model: azure/gpt-5-mini-2025-08-07
    api_base: https://api-base-swedencentral-0.openai.azure.com/
    litellm_credential_name: default_azure_openai_credential
  model_info:
    id: gpt-5-mini-2025-08-07-swedencentral-0
    base_model: azure/gpt-5-mini-2025-08-07
    label: "GPT-5 Mini 2025-08-07"
    top_p: false

GPT-5 Nano

GPT-5 Nano

# US Region
- model_name: gpt-5-nano-2025-08-07
  litellm_params:
    model: azure/gpt-5-nano-2025-08-07
    api_base: https://api-base-eastus2-0.openai.azure.com/
    litellm_credential_name: default_azure_openai_credential
  model_info:
    id: gpt-5-nano-2025-08-07-eastus2-0
    base_model: azure/gpt-5-nano-2025-08-07
    label: "GPT-5 Nano 2025-08-07"
    top_p: false

# EU Region
- model_name: gpt-5-nano-2025-08-07
  litellm_params:
    model: azure/gpt-5-nano-2025-08-07
    api_base: https://api-base-swedencentral-0.openai.azure.com/
    litellm_credential_name: default_azure_openai_credential
  model_info:
    id: gpt-5-nano-2025-08-07-swedencentral-0
    base_model: azure/gpt-5-nano-2025-08-07
    label: "GPT-5 Nano 2025-08-07"
    top_p: false

GPT-5.1 series

GPT-5.1 Codex

GPT-5.1 Codex

# US Region
- model_name: gpt-5-1-codex-2025-11-13
  litellm_params:
    model: azure/gpt-5.1-codex-2025-11-13
    api_base: https://api-base-eastus2-0.openai.azure.com/
    litellm_credential_name: default_azure_openai_credential
  model_info:
    id: gpt-5-1-codex-2025-11-13-eastus2-0
    base_model: azure/gpt-5.1-codex-2025-11-13
    label: "GPT-5.1 Codex 2025-11-13"
    forbidden_for_web: true

# EU Region
- model_name: gpt-5-1-codex-2025-11-13
  litellm_params:
    model: azure/gpt-5.1-codex-2025-11-13
    api_base: https://api-base-swedencentral-0.openai.azure.com/
    litellm_credential_name: default_azure_openai_credential
  model_info:
    id: gpt-5-1-codex-2025-11-13-swedencentral-0
    base_model: azure/gpt-5.1-codex-2025-11-13
    label: "GPT-5.1 Codex 2025-11-13"
    forbidden_for_web: true

GPT-5.2 series

GPT-5.2

GPT-5.2

# US Region
- model_name: gpt-5-2-2025-12-11
  litellm_params:
    model: azure/gpt-5.2-2025-12-11
    api_base: https://api-base-eastus2-0.openai.azure.com/
    litellm_credential_name: default_azure_openai_credential
  model_info:
    id: gpt-5-2-2025-12-11-eastus2-0
    base_model: azure/gpt-5.2-2025-12-11
    label: "GPT-5.2 2025-12-11"

# EU Region
- model_name: gpt-5-2-2025-12-11
  litellm_params:
    model: azure/gpt-5.2-2025-12-11
    api_base: https://api-base-swedencentral-0.openai.azure.com/
    litellm_credential_name: default_azure_openai_credential
  model_info:
    id: gpt-5-2-2025-12-11-swedencentral-0
    base_model: azure/gpt-5.2-2025-12-11
    label: "GPT-5.2 2025-12-11"

GPT-5.3 series

GPT-5.3 Codex

GPT-5.3 Codex

# US Region
- model_name: gpt-5.3-codex-2026-02-24
  litellm_params:
    model: azure/gpt-5.3-codex-2026-02-24
    api_base: https://api-base-eastus2-0.openai.azure.com/
    litellm_credential_name: default_azure_openai_credential
  model_info:
    id: gpt-5-3-codex-2026-02-24-eastus2-0
    base_model: azure/gpt-5.3-codex
    label: "GPT-5.3 Codex 2026-02-24"
    forbidden_for_web: true

# EU Region
- model_name: gpt-5.3-codex-2026-02-24
  litellm_params:
    model: azure/gpt-5.3-codex-2026-02-24
    api_base: https://api-base-swedencentral-0.openai.azure.com/
    litellm_credential_name: default_azure_openai_credential
  model_info:
    id: gpt-5-3-codex-2026-02-24-swedencentral-0
    base_model: azure/gpt-5.3-codex
    label: "GPT-5.3 Codex 2026-02-24"
    forbidden_for_web: true

GPT-5.4 series

GPT-5.4

GPT-5.4

# US Region
- model_name: gpt-5.4-2026-03-05
  litellm_params:
    model: azure/gpt-5.4-2026-03-05
    api_base: https://api-base-eastus2-0.openai.azure.com/
    litellm_credential_name: default_azure_openai_credential
  model_info:
    id: gpt-5-4-2026-03-05-eastus2-0
    base_model: azure/gpt-5.4-2026-03-05
    label: "GPT-5.4"

# EU Region
- model_name: gpt-5.4-2026-03-05
  litellm_params:
    model: azure/gpt-5.4-2026-03-05
    api_base: https://api-base-swedencentral-0.openai.azure.com/
    litellm_credential_name: default_azure_openai_credential
  model_info:
    id: gpt-5-4-2026-03-05-swedencentral-0
    base_model: azure/gpt-5.4-2026-03-05
    label: "GPT-5.4"

o-series

o1

- model_name: o1
  litellm_params:
    model: azure/o1-2024-12-17
    api_base: https://api-base-swedencentral-0.openai.azure.com/
    litellm_credential_name: default_azure_openai_credential
    api_version: 2024-12-01-preview
  model_info:
    id: o1-swedencentral-0-eu
    base_model: azure/eu/o1-2024-12-17
    label: "o1 2024-12-17"
    supports_native_streaming: false

o3

- model_name: o3-2025-04-16
  litellm_params:
    model: azure/o3-2025-04-16
    api_base: https://api-base-swedencentral-0.openai.azure.com/
    litellm_credential_name: default_azure_openai_credential
    api_version: 2024-12-01-preview
  model_info:
    id: o3-2025-04-16-swedencentral-0-eu
    base_model: azure/o3-2025-04-16
    label: "o3 2025-04-16"
    supports_native_streaming: false

o3-Mini

o3-Mini

- model_name: o3-mini
  litellm_params:
    model: azure/o3-mini-2025-01-31
    api_base: https://api-base-swedencentral-0.openai.azure.com/
    litellm_credential_name: default_azure_openai_credential
    api_version: 2024-12-01-preview
  model_info:
    id: o3-mini-swedencentral-0-eu
    base_model: azure/eu/o3-mini-2025-01-31
    label: "o3 Mini 2025-01-31"
    supports_native_streaming: false

o4-Mini

o4-Mini

- model_name: o4-mini-2025-04-16
  litellm_params:
    model: azure/o4-mini-2025-04-16
    api_base: https://api-base-swedencentral-0.openai.azure.com/
    litellm_credential_name: default_azure_openai_credential
    api_version: 2024-12-01-preview
  model_info:
    id: o4-mini-2025-04-16-swedencentral-0-eu
    base_model: azure/o4-mini-2025-04-16
    label: "o4-mini 2025-04-16"
    supports_native_streaming: false

Text-embedding-ada-002

Text-embedding-ada-002

# US Region
- model_name: codemie-text-embedding-ada-002
  litellm_params:
    model: azure/text-embedding-ada-002
    api_base: https://api-base-eastus-0.openai.azure.com/
    litellm_credential_name: default_azure_openai_credential
  model_info:
    id: ada-002-eastus-0
    base_model: azure/text-embedding-ada-002
    label: "Text Embedding Ada"
    mode: embedding

# EU Region
- model_name: codemie-text-embedding-ada-002
  litellm_params:
    model: azure/text-embedding-ada-002
    api_base: https://api-base-swedencentral-0.openai.azure.com/
    litellm_credential_name: default_azure_openai_credential
  model_info:
    id: ada-002-swedencentral-0
    base_model: azure/text-embedding-ada-002
    label: "Text Embedding Ada"
    mode: embedding

Google Vertex AI Provider Examples

Gemini

Gemini Region Configuration

For Gemini models, vertex_project and vertex_location can be set in two ways:

Globally in litellm_settings – applies to all Gemini models at once:

litellm_settings:
  vertex_project: os.environ/VERTEX_PROJECT
  vertex_location: "us-central1"

Per model in litellm_params – overrides the global value for a specific entry:

- model_name: gemini-3-flash
  litellm_params:
    model: vertex_ai/gemini-3-flash-preview
    vertex_location: "global"

The litellm_settings approach is recommended when all Gemini models share the same project and region.

Gemini 3 Flash

Gemini 3 Flash

- model_name: gemini-3-flash
  litellm_params:
    model: vertex_ai/gemini-3-flash-preview
    vertex_location: "global"
  model_info:
    id: gemini-3-flash-preview-global
    base_model: gemini-3-flash-preview
    label: "Gemini 3 Flash"

Gemini 3.1 Pro

Gemini 3.1 Pro

- model_name: gemini-3.1-pro
  litellm_params:
    model: vertex_ai/gemini-3.1-pro-preview
    vertex_location: "global"
  model_info:
    id: gemini-3.1-pro-preview-global
    base_model: gemini-3.1-pro-preview
    label: "Gemini 3.1 Pro"

Claude Sonnet

Claude on Vertex AI: Required Parameters

Claude models on Vertex AI require two parameters specified per model entry:

vertex_ai_project – your GCP project ID (e.g. os.environ/VERTEX_PROJECT)
vertex_ai_location – the region where the model is deployed (e.g. "europe-west1", "us-east5")

Claude Sonnet 4.5

Claude Sonnet 4.5

- model_name: claude-4-5-sonnet-vertex
  litellm_params:
    model: vertex_ai/claude-sonnet-4-5
    vertex_ai_project: os.environ/VERTEX_PROJECT
    vertex_ai_location: "europe-west1"
  model_info:
    id: claude-4-5-sonnet-europe-west1-vertex-ai
    base_model: vertex_ai/claude-sonnet-4-5@20250929
    label: "VertexAI Claude Sonnet 4.5"

- model_name: claude-4-5-sonnet-vertex
  litellm_params:
    model: vertex_ai/claude-sonnet-4-5
    vertex_ai_project: os.environ/VERTEX_PROJECT
    vertex_ai_location: "us-east5"
  model_info:
    id: claude-4-5-sonnet-us-east5-vertex-ai
    base_model: vertex_ai/claude-sonnet-4-5@20250929
    label: "VertexAI Claude Sonnet 4.5"

Embeddings for Text

Embeddings for Text

- model_name: text-embedding-005
  litellm_params:
    model: vertex_ai/text-embedding-005
    project: os.environ/VERTEX_PROJECT
    location: us-central1
  model_info:
    id: gecko
    base_model: text-embedding-005
    label: "Text Embedding Gecko"

GitHub Copilot Provider Examples

GitHub Copilot Authentication

GitHub Copilot requires an OAuth access token mounted as a file. See Authentication Secrets for setup instructions.

GPT-5 series

GitHub Copilot GPT-5

GitHub Copilot GPT-5

- model_name: github-copilot-gpt-5
  litellm_params:
    model: github_copilot/gpt-5
    extra_headers:
      Editor-Version: "vscode/1.85.1"
      Copilot-Integration-Id: "vscode-chat"
  model_info:
    id: gh-copilot-gpt-5
    base_model: github_copilot/gpt-5
    label: "GitHub Copilot GPT-5"

GitHub Copilot GPT-5 Mini

GitHub Copilot GPT-5 Mini

- model_name: github-copilot-gpt-5-mini
  litellm_params:
    model: github_copilot/gpt-5-mini
    extra_headers:
      Editor-Version: "vscode/1.85.1"
      Copilot-Integration-Id: "vscode-chat"
  model_info:
    id: gh-copilot-gpt-5-mini
    base_model: github_copilot/gpt-5-mini
    label: "GitHub Copilot GPT-5 Mini"

GPT-5.1 series

GitHub Copilot GPT-5.1

GitHub Copilot GPT-5.1

- model_name: github-copilot-gpt-5-1
  litellm_params:
    model: github_copilot/gpt-5.1
    extra_headers:
      Editor-Version: "vscode/1.85.1"
      Copilot-Integration-Id: "vscode-chat"
  model_info:
    id: gh-copilot-gpt-5-1
    base_model: github_copilot/gpt-5.1
    label: "GitHub Copilot GPT-5.1"

GitHub Copilot GPT-5.1 Codex Max

GitHub Copilot GPT-5.1 Codex Max

- model_name: github-copilot-gpt-5-1-codex-max
  litellm_params:
    model: github_copilot/gpt-5.1-codex-max
    extra_headers:
      Editor-Version: "vscode/1.85.1"
      Copilot-Integration-Id: "vscode-chat"
  model_info:
    id: gh-copilot-gpt-5-1-codex-max
    base_model: github_copilot/gpt-5.1-codex-max
    label: "GitHub Copilot GPT-5.1 Codex Max"

GPT-5.2 series

GitHub Copilot GPT-5.2

GitHub Copilot GPT-5.2

- model_name: github-copilot-gpt-5-2
  litellm_params:
    model: github_copilot/gpt-5.2
    extra_headers:
      Editor-Version: "vscode/1.85.1"
      Copilot-Integration-Id: "vscode-chat"
  model_info:
    id: gh-copilot-gpt-5-2
    base_model: github_copilot/gpt-5.2
    label: "GitHub Copilot GPT-5.2"

Claude Haiku

GitHub Copilot Claude Haiku

GitHub Copilot Claude Haiku 4.5

- model_name: github-copilot-claude-haiku-4.5
  litellm_params:
    model: github_copilot/claude-haiku-4.5
    extra_headers:
      Editor-Version: "vscode/1.85.1"
      Copilot-Integration-Id: "vscode-chat"
  model_info:
    id: gh-copilot-claude-haiku-4-5
    base_model: github_copilot/claude-haiku-4.5
    label: "GitHub Copilot Claude Haiku 4.5"

Claude Sonnet

GitHub Copilot Claude Sonnet

GitHub Copilot Claude Sonnet 4.5

- model_name: github-copilot-claude-sonnet-4.5
  litellm_params:
    model: github_copilot/claude-sonnet-4.5
    extra_headers:
      Editor-Version: "vscode/1.85.1"
      Copilot-Integration-Id: "vscode-chat"
  model_info:
    id: gh-copilot-claude-sonnet-4-5
    base_model: github_copilot/claude-sonnet-4.5
    label: "GitHub Copilot Claude Sonnet 4.5"

Claude Opus

GitHub Copilot Claude Opus

GitHub Copilot Claude Opus 4.5

- model_name: github-copilot-claude-opus-4-5
  litellm_params:
    model: github_copilot/claude-opus-4.5
    extra_headers:
      Editor-Version: "vscode/1.85.1"
      Copilot-Integration-Id: "vscode-chat"
  model_info:
    id: gh-copilot-claude-opus-4-5
    base_model: github_copilot/claude-opus-4.5
    label: "GitHub Copilot Claude Opus 4.5"

Overview​

LiteLLM Proxy Config Structure​

Model Configuration Structure​

Model Configuration Examples​

Models Reference​

AWS Bedrock Models​

Azure OpenAI Models​

Vertex AI Models​

GitHub Copilot Models​

AWS Bedrock Provider Examples​

Claude Sonnet​

Claude Sonnet 4.5​

Claude Sonnet 4.6​

Claude Haiku​

Claude Haiku 4.5​

Claude Opus​

Claude Opus 4.5​

Claude Opus 4.6​

Amazon Titan​

Azure OpenAI Provider Examples​

GPT-4.1 series​

GPT-4.1​

GPT-4.1 Mini​

GPT-5 series​

GPT-5​

GPT-5 Mini​

GPT-5 Nano​

GPT-5.1 series​

GPT-5.1 Codex​

GPT-5.2 series​

GPT-5.2​

GPT-5.3 series​

GPT-5.3 Codex​

GPT-5.4 series​

GPT-5.4​

o-series​

o1​

o3​

o3-Mini​

o4-Mini​

Text-embedding-ada-002​

Google Vertex AI Provider Examples​

Gemini​

Gemini 3 Flash​

Gemini 3.1 Pro​

Claude Sonnet​

Claude Sonnet 4.5​

Embeddings for Text​

GitHub Copilot Provider Examples​

GPT-5 series​

GitHub Copilot GPT-5​

GitHub Copilot GPT-5 Mini​

GPT-5.1 series​

GitHub Copilot GPT-5.1​

GitHub Copilot GPT-5.1 Codex Max​

GPT-5.2 series​

GitHub Copilot GPT-5.2​

Claude Haiku​

GitHub Copilot Claude Haiku​

Claude Sonnet​

GitHub Copilot Claude Sonnet​

Claude Opus​

GitHub Copilot Claude Opus​

See Also​

Overview

LiteLLM Proxy Config Structure

Model Configuration Structure

Model Configuration Examples

Models Reference

AWS Bedrock Models

Azure OpenAI Models

Vertex AI Models

GitHub Copilot Models

AWS Bedrock Provider Examples

Claude Sonnet

Claude Sonnet 4.5

Claude Sonnet 4.6

Claude Haiku

Claude Haiku 4.5

Claude Opus

Claude Opus 4.5

Claude Opus 4.6

Amazon Titan

Azure OpenAI Provider Examples

GPT-4.1 series

GPT-4.1

GPT-4.1 Mini

GPT-5 series

GPT-5

GPT-5 Mini

GPT-5 Nano

GPT-5.1 series

GPT-5.1 Codex

GPT-5.2 series

GPT-5.2

GPT-5.3 series

GPT-5.3 Codex

GPT-5.4 series

GPT-5.4

o-series

o1

o3

o3-Mini

o4-Mini

Text-embedding-ada-002

Google Vertex AI Provider Examples

Gemini

Gemini 3 Flash

Gemini 3.1 Pro

Claude Sonnet

Claude Sonnet 4.5

Embeddings for Text

GitHub Copilot Provider Examples

GPT-5 series

GitHub Copilot GPT-5

GitHub Copilot GPT-5 Mini

GPT-5.1 series

GitHub Copilot GPT-5.1

GitHub Copilot GPT-5.1 Codex Max

GPT-5.2 series

GitHub Copilot GPT-5.2

Claude Haiku

GitHub Copilot Claude Haiku

Claude Sonnet

GitHub Copilot Claude Sonnet

Claude Opus

GitHub Copilot Claude Opus

See Also