2023-08-24 07:51:02 -07:00
<h1 align="center">
2023-08-24 08:32:50 -07:00
🚅 LiteLLM
</h1>
<p align="center">
2025-12-19 01:30:48 +05:30
<p align="center">Call 100+ LLMs in OpenAI format. [Bedrock, Azure, OpenAI, VertexAI, Anthropic, Groq, etc.]
</p>
2024-05-24 09:19:28 -07:00
<p align="center">
2024-05-24 09:19:48 -07:00
<a href="https://render.com/deploy?repo=https://github.com/BerriAI/litellm" target="_blank" rel="nofollow"><img src="https://render.com/images/deploy-to-render-button.svg" alt="Deploy to Render"></a>
2024-05-24 09:07:32 -07:00
<a href="https://railway.app/template/HLP0Ub?referralCode=jch2ME">
<img src="https://railway.app/button.svg" alt="Deploy on Railway">
</a>
2024-05-24 09:19:28 -07:00
</p>
2023-08-24 08:32:50 -07:00
</p>
2025-12-19 01:30:48 +05:30
<h4 align="center"><a href="https://docs.litellm.ai/docs/simple_proxy" target="_blank">LiteLLM Proxy Server (AI Gateway)</a> | <a href="https://docs.litellm.ai/docs/enterprise#hosted -litellm-proxy" target="_blank"> Hosted Proxy</a> | <a href="https://docs.litellm.ai/docs/enterprise"target="_blank">Enterprise Tier</a></h4>
2023-08-24 07:55:02 -07:00
<h4 align="center">
2023-08-24 08:32:50 -07:00
<a href="https://pypi.org/project/litellm/" target="_blank">
2023-08-24 07:55:02 -07:00
<img src="https://img.shields.io/pypi/v/litellm.svg" alt="PyPI Version">
</a>
2023-08-26 14:04:51 -07:00
<a href="https://www.ycombinator.com/companies/berriai">
<img src="https://img.shields.io/badge/Y%20Combinator-W23-orange?style=flat-square" alt="Y Combinator W23">
</a>
2023-10-17 13:57:41 -07:00
<a href="https://wa.link/huol9n">
<img src="https://img.shields.io/static/v1?label=Chat%20on&message=WhatsApp&color=success&logo=WhatsApp&style=flat-square" alt="Whatsapp">
</a>
<a href="https://discord.gg/wuPM9dRgDw">
<img src="https://img.shields.io/static/v1?label=Chat%20on&message=Discord&color=blue&logo=Discord&style=flat-square" alt="Discord">
</a>
2025-09-10 08:48:37 -07:00
<a href="https://www.litellm.ai/support">
2025-06-30 10:23:53 -07:00
<img src="https://img.shields.io/static/v1?label=Chat%20on&message=Slack&color=black&logo=Slack&style=flat-square" alt="Slack">
</a>
2023-08-24 07:55:02 -07:00
</h4>
2023-09-27 08:03:39 -07:00
2025-12-19 01:35:48 +05:30
<img width="2688" height="1600" alt="Group 7154 (1)" src="https://github.com/user-attachments/assets/c5ee0412-6fb5-4fb6-ab5b-bafae4209ca6" />
2025-12-19 01:35:08 +05:30
2023-09-02 17:42:59 -07:00
2025-12-19 01:30:48 +05:30
## Use LiteLLM for
2024-03-26 10:34:16 -07:00
2025-12-19 01:30:48 +05:30
<details open>
<summary><b>LLMs</b> - Call 100+ LLMs (Python SDK + AI Gateway)</summary>
2024-02-13 17:19:41 -08:00
2025-12-19 01:30:48 +05:30
[**All Supported Endpoints** ](https://docs.litellm.ai/docs/supported_endpoints ) - `/chat/completions` , `/responses` , `/embeddings` , `/images` , `/audio` , `/batches` , `/rerank` , `/a2a` , `/messages` and more.
2024-02-27 22:07:13 -08:00
2025-12-19 01:30:48 +05:30
### Python SDK
2023-09-27 08:04:37 -07:00
2023-12-27 17:30:37 +05:30
``` shell
2023-07-31 07:58:10 -07:00
pip install litellm
```
2023-07-26 17:44:43 -07:00
``` python
2023-07-26 17:46:23 -07:00
from litellm import completion
2023-08-29 10:39:33 -07:00
import os
2023-09-19 17:10:55 -07:00
2024-02-27 22:07:13 -08:00
os . environ [ " OPENAI_API_KEY " ] = " your-openai-key "
2025-02-17 08:34:15 -07:00
os . environ [ " ANTHROPIC_API_KEY " ] = " your-anthropic-key "
2023-07-26 17:46:23 -07:00
2025-12-19 01:30:48 +05:30
# OpenAI
response = completion ( model = " openai/gpt-4o " , messages = [ { " role " : " user " , " content " : " Hello! " } ] )
2023-07-26 17:44:34 -07:00
2025-12-19 01:30:48 +05:30
# Anthropic
response = completion ( model = " anthropic/claude-sonnet-4-20250514 " , messages = [ { " role " : " user " , " content " : " Hello! " } ] )
2023-07-26 17:44:34 -07:00
```
2023-08-01 08:18:44 -07:00
2025-12-19 01:30:48 +05:30
### AI Gateway (Proxy Server)
2024-12-16 08:36:57 -08:00
2025-12-19 01:30:48 +05:30
[**Getting Started - E2E Tutorial** ](https://docs.litellm.ai/docs/proxy/docker_quick_start ) - Setup virtual keys, make your first request
2025-12-17 18:00:15 -03:00
2025-12-19 01:30:48 +05:30
``` shell
pip install 'litellm[proxy]'
litellm --model gpt-4o
```
2025-12-17 18:00:15 -03:00
``` python
2025-12-19 01:30:48 +05:30
import openai
2025-12-17 18:00:15 -03:00
2025-12-19 01:30:48 +05:30
client = openai . OpenAI ( api_key = " anything " , base_url = " http://0.0.0.0:4000 " )
response = client . chat . completions . create (
model = " gpt-4o " ,
messages = [ { " role " : " user " , " content " : " Hello! " } ]
2025-12-17 18:00:15 -03:00
)
```
2025-12-19 01:30:48 +05:30
[**Docs: LLM Providers** ](https://docs.litellm.ai/docs/providers )
2023-12-11 08:30:55 -08:00
2025-12-19 01:30:48 +05:30
</details>
2023-12-11 08:30:55 -08:00
2025-12-19 01:30:48 +05:30
<details>
<summary><b>Agents</b> - Invoke A2A Agents (Python SDK + AI Gateway)</summary>
2023-12-11 08:30:55 -08:00
2025-12-19 01:30:48 +05:30
[**Supported Providers** ](https://docs.litellm.ai/docs/a2a#add-a2a-agents ) - LangGraph, Vertex AI Agent Engine, Azure AI Foundry, Bedrock AgentCore, Pydantic AI
2024-02-27 22:07:13 -08:00
2025-12-19 01:30:48 +05:30
### Python SDK - A2A Protocol
2024-02-27 22:07:13 -08:00
2023-08-05 15:12:34 -07:00
``` python
2025-12-19 01:30:48 +05:30
from litellm . a2a_protocol import A2AClient
from a2a . types import SendMessageRequest , MessageSendParams
from uuid import uuid4
client = A2AClient ( base_url = " http://localhost:10001 " )
request = SendMessageRequest (
id = str ( uuid4 ( ) ) ,
params = MessageSendParams (
message = {
" role " : " user " ,
" parts " : [ { " kind " : " text " , " text " : " Hello! " } ] ,
" messageId " : uuid4 ( ) . hex ,
2024-12-16 08:36:57 -08:00
}
2025-12-19 01:30:48 +05:30
)
)
response = await client . send_message ( request )
2023-08-05 15:10:49 -07:00
```
2023-10-28 12:29:44 -07:00
2025-12-19 01:30:48 +05:30
### AI Gateway (Proxy Server)
2023-10-28 12:29:44 -07:00
2025-12-19 01:30:48 +05:30
**Step 1. ** [Add your Agent to the AI Gateway ](https://docs.litellm.ai/docs/a2a#adding-your-agent )
2023-10-28 12:29:44 -07:00
2025-12-19 01:30:48 +05:30
**Step 2. ** Call Agent via A2A SDK
2023-10-28 12:29:44 -07:00
2025-12-19 01:30:48 +05:30
``` python
from a2a . client import A2ACardResolver , A2AClient
from a2a . types import MessageSendParams , SendMessageRequest
from uuid import uuid4
import httpx
base_url = " http://localhost:4000/a2a/my-agent " # LiteLLM proxy + agent name
headers = { " Authorization " : " Bearer sk-1234 " } # LiteLLM Virtual Key
async with httpx . AsyncClient ( headers = headers ) as httpx_client :
resolver = A2ACardResolver ( httpx_client = httpx_client , base_url = base_url )
agent_card = await resolver . get_agent_card ( )
client = A2AClient ( httpx_client = httpx_client , agent_card = agent_card )
request = SendMessageRequest (
id = str ( uuid4 ( ) ) ,
params = MessageSendParams (
message = {
" role " : " user " ,
" parts " : [ { " kind " : " text " , " text " : " Hello! " } ] ,
" messageId " : uuid4 ( ) . hex ,
}
)
)
response = await client . send_message ( request )
2023-10-28 12:25:58 -07:00
```
2025-12-19 01:30:48 +05:30
[**Docs: A2A Agent Gateway** ](https://docs.litellm.ai/docs/a2a )
2024-04-20 09:48:49 -07:00
2025-12-19 01:30:48 +05:30
</details>
2023-12-26 12:27:37 +05:30
2025-12-19 01:30:48 +05:30
<details>
<summary><b>MCP Tools</b> - Connect MCP servers to any LLM (Python SDK + AI Gateway)</summary>
2024-02-27 22:07:13 -08:00
2025-12-19 01:30:48 +05:30
### Python SDK - MCP Bridge
2023-12-28 11:58:33 +05:30
2025-12-19 01:30:48 +05:30
``` python
from mcp import ClientSession , StdioServerParameters
from mcp . client . stdio import stdio_client
from litellm import experimental_mcp_client
import litellm
2024-06-01 16:53:06 -07:00
2025-12-19 01:30:48 +05:30
server_params = StdioServerParameters ( command = " python " , args = [ " mcp_server.py " ] )
2023-12-27 17:30:37 +05:30
2025-12-19 01:30:48 +05:30
async with stdio_client ( server_params ) as ( read , write ) :
async with ClientSession ( read , write ) as session :
await session . initialize ( )
2023-12-27 17:30:37 +05:30
2025-12-19 01:30:48 +05:30
# Load MCP tools in OpenAI format
tools = await experimental_mcp_client . load_mcp_tools ( session = session , format = " openai " )
2024-02-27 22:07:13 -08:00
2025-12-19 01:30:48 +05:30
# Use with any LiteLLM model
response = await litellm . acompletion (
model = " gpt-4o " ,
messages = [ { " role " : " user " , " content " : " What ' s 3 + 5? " } ] ,
tools = tools
)
2023-12-26 12:27:37 +05:30
```
2025-12-19 01:30:48 +05:30
### AI Gateway - MCP Gateway
2024-07-25 20:10:02 -07:00
2025-12-19 01:30:48 +05:30
**Step 1. ** [Add your MCP Server to the AI Gateway ](https://docs.litellm.ai/docs/mcp#adding-your-mcp )
2024-07-25 20:10:02 -07:00
2025-12-19 01:30:48 +05:30
**Step 2. ** Call MCP tools via `/chat/completions`
2024-06-01 16:53:06 -07:00
``` bash
2025-12-19 01:30:48 +05:30
curl -X POST 'http://0.0.0.0:4000/v1/chat/completions' \
-H 'Authorization: Bearer sk-1234' \
-H 'Content-Type: application/json' \
-d '{
"model": "gpt-4o",
"messages": [{"role": "user", "content": "Summarize the latest open PR"}],
"tools": [{
"type": "mcp",
"server_url": "litellm_proxy/mcp/github",
"server_label": "github_mcp",
"require_approval": "never"
}]
}'
2024-06-01 16:53:06 -07:00
```
2025-12-19 01:30:48 +05:30
### Use with Cursor IDE
2024-06-01 16:53:06 -07:00
2025-12-19 01:30:48 +05:30
``` json
{
"mcpServers" : {
"LiteLLM" : {
"url" : "http://localhost:4000/mcp" ,
"headers" : {
"x-litellm-api-key" : "Bearer sk-1234"
}
}
}
}
```
2024-02-09 09:30:16 -08:00
2025-12-19 01:30:48 +05:30
[**Docs: MCP Gateway** ](https://docs.litellm.ai/docs/mcp )
</details>
---
## How to use LiteLLM
You can use LiteLLM through either the Proxy Server or Python SDK. Both gives you a unified interface to access multiple LLMs (100+ LLMs). Choose the option that best fits your needs:
<table style={{width: '100%', tableLayout: 'fixed'}}>
<thead>
<tr>
<th style={{width: '14%'}}></th>
<th style={{width: '43%'}}><strong><a href="https://docs.litellm.ai/docs/simple_proxy">LiteLLM AI Gateway</a></strong></th>
<th style={{width: '43%'}}><strong><a href="https://docs.litellm.ai/docs/">LiteLLM Python SDK</a></strong></th>
</tr>
</thead>
<tbody>
<tr>
<td style={{width: '14%'}}><strong>Use Case</strong></td>
<td style={{width: '43%'}}>Central service (LLM Gateway) to access multiple LLMs</td>
<td style={{width: '43%'}}>Use LiteLLM directly in your Python code</td>
</tr>
<tr>
<td style={{width: '14%'}}><strong>Who Uses It?</strong></td>
<td style={{width: '43%'}}>Gen AI Enablement / ML Platform Teams</td>
<td style={{width: '43%'}}>Developers building LLM projects</td>
</tr>
<tr>
<td style={{width: '14%'}}><strong>Key Features</strong></td>
<td style={{width: '43%'}}>Centralized API gateway with authentication and authorization, multi-tenant cost tracking and spend management per project/user, per-project customization (logging, guardrails, caching), virtual keys for secure access control, admin dashboard UI for monitoring and management</td>
<td style={{width: '43%'}}>Direct Python library integration in your codebase, Router with retry/fallback logic across multiple deployments (e.g. Azure/OpenAI) - <a href="https://docs.litellm.ai/docs/routing">Router</a>, application-level load balancing and cost tracking, exception handling with OpenAI-compatible errors, observability callbacks (Lunary, MLflow, Langfuse, etc.)</td>
</tr>
</tbody>
</table>
2023-12-27 11:53:35 +05:30
2025-12-19 01:30:48 +05:30
LiteLLM Performance: **8ms P95 latency ** at 1k RPS (See benchmarks [here ](https://docs.litellm.ai/docs/benchmarks ))
2024-02-27 22:07:13 -08:00
2025-12-19 01:30:48 +05:30
[**Jump to LiteLLM Proxy (LLM Gateway) Docs** ](https://docs.litellm.ai/docs/simple_proxy ) <br>
[**Jump to Supported LLM Providers** ](https://docs.litellm.ai/docs/providers )
2023-12-27 11:53:35 +05:30
2025-12-19 01:30:48 +05:30
**Stable Release: ** Use docker images with the `-stable` tag. These have undergone 12 hour load tests, before being published. [More information about the release cycle here ](https://docs.litellm.ai/docs/proxy/release_cycle )
2024-02-27 22:07:13 -08:00
2025-12-19 01:30:48 +05:30
Support for more providers. Missing a provider or LLM Platform, raise a [feature request ](https://github.com/BerriAI/litellm/issues/new?assignees=&labels=enhancement&projects=&template=feature_request.yml&title=%5BFeature%5D%3A+ ).
2023-12-27 12:01:05 +05:30
2025-10-30 18:25:51 -07:00
## Supported Providers ([Website Supported Models](https://models.litellm.ai/) | [Docs](https://docs.litellm.ai/docs/providers))
2025-10-30 17:05:32 -07:00
| Provider | `/chat/completions` | `/messages` | `/responses` | `/embeddings` | `/image/generations` | `/audio/transcriptions` | `/audio/speech` | `/moderations` | `/batches` | `/rerank` |
|-------------------------------------------------------------------------------------|---------------------|-------------|--------------|---------------|----------------------|-------------------------|-----------------|----------------|-----------|-----------|
| [AI/ML API (`aiml`) ](https://docs.litellm.ai/docs/providers/aiml ) | ✅ | ✅ | ✅ | ✅ | ✅ | | | | | |
| [AI21 (`ai21`) ](https://docs.litellm.ai/docs/providers/ai21 ) | ✅ | ✅ | ✅ | | | | | | | |
| [AI21 Chat (`ai21_chat`) ](https://docs.litellm.ai/docs/providers/ai21 ) | ✅ | ✅ | ✅ | | | | | | | |
| [Aleph Alpha ](https://docs.litellm.ai/docs/providers/aleph_alpha ) | ✅ | ✅ | ✅ | | | | | | | |
2025-12-19 05:37:34 -08:00
| [Amazon Nova ](https://docs.litellm.ai/docs/providers/amazon_nova ) | ✅ | ✅ | ✅ | | | | | | | |
2025-10-30 17:05:32 -07:00
| [Anthropic (`anthropic`) ](https://docs.litellm.ai/docs/providers/anthropic ) | ✅ | ✅ | ✅ | | | | | | ✅ | |
| [Anthropic Text (`anthropic_text`) ](https://docs.litellm.ai/docs/providers/anthropic ) | ✅ | ✅ | ✅ | | | | | | ✅ | |
| [Anyscale ](https://docs.litellm.ai/docs/providers/anyscale ) | ✅ | ✅ | ✅ | | | | | | | |
| [AssemblyAI (`assemblyai`) ](https://docs.litellm.ai/docs/pass_through/assembly_ai ) | ✅ | ✅ | ✅ | | | ✅ | | | | |
| [Auto Router (`auto_router`) ](https://docs.litellm.ai/docs/proxy/auto_routing ) | ✅ | ✅ | ✅ | | | | | | | |
| [AWS - Bedrock (`bedrock`) ](https://docs.litellm.ai/docs/providers/bedrock ) | ✅ | ✅ | ✅ | ✅ | | | | | | ✅ |
| [AWS - Sagemaker (`sagemaker`) ](https://docs.litellm.ai/docs/providers/aws_sagemaker ) | ✅ | ✅ | ✅ | ✅ | | | | | | |
| [Azure (`azure`) ](https://docs.litellm.ai/docs/providers/azure ) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | |
| [Azure AI (`azure_ai`) ](https://docs.litellm.ai/docs/providers/azure_ai ) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | |
| [Azure Text (`azure_text`) ](https://docs.litellm.ai/docs/providers/azure ) | ✅ | ✅ | ✅ | | | ✅ | ✅ | ✅ | ✅ | |
| [Baseten (`baseten`) ](https://docs.litellm.ai/docs/providers/baseten ) | ✅ | ✅ | ✅ | | | | | | | |
| [Bytez (`bytez`) ](https://docs.litellm.ai/docs/providers/bytez ) | ✅ | ✅ | ✅ | | | | | | | |
| [Cerebras (`cerebras`) ](https://docs.litellm.ai/docs/providers/cerebras ) | ✅ | ✅ | ✅ | | | | | | | |
| [Clarifai (`clarifai`) ](https://docs.litellm.ai/docs/providers/clarifai ) | ✅ | ✅ | ✅ | | | | | | | |
| [Cloudflare AI Workers (`cloudflare`) ](https://docs.litellm.ai/docs/providers/cloudflare_workers ) | ✅ | ✅ | ✅ | | | | | | | |
| [Codestral (`codestral`) ](https://docs.litellm.ai/docs/providers/codestral ) | ✅ | ✅ | ✅ | | | | | | | |
| [Cohere (`cohere`) ](https://docs.litellm.ai/docs/providers/cohere ) | ✅ | ✅ | ✅ | ✅ | | | | | | ✅ |
| [Cohere Chat (`cohere_chat`) ](https://docs.litellm.ai/docs/providers/cohere ) | ✅ | ✅ | ✅ | | | | | | | |
| [CometAPI (`cometapi`) ](https://docs.litellm.ai/docs/providers/cometapi ) | ✅ | ✅ | ✅ | ✅ | | | | | | |
| [CompactifAI (`compactifai`) ](https://docs.litellm.ai/docs/providers/compactifai ) | ✅ | ✅ | ✅ | | | | | | | |
| [Custom (`custom`) ](https://docs.litellm.ai/docs/providers/custom_llm_server ) | ✅ | ✅ | ✅ | | | | | | | |
| [Custom OpenAI (`custom_openai`) ](https://docs.litellm.ai/docs/providers/openai_compatible ) | ✅ | ✅ | ✅ | | | ✅ | ✅ | ✅ | ✅ | |
| [Dashscope (`dashscope`) ](https://docs.litellm.ai/docs/providers/dashscope ) | ✅ | ✅ | ✅ | | | | | | | |
| [Databricks (`databricks`) ](https://docs.litellm.ai/docs/providers/databricks ) | ✅ | ✅ | ✅ | | | | | | | |
| [DataRobot (`datarobot`) ](https://docs.litellm.ai/docs/providers/datarobot ) | ✅ | ✅ | ✅ | | | | | | | |
| [Deepgram (`deepgram`) ](https://docs.litellm.ai/docs/providers/deepgram ) | ✅ | ✅ | ✅ | | | ✅ | | | | |
| [DeepInfra (`deepinfra`) ](https://docs.litellm.ai/docs/providers/deepinfra ) | ✅ | ✅ | ✅ | | | | | | | |
| [Deepseek (`deepseek`) ](https://docs.litellm.ai/docs/providers/deepseek ) | ✅ | ✅ | ✅ | | | | | | | |
| [ElevenLabs (`elevenlabs`) ](https://docs.litellm.ai/docs/providers/elevenlabs ) | ✅ | ✅ | ✅ | | | | ✅ | | | |
| [Empower (`empower`) ](https://docs.litellm.ai/docs/providers/empower ) | ✅ | ✅ | ✅ | | | | | | | |
| [Fal AI (`fal_ai`) ](https://docs.litellm.ai/docs/providers/fal_ai ) | ✅ | ✅ | ✅ | | ✅ | | | | | |
| [Featherless AI (`featherless_ai`) ](https://docs.litellm.ai/docs/providers/featherless_ai ) | ✅ | ✅ | ✅ | | | | | | | |
| [Fireworks AI (`fireworks_ai`) ](https://docs.litellm.ai/docs/providers/fireworks_ai ) | ✅ | ✅ | ✅ | | | | | | | |
| [FriendliAI (`friendliai`) ](https://docs.litellm.ai/docs/providers/friendliai ) | ✅ | ✅ | ✅ | | | | | | | |
| [Galadriel (`galadriel`) ](https://docs.litellm.ai/docs/providers/galadriel ) | ✅ | ✅ | ✅ | | | | | | | |
2025-12-04 13:22:08 +08:00
| [GitHub Copilot (`github_copilot`) ](https://docs.litellm.ai/docs/providers/github_copilot ) | ✅ | ✅ | ✅ | ✅ | | | | | | |
2025-10-30 17:05:32 -07:00
| [GitHub Models (`github`) ](https://docs.litellm.ai/docs/providers/github ) | ✅ | ✅ | ✅ | | | | | | | |
| [Google - PaLM ](https://docs.litellm.ai/docs/providers/palm ) | ✅ | ✅ | ✅ | | | | | | | |
| [Google - Vertex AI (`vertex_ai`) ](https://docs.litellm.ai/docs/providers/vertex ) | ✅ | ✅ | ✅ | ✅ | ✅ | | | | | |
| [Google AI Studio - Gemini (`gemini`) ](https://docs.litellm.ai/docs/providers/gemini ) | ✅ | ✅ | ✅ | | | | | | | |
| [GradientAI (`gradient_ai`) ](https://docs.litellm.ai/docs/providers/gradient_ai ) | ✅ | ✅ | ✅ | | | | | | | |
| [Groq AI (`groq`) ](https://docs.litellm.ai/docs/providers/groq ) | ✅ | ✅ | ✅ | | | | | | | |
| [Heroku (`heroku`) ](https://docs.litellm.ai/docs/providers/heroku ) | ✅ | ✅ | ✅ | | | | | | | |
| [Hosted VLLM (`hosted_vllm`) ](https://docs.litellm.ai/docs/providers/vllm ) | ✅ | ✅ | ✅ | | | | | | | |
| [Huggingface (`huggingface`) ](https://docs.litellm.ai/docs/providers/huggingface ) | ✅ | ✅ | ✅ | ✅ | | | | | | ✅ |
| [Hyperbolic (`hyperbolic`) ](https://docs.litellm.ai/docs/providers/hyperbolic ) | ✅ | ✅ | ✅ | | | | | | | |
| [IBM - Watsonx.ai (`watsonx`) ](https://docs.litellm.ai/docs/providers/watsonx ) | ✅ | ✅ | ✅ | ✅ | | | | | | |
| [Infinity (`infinity`) ](https://docs.litellm.ai/docs/providers/infinity ) | | | | ✅ | | | | | | |
| [Jina AI (`jina_ai`) ](https://docs.litellm.ai/docs/providers/jina_ai ) | | | | ✅ | | | | | | |
| [Lambda AI (`lambda_ai`) ](https://docs.litellm.ai/docs/providers/lambda_ai ) | ✅ | ✅ | ✅ | | | | | | | |
| [Lemonade (`lemonade`) ](https://docs.litellm.ai/docs/providers/lemonade ) | ✅ | ✅ | ✅ | | | | | | | |
| [LiteLLM Proxy (`litellm_proxy`) ](https://docs.litellm.ai/docs/providers/litellm_proxy ) | ✅ | ✅ | ✅ | ✅ | ✅ | | | | | |
| [Llamafile (`llamafile`) ](https://docs.litellm.ai/docs/providers/llamafile ) | ✅ | ✅ | ✅ | | | | | | | |
| [LM Studio (`lm_studio`) ](https://docs.litellm.ai/docs/providers/lm_studio ) | ✅ | ✅ | ✅ | | | | | | | |
| [Maritalk (`maritalk`) ](https://docs.litellm.ai/docs/providers/maritalk ) | ✅ | ✅ | ✅ | | | | | | | |
| [Meta - Llama API (`meta_llama`) ](https://docs.litellm.ai/docs/providers/meta_llama ) | ✅ | ✅ | ✅ | | | | | | | |
| [Mistral AI API (`mistral`) ](https://docs.litellm.ai/docs/providers/mistral ) | ✅ | ✅ | ✅ | ✅ | | | | | | |
| [Moonshot (`moonshot`) ](https://docs.litellm.ai/docs/providers/moonshot ) | ✅ | ✅ | ✅ | | | | | | | |
| [Morph (`morph`) ](https://docs.litellm.ai/docs/providers/morph ) | ✅ | ✅ | ✅ | | | | | | | |
| [Nebius AI Studio (`nebius`) ](https://docs.litellm.ai/docs/providers/nebius ) | ✅ | ✅ | ✅ | ✅ | | | | | | |
| [NLP Cloud (`nlp_cloud`) ](https://docs.litellm.ai/docs/providers/nlp_cloud ) | ✅ | ✅ | ✅ | | | | | | | |
| [Novita AI (`novita`) ](https://novita.ai/models/llm?utm_source=github_litellm&utm_medium=github_readme&utm_campaign=github_link ) | ✅ | ✅ | ✅ | | | | | | | |
| [Nscale (`nscale`) ](https://docs.litellm.ai/docs/providers/nscale ) | ✅ | ✅ | ✅ | | | | | | | |
| [Nvidia NIM (`nvidia_nim`) ](https://docs.litellm.ai/docs/providers/nvidia_nim ) | ✅ | ✅ | ✅ | | | | | | | |
| [OCI (`oci`) ](https://docs.litellm.ai/docs/providers/oci ) | ✅ | ✅ | ✅ | | | | | | | |
| [Ollama (`ollama`) ](https://docs.litellm.ai/docs/providers/ollama ) | ✅ | ✅ | ✅ | ✅ | | | | | | |
| [Ollama Chat (`ollama_chat`) ](https://docs.litellm.ai/docs/providers/ollama ) | ✅ | ✅ | ✅ | | | | | | | |
| [Oobabooga (`oobabooga`) ](https://docs.litellm.ai/docs/providers/openai_compatible ) | ✅ | ✅ | ✅ | | | ✅ | ✅ | ✅ | ✅ | |
| [OpenAI (`openai`) ](https://docs.litellm.ai/docs/providers/openai ) | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | |
| [OpenAI-like (`openai_like`) ](https://docs.litellm.ai/docs/providers/openai_compatible ) | | | | ✅ | | | | | | |
| [OpenRouter (`openrouter`) ](https://docs.litellm.ai/docs/providers/openrouter ) | ✅ | ✅ | ✅ | | | | | | | |
| [OVHCloud AI Endpoints (`ovhcloud`) ](https://docs.litellm.ai/docs/providers/ovhcloud ) | ✅ | ✅ | ✅ | | | | | | | |
| [Perplexity AI (`perplexity`) ](https://docs.litellm.ai/docs/providers/perplexity ) | ✅ | ✅ | ✅ | | | | | | | |
| [Petals (`petals`) ](https://docs.litellm.ai/docs/providers/petals ) | ✅ | ✅ | ✅ | | | | | | | |
| [Predibase (`predibase`) ](https://docs.litellm.ai/docs/providers/predibase ) | ✅ | ✅ | ✅ | | | | | | | |
| [Recraft (`recraft`) ](https://docs.litellm.ai/docs/providers/recraft ) | | | | | ✅ | | | | | |
| [Replicate (`replicate`) ](https://docs.litellm.ai/docs/providers/replicate ) | ✅ | ✅ | ✅ | | | | | | | |
| [Sagemaker Chat (`sagemaker_chat`) ](https://docs.litellm.ai/docs/providers/aws_sagemaker ) | ✅ | ✅ | ✅ | | | | | | | |
| [Sambanova (`sambanova`) ](https://docs.litellm.ai/docs/providers/sambanova ) | ✅ | ✅ | ✅ | | | | | | | |
| [Snowflake (`snowflake`) ](https://docs.litellm.ai/docs/providers/snowflake ) | ✅ | ✅ | ✅ | | | | | | | |
| [Text Completion Codestral (`text-completion-codestral`) ](https://docs.litellm.ai/docs/providers/codestral ) | ✅ | ✅ | ✅ | | | | | | | |
| [Text Completion OpenAI (`text-completion-openai`) ](https://docs.litellm.ai/docs/providers/text_completion_openai ) | ✅ | ✅ | ✅ | | | ✅ | ✅ | ✅ | ✅ | |
| [Together AI (`together_ai`) ](https://docs.litellm.ai/docs/providers/togetherai ) | ✅ | ✅ | ✅ | | | | | | | |
| [Topaz (`topaz`) ](https://docs.litellm.ai/docs/providers/topaz ) | ✅ | ✅ | ✅ | | | | | | | |
| [Triton (`triton`) ](https://docs.litellm.ai/docs/providers/triton-inference-server ) | ✅ | ✅ | ✅ | | | | | | | |
| [V0 (`v0`) ](https://docs.litellm.ai/docs/providers/v0 ) | ✅ | ✅ | ✅ | | | | | | | |
| [Vercel AI Gateway (`vercel_ai_gateway`) ](https://docs.litellm.ai/docs/providers/vercel_ai_gateway ) | ✅ | ✅ | ✅ | | | | | | | |
| [VLLM (`vllm`) ](https://docs.litellm.ai/docs/providers/vllm ) | ✅ | ✅ | ✅ | | | | | | | |
| [Volcengine (`volcengine`) ](https://docs.litellm.ai/docs/providers/volcano ) | ✅ | ✅ | ✅ | | | | | | | |
| [Voyage AI (`voyage`) ](https://docs.litellm.ai/docs/providers/voyage ) | | | | ✅ | | | | | | |
| [WandB Inference (`wandb`) ](https://docs.litellm.ai/docs/providers/wandb_inference ) | ✅ | ✅ | ✅ | | | | | | | |
| [Watsonx Text (`watsonx_text`) ](https://docs.litellm.ai/docs/providers/watsonx ) | ✅ | ✅ | ✅ | | | | | | | |
| [xAI (`xai`) ](https://docs.litellm.ai/docs/providers/xai ) | ✅ | ✅ | ✅ | | | | | | | |
| [Xinference (`xinference`) ](https://docs.litellm.ai/docs/providers/xinference ) | | | | ✅ | | | | | | |
2025-05-27 20:05:22 +02:00
2023-09-30 09:48:45 -07:00
[**Read the Docs** ](https://docs.litellm.ai/docs/ )
2023-10-17 13:38:55 -07:00
2025-09-25 00:57:21 +08:00
## Run in Developer mode
### Services
1. Setup .env file in root
2. Run dependant services `docker-compose up db prometheus`
2025-06-06 15:19:28 -06:00
2025-09-25 00:57:21 +08:00
### Backend
1. (In root) create virtual environment `python -m venv .venv`
2. Activate virtual environment `source .venv/bin/activate`
3. Install dependencies `pip install -e ".[all]"`
4. Start proxy backend `python litellm/proxy_cli.py`
2025-06-06 15:19:28 -06:00
2025-09-25 00:57:21 +08:00
### Frontend
1. Navigate to `ui/litellm-dashboard`
2. Install dependencies `npm install`
3. Run `npm run dev` to start the dashboard
2024-11-13 20:52:17 -08:00
2024-03-02 17:18:48 -08:00
# Enterprise
For companies that need better security, user management and professional support
[Talk to founders ](https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat )
2025-08-10 04:26:33 +05:00
This covers:
2024-03-02 17:18:48 -08:00
- ✅ **Features under the [LiteLLM Commercial License](https://docs.litellm.ai/docs/proxy/enterprise): **
- ✅ **Feature Prioritization **
- ✅ **Custom Integrations **
- ✅ **Professional Support - Dedicated discord + slack **
- ✅ **Custom SLAs **
- ✅ **Secure access with Single Sign-On **
2025-06-06 15:19:28 -06:00
# Contributing
We welcome contributions to LiteLLM! Whether you're fixing bugs, adding features, or improving documentation, we appreciate your help.
## Quick Start for Contributors
2025-08-14 17:15:29 -07:00
This requires poetry to be installed.
2025-06-06 15:19:28 -06:00
``` bash
git clone https://github.com/BerriAI/litellm.git
cd litellm
make install-dev # Install development dependencies
make format # Format your code
make lint # Run all linting checks
make test-unit # Run unit tests
2025-08-14 17:15:29 -07:00
make format-check # Check formatting only
2025-06-06 15:19:28 -06:00
```
For detailed contributing guidelines, see [CONTRIBUTING.md ](CONTRIBUTING.md ).
## Code Quality / Linting
2024-12-10 15:44:59 -08:00
LiteLLM follows the [Google Python Style Guide ](https://google.github.io/styleguide/pyguide.html ).
2025-06-06 15:19:28 -06:00
Our automated checks include:
- **Black** for code formatting
- **Ruff** for linting and code quality
- **MyPy** for type checking
- **Circular import detection**
- **Import safety checks**
2024-12-10 15:44:59 -08:00
2025-06-06 15:19:28 -06:00
All these checks must pass before your PR can be merged.
2024-12-10 15:44:59 -08:00
2023-08-28 08:59:34 -07:00
# Support / talk with founders
2024-02-27 22:07:13 -08:00
2023-08-24 08:43:18 -07:00
- [Schedule Demo 👋 ](https://calendly.com/d/4mp-gd3-k5k/berriai-1-1-onboarding-litellm-hosted-version )
2023-08-09 06:09:28 -07:00
- [Community Discord 💭 ](https://discord.gg/wuPM9dRgDw )
2025-09-10 08:48:37 -07:00
- [Community Slack 💭 ](https://www.litellm.ai/support )
2023-08-09 06:09:28 -07:00
- Our numbers 📞 +1 (770) 8783-106 /
- Our emails ✉️ ishaan@berri .ai / krrish@berri .ai
2023-07-28 09:11:27 -07:00
2024-02-27 22:07:13 -08:00
# Why did we build this
2023-10-17 00:10:46 +05:30
- **Need for simplicity**: Our code started to get extremely complicated managing & translating calls between Azure, OpenAI and Cohere.
2023-08-28 08:50:12 -07:00
2023-08-28 08:59:34 -07:00
# Contributors
2023-08-28 08:50:12 -07:00
<!-- ALL-CONTRIBUTORS-LIST:START - Do not remove or modify this section -->
<!-- prettier-ignore-start -->
<!-- markdownlint-disable -->
<!-- markdownlint-restore -->
<!-- prettier-ignore-end -->
<!-- ALL-CONTRIBUTORS-LIST:END -->
2023-08-28 08:56:19 -07:00
<a href="https://github.com/BerriAI/litellm/graphs/contributors">
<img src="https://contrib.rocks/image?repo=BerriAI/litellm" />
</a>
2025-02-08 19:29:44 -05:00
