🕷️ Crawler Inspector

URL Lookup

Direct Parameter Lookup

Raw Queries and Responses

1. Shard Calculation

Query:

Response:

Calculated Shard: 11 (from laksa189)

2. Crawled Status Check

Query:

curl -X POST \
  'http://laksa011.int.ahrefs:8124/' \
  -H 'Content-Type: text/plain' \
  -H 'X-ClickHouse-Database: crawler3' \
  -H 'Authorization: Basic YXBpOg==' \
  -d 'SELECT getAhrefsURLFromUnparsed(src_unparsed) AS found_url, ifNull(toUnixTimestamp(download_stamp), 0) AS crawl_time, ifNull(toUnixTimestamp(props_url_first_seen), 0) AS first_indexed_time, download_http_code AS http_code, src_unparsed AS src_unparsed, src_root_hash AS src_root_hash, history_drop_reason AS history_drop_reason, meta_title AS meta_title, meta_descriptions AS meta_descriptions, attrs_boilerpipe_text AS attrs_boilerpipe_text, attrs_markdown AS attrs_markdown, attrs_readable_markdown AS attrs_readable_markdown, meta_canonical AS meta_canonical FROM crawler3.page_info_local FINAL PREWHERE (src_root_hash, src_unparsed) IN ((getAhrefsRootHashFromUnparsed(getAhrefsUnparsedNoserviceFromURL(\'https://docs.litellm.ai/docs/\')), getAhrefsUnparsedNoserviceFromURL(\'https://docs.litellm.ai/docs/\'))) FORMAT JSONEachRow'

Response:

{"found_url":"https:\/\/docs.litellm.ai\/docs\/","crawl_time":1775480720,"first_indexed_time":1692060293,"http_code":200,"src_unparsed":"ai,litellm!docs,\/docs\/ s443","src_root_hash":"4050687676692930411","history_drop_reason":null,"meta_title":"Getting Started | liteLLM","meta_descriptions":["LiteLLM is an open-source library that gives you a single, unified interface to call 100+ LLMs — OpenAI, Anthropic, Vertex AI, Bedrock, and more — using the OpenAI format."],"attrs_boilerpipe_text":"LiteLLM\nis an open-source library that gives you a single, unified interface to call 100+ LLMs — OpenAI, Anthropic, Vertex AI, Bedrock, and more — using the OpenAI format.\nCall any provider using the same\ncompletion()\ninterface — no re-learning the API for each one\nConsistent output format regardless of which provider or model you use\nBuilt-in retry \/ fallback logic across multiple deployments via the\nRouter\nSelf-hosted\nLLM Gateway (Proxy)\nwith virtual keys, cost tracking, and an admin UI\nInstallation\n\npip install litellm\nTo run the full Proxy Server (LLM Gateway):\npip install 'litellm[proxy]'\nQuick Start\n\nMake your first LLM call using the provider of your choice:\nOpenAI\nAnthropic\nVertex AI\nBedrock\nOllama\nAzure OpenAI\nfrom\nlitellm\nimport\ncompletion\nimport\nos\nos\n.\nenviron\n[\n\"OPENAI_API_KEY\"\n]\n=\n\"your-api-key\"\nresponse\n=\ncompletion\n(\nmodel\n=\n\"openai\/gpt-4o\"\n,\nmessages\n=\n[\n{\n\"role\"\n:\n\"user\"\n,\n\"content\"\n:\n\"Hello, how are you?\"\n}\n]\n)\nprint\n(\nresponse\n.\nchoices\n[\n0\n]\n.\nmessage\n.\ncontent\n)\nEvery response follows the OpenAI Chat Completions format, regardless of provider. ✅\nResponse Format\n\nNon-streaming responses return a\nModelResponse\nobject:\n{\n\"id\"\n:\n\"chatcmpl-abc123\"\n,\n\"object\"\n:\n\"chat.completion\"\n,\n\"created\"\n:\n1677858242\n,\n\"model\"\n:\n\"gpt-4o\"\n,\n\"choices\"\n:\n[\n{\n\"index\"\n:\n0\n,\n\"message\"\n:\n{\n\"role\"\n:\n\"assistant\"\n,\n\"content\"\n:\n\"Hello! I'm doing well, thanks for asking.\"\n}\n,\n\"finish_reason\"\n:\n\"stop\"\n}\n]\n,\n\"usage\"\n:\n{\n\"prompt_tokens\"\n:\n13\n,\n\"completion_tokens\"\n:\n12\n,\n\"total_tokens\"\n:\n25\n}\n}\nStreaming responses (\nstream=True\n) yield\nModelResponseStream\nchunks:\n{\n\"id\"\n:\n\"chatcmpl-abc123\"\n,\n\"object\"\n:\n\"chat.completion.chunk\"\n,\n\"created\"\n:\n1677858242\n,\n\"model\"\n:\n\"gpt-4o\"\n,\n\"choices\"\n:\n[\n{\n\"index\"\n:\n0\n,\n\"delta\"\n:\n{\n\"role\"\n:\n\"assistant\"\n,\n\"content\"\n:\n\"Hello\"\n}\n,\n\"finish_reason\"\n:\nnull\n}\n]\n}\n📖\nFull output format reference →\nOpen in Colab\nNew to LiteLLM?\n\nWant to get started fast?\nHead to\nTutorials\nfor step-by-step walkthroughs — AI coding tools, agent SDKs, proxy setup, and more.\nNeed to understand a specific feature?\nCheck\nGuides\nfor streaming, function calling, prompt caching, and other how-tos.\nChoose Your Path\n\nLiteLLM Python SDK\n\nStreaming\n\nAdd\nstream=True\nto receive chunks as they are generated:\nfrom\nlitellm\nimport\ncompletion\nimport\nos\nos\n.\nenviron\n[\n\"OPENAI_API_KEY\"\n]\n=\n\"your-api-key\"\nfor\nchunk\nin\ncompletion\n(\nmodel\n=\n\"openai\/gpt-4o\"\n,\nmessages\n=\n[\n{\n\"role\"\n:\n\"user\"\n,\n\"content\"\n:\n\"Write a short poem\"\n}\n]\n,\nstream\n=\nTrue\n,\n)\n:\nprint\n(\nchunk\n.\nchoices\n[\n0\n]\n.\ndelta\n.\ncontent\nor\n\"\"\n,\nend\n=\n\"\"\n)\nException Handling\n\nLiteLLM maps every provider's errors to the OpenAI exception types — your existing error handling works out of the box:\nimport\nlitellm\ntry\n:\nlitellm\n.\ncompletion\n(\nmodel\n=\n\"anthropic\/claude-instant-1\"\n,\nmessages\n=\n[\n{\n\"role\"\n:\n\"user\"\n,\n\"content\"\n:\n\"Hey!\"\n}\n]\n)\nexcept\nlitellm\n.\nAuthenticationError\nas\ne\n:\nprint\n(\nf\"Bad API key:\n{\ne\n}\n\"\n)\nexcept\nlitellm\n.\nRateLimitError\nas\ne\n:\nprint\n(\nf\"Rate limited:\n{\ne\n}\n\"\n)\nexcept\nlitellm\n.\nAPIError\nas\ne\n:\nprint\n(\nf\"API error:\n{\ne\n}\n\"\n)\nLogging & Observability\n\nSend input\/output to Langfuse, MLflow, Helicone, Lunary, and more with a single line:\nimport\nlitellm\nlitellm\n.\nsuccess_callback\n=\n[\n\"langfuse\"\n,\n\"mlflow\"\n,\n\"helicone\"\n]\nresponse\n=\nlitellm\n.\ncompletion\n(\nmodel\n=\n\"gpt-4o\"\n,\nmessages\n=\n[\n{\n\"role\"\n:\n\"user\"\n,\n\"content\"\n:\n\"Hi!\"\n}\n]\n)\n📖\nSee all observability integrations →\nTrack Costs & Usage\n\nUse a callback to capture cost per response:\nimport\nlitellm\ndef\ntrack_cost\n(\nkwargs\n,\ncompletion_response\n,\nstart_time\n,\nend_time\n)\n:\nprint\n(\n\"Cost:\"\n,\nkwargs\n.\nget\n(\n\"response_cost\"\n,\n0\n)\n)\nlitellm\n.\nsuccess_callback\n=\n[\ntrack_cost\n]\nlitellm\n.\ncompletion\n(\nmodel\n=\n\"gpt-4o\"\n,\nmessages\n=\n[\n{\n\"role\"\n:\n\"user\"\n,\n\"content\"\n:\n\"Hello!\"\n}\n]\n,\nstream\n=\nTrue\n)\n📖\nCustom callback docs →\nLiteLLM Proxy Server (LLM Gateway)\n\nThe proxy is a self-hosted OpenAI-compatible gateway. Any client that works with OpenAI works with the proxy — no code changes needed.\nStep 1 — Start the proxy\n\npip\nDocker\nlitellm --model huggingface\/bigcode\/starcoder\n# Proxy running on http:\/\/0.0.0.0:4000\nStep 2 — Call it with the OpenAI client\n\nimport\nopenai\nclient\n=\nopenai\n.\nOpenAI\n(\napi_key\n=\n\"anything\"\n,\nbase_url\n=\n\"http:\/\/0.0.0.0:4000\"\n)\nresponse\n=\nclient\n.\nchat\n.\ncompletions\n.\ncreate\n(\nmodel\n=\n\"gpt-3.5-turbo\"\n,\nmessages\n=\n[\n{\n\"role\"\n:\n\"user\"\n,\n\"content\"\n:\n\"Write a short poem\"\n}\n]\n)\nprint\n(\nresponse\n.\nchoices\n[\n0\n]\n.\nmessage\n.\ncontent\n)\n👉\nFull proxy quickstart with Docker →\nDebugging tool\nUse\n\/utils\/transform_request\nto inspect exactly what LiteLLM sends to any provider — useful for debugging prompt formatting, header issues, and provider-specific parameters.\n🔗\nInteractive API explorer (Swagger) →\nAgent & MCP Gateway\n\nLiteLLM is a unified gateway for\nLLMs, agents, and MCP\n— you don't need a separate agent or MCP gateway. One endpoint for 100+ models, A2A agents, and MCP tools.\nWhat to Explore Next\n\n🔀\nRouting & Load Balancing\nLoad balance across deployments and set automatic fallbacks.\n🔑\nVirtual Keys\nManage access, budgets, and rate limits per team or user.\n📊\nSpend Tracking\nTrack costs per key, team, and user across all providers.\n🛡️\nGuardrails\nAdd content filtering, PII masking, and safety checks.\n📡\nObservability\nIntegrate with Langfuse, MLflow, Helicone, and more.\n🏭\nEnterprise\nSSO\/SAML, audit logs, and advanced security for production.","attrs_markdown":"[Skip to main content](https:\/\/docs.litellm.ai\/docs\/#__docusaurus_skipToContent_fallback)\n\n[**🚅 LiteLLM**](https:\/\/docs.litellm.ai\/)\n\n[Docs](https:\/\/docs.litellm.ai\/docs\/)[Learn](https:\/\/docs.litellm.ai\/docs\/learn)[Integrations](https:\/\/docs.litellm.ai\/docs\/integrations\/)[Enterprise](https:\/\/docs.litellm.ai\/docs\/enterprise)[Changelog](https:\/\/docs.litellm.ai\/release_notes)[Blog](https:\/\/docs.litellm.ai\/blog)\n\n- [Get Started]()\n  - [Quickstart](https:\/\/docs.litellm.ai\/docs\/)\n  - [Models & Pricing](https:\/\/models.litellm.ai\/)\n  - [Changelog](https:\/\/docs.litellm.ai\/release_notes)\n- [LiteLLM Python SDK](https:\/\/docs.litellm.ai\/docs\/#litellm-python-sdk)\n- [LiteLLM AI Gateway (Proxy)](https:\/\/docs.litellm.ai\/docs\/simple_proxy)\n- [Supported Endpoints](https:\/\/docs.litellm.ai\/docs\/supported_endpoints)\n- [Supported Models & Providers](https:\/\/docs.litellm.ai\/docs\/providers)\n- [Routing & Load Balancing](https:\/\/docs.litellm.ai\/docs\/routing-load-balancing)\n- [Load Testing](https:\/\/docs.litellm.ai\/docs\/benchmarks)\n- [Contributing](https:\/\/docs.litellm.ai\/docs\/extras\/contributing_code)\n- [Extras](https:\/\/docs.litellm.ai\/docs\/sdk_custom_pricing)\n- [Troubleshooting](https:\/\/docs.litellm.ai\/docs\/troubleshoot\/ui_issues)\n\n- Get Started\n- Quickstart\n\nOn this page\n\n# Getting Started\n![](https:\/\/docs.litellm.ai\/assets\/ideal-img\/hero.1927c06.640.png)\n\n**LiteLLM** is an open-source library that gives you a single, unified interface to call 100+ LLMs — OpenAI, Anthropic, Vertex AI, Bedrock, and more — using the OpenAI format.\n\n- Call any provider using the same `completion()` interface — no re-learning the API for each one\n- Consistent output format regardless of which provider or model you use\n- Built-in retry \/ fallback logic across multiple deployments via the [Router](https:\/\/docs.litellm.ai\/docs\/routing)\n- Self-hosted [LLM Gateway (Proxy)](https:\/\/docs.litellm.ai\/docs\/simple_proxy) with virtual keys, cost tracking, and an admin UI\n\n[![PyPI](https:\/\/img.shields.io\/pypi\/v\/litellm.svg)](https:\/\/pypi.org\/project\/litellm\/) [![GitHub Stars](https:\/\/img.shields.io\/github\/stars\/BerriAI\/litellm?style=social)](https:\/\/github.com\/BerriAI\/litellm)\n***\n## Installation[](https:\/\/docs.litellm.ai\/docs\/#installation \"Direct link to Installation\")\n```\npip install litellm\n```\nTo run the full Proxy Server (LLM Gateway):\n```\npip install 'litellm[proxy]'\n```\n***\n## Quick Start[](https:\/\/docs.litellm.ai\/docs\/#quick-start \"Direct link to Quick Start\")\nMake your first LLM call using the provider of your choice:\n\n- OpenAI\n- Anthropic\n- Vertex AI\n- Bedrock\n- Ollama\n- Azure OpenAI\n\n```\nfrom litellm import completion\nimport os\n\nos.environ[\"OPENAI_API_KEY\"] = \"your-api-key\"\n\nresponse = completion(\n  model=\"openai\/gpt-4o\",\n  messages=[{\"role\": \"user\", \"content\": \"Hello, how are you?\"}]\n)\nprint(response.choices[0].message.content)\n```\n```\nfrom litellm import completion\nimport os\n\nos.environ[\"ANTHROPIC_API_KEY\"] = \"your-api-key\"\n\nresponse = completion(\n  model=\"anthropic\/claude-3-5-sonnet-20241022\",\n  messages=[{\"role\": \"user\", \"content\": \"Hello, how are you?\"}]\n)\nprint(response.choices[0].message.content)\n```\n```\nfrom litellm import completion\nimport os\n\n# auth: run 'gcloud auth application-default login'\nos.environ[\"VERTEXAI_PROJECT\"] = \"your-project-id\"\nos.environ[\"VERTEXAI_LOCATION\"] = \"us-central1\"\n\nresponse = completion(\n  model=\"vertex_ai\/gemini-1.5-pro\",\n  messages=[{\"role\": \"user\", \"content\": \"Hello, how are you?\"}]\n)\nprint(response.choices[0].message.content)\n```\n```\nfrom litellm import completion\nimport os\n\nos.environ[\"AWS_ACCESS_KEY_ID\"] = \"your-key\"\nos.environ[\"AWS_SECRET_ACCESS_KEY\"] = \"your-secret\"\nos.environ[\"AWS_REGION_NAME\"] = \"us-east-1\"\n\nresponse = completion(\n  model=\"bedrock\/anthropic.claude-haiku-4-5-20251001:0\",\n  messages=[{\"role\": \"user\", \"content\": \"Hello, how are you?\"}]\n)\nprint(response.choices[0].message.content)\n```\n```\nfrom litellm import completion\n\nresponse = completion(\n  model=\"ollama\/llama3\",\n  messages=[{\"role\": \"user\", \"content\": \"Hello, how are you?\"}],\n  api_base=\"http:\/\/localhost:11434\"\n)\nprint(response.choices[0].message.content)\n```\n```\nfrom litellm import completion\nimport os\n\nos.environ[\"AZURE_API_KEY\"] = \"your-key\"\nos.environ[\"AZURE_API_BASE\"] = \"https:\/\/your-resource.openai.azure.com\"\nos.environ[\"AZURE_API_VERSION\"] = \"2024-02-01\"\n\nresponse = completion(\n  model=\"azure\/your-deployment-name\",\n  messages=[{\"role\": \"user\", \"content\": \"Hello, how are you?\"}]\n)\nprint(response.choices[0].message.content)\n```\n\nEvery response follows the OpenAI Chat Completions format, regardless of provider. ✅\n\n### Response Format[](https:\/\/docs.litellm.ai\/docs\/#response-format \"Direct link to Response Format\")\nNon-streaming responses return a `ModelResponse` object:\n```\n{\n  \"id\": \"chatcmpl-abc123\",\n  \"object\": \"chat.completion\",\n  \"created\": 1677858242,\n  \"model\": \"gpt-4o\",\n  \"choices\": [\n    {\n      \"index\": 0,\n      \"message\": {\n        \"role\": \"assistant\",\n        \"content\": \"Hello! I'm doing well, thanks for asking.\"\n      },\n      \"finish_reason\": \"stop\"\n    }\n  ],\n  \"usage\": {\n    \"prompt_tokens\": 13,\n    \"completion_tokens\": 12,\n    \"total_tokens\": 25\n  }\n}\n```\nStreaming responses (`stream=True`) yield `ModelResponseStream` chunks:\n```\n{\n  \"id\": \"chatcmpl-abc123\",\n  \"object\": \"chat.completion.chunk\",\n  \"created\": 1677858242,\n  \"model\": \"gpt-4o\",\n  \"choices\": [\n    {\n      \"index\": 0,\n      \"delta\": {\n        \"role\": \"assistant\",\n        \"content\": \"Hello\"\n      },\n      \"finish_reason\": null\n    }\n  ]\n}\n```\n📖 [Full output format reference →](https:\/\/docs.litellm.ai\/docs\/completion\/output)\n\nOpen in Colab\n\n[![Open In Colab](https:\/\/colab.research.google.com\/assets\/colab-badge.svg)](https:\/\/colab.research.google.com\/github\/BerriAI\/litellm\/blob\/main\/cookbook\/liteLLM_Getting_Started.ipynb)\n***\n## New to LiteLLM?[](https:\/\/docs.litellm.ai\/docs\/#new-to-litellm \"Direct link to New to LiteLLM?\")\n**Want to get started fast?** Head to [Tutorials](https:\/\/docs.litellm.ai\/docs\/tutorials) for step-by-step walkthroughs — AI coding tools, agent SDKs, proxy setup, and more.\n\n**Need to understand a specific feature?** Check [Guides](https:\/\/docs.litellm.ai\/docs\/guides) for streaming, function calling, prompt caching, and other how-tos.\n***\n## Choose Your Path[](https:\/\/docs.litellm.ai\/docs\/#choose-your-path \"Direct link to Choose Your Path\")\n[🐍Python SDKIntegrate LiteLLM directly into your Python application. Drop-in replacement for the OpenAI client.completion(), embedding(), image\\_generation() and more Router with retry, fallback, and load balancing OpenAI-compatible exceptions across all providers Observability callbacks (Langfuse, MLflow, Helicone…)](https:\/\/docs.litellm.ai\/docs\/#litellm-python-sdk)\n\n[🖥️Proxy Server (LLM Gateway)Self-hosted gateway for platform teams managing LLM access across an organization.Virtual keys with per-key\/team\/user budgets Centralized logging, guardrails, and caching Admin UI for monitoring and management Drop-in replacement for any OpenAI-compatible client](https:\/\/docs.litellm.ai\/docs\/#litellm-proxy-server-llm-gateway)\n***\n## LiteLLM Python SDK[](https:\/\/docs.litellm.ai\/docs\/#litellm-python-sdk \"Direct link to LiteLLM Python SDK\")\n### Streaming[](https:\/\/docs.litellm.ai\/docs\/#streaming \"Direct link to Streaming\")\nAdd `stream=True` to receive chunks as they are generated:\n```\nfrom litellm import completion\nimport os\n\nos.environ[\"OPENAI_API_KEY\"] = \"your-api-key\"\n\nfor chunk in completion(\n  model=\"openai\/gpt-4o\",\n  messages=[{\"role\": \"user\", \"content\": \"Write a short poem\"}],\n  stream=True,\n):\n    print(chunk.choices[0].delta.content or \"\", end=\"\")\n```\n### Exception Handling[](https:\/\/docs.litellm.ai\/docs\/#exception-handling \"Direct link to Exception Handling\")\nLiteLLM maps every provider's errors to the OpenAI exception types — your existing error handling works out of the box:\n```\nimport litellm\n\ntry:\n    litellm.completion(\n      model=\"anthropic\/claude-instant-1\",\n      messages=[{\"role\": \"user\", \"content\": \"Hey!\"}]\n    )\nexcept litellm.AuthenticationError as e:\n    print(f\"Bad API key: {e}\")\nexcept litellm.RateLimitError as e:\n    print(f\"Rate limited: {e}\")\nexcept litellm.APIError as e:\n    print(f\"API error: {e}\")\n```\n### Logging & Observability[](https:\/\/docs.litellm.ai\/docs\/#logging--observability \"Direct link to Logging & Observability\")\nSend input\/output to Langfuse, MLflow, Helicone, Lunary, and more with a single line:\n```\nimport litellm\n\nlitellm.success_callback = [\"langfuse\", \"mlflow\", \"helicone\"]\n\nresponse = litellm.completion(\n  model=\"gpt-4o\",\n  messages=[{\"role\": \"user\", \"content\": \"Hi!\"}]\n)\n```\n📖 [See all observability integrations →](https:\/\/docs.litellm.ai\/docs\/observability\/agentops_integration)\n\n### Track Costs & Usage[](https:\/\/docs.litellm.ai\/docs\/#track-costs--usage \"Direct link to Track Costs & Usage\")\nUse a callback to capture cost per response:\n```\nimport litellm\n\ndef track_cost(kwargs, completion_response, start_time, end_time):\n    print(\"Cost:\", kwargs.get(\"response_cost\", 0))\n\nlitellm.success_callback = [track_cost]\n\nlitellm.completion(\n  model=\"gpt-4o\",\n  messages=[{\"role\": \"user\", \"content\": \"Hello!\"}],\n  stream=True\n)\n```\n📖 [Custom callback docs →](https:\/\/docs.litellm.ai\/docs\/observability\/custom_callback)\n***\n## LiteLLM Proxy Server (LLM Gateway)[](https:\/\/docs.litellm.ai\/docs\/#litellm-proxy-server-llm-gateway \"Direct link to LiteLLM Proxy Server (LLM Gateway)\")\nThe proxy is a self-hosted OpenAI-compatible gateway. Any client that works with OpenAI works with the proxy — no code changes needed.\n\n![LiteLLM Proxy Dashboard](https:\/\/github.com\/BerriAI\/litellm\/assets\/29436595\/47c97d5e-b9be-4839-b28c-43d7f4f10033)\n\n#### Step 1 — Start the proxy[](https:\/\/docs.litellm.ai\/docs\/#step-1--start-the-proxy \"Direct link to Step 1 — Start the proxy\")\n- pip\n- Docker\n\n```\nlitellm --model huggingface\/bigcode\/starcoder\n# Proxy running on http:\/\/0.0.0.0:4000\n```\nlitellm\\_config.yaml\n```\nmodel_list:\n  - model_name: gpt-3.5-turbo\n    litellm_params:\n      model: azure\/your-deployment\n      api_base: os.environ\/AZURE_API_BASE\n      api_key: os.environ\/AZURE_API_KEY\n      api_version: \"2023-07-01-preview\"\n```\n```\ndocker run \\\n  -v $(pwd)\/litellm_config.yaml:\/app\/config.yaml \\\n  -e AZURE_API_KEY=your-key \\\n  -e AZURE_API_BASE=https:\/\/your-resource.openai.azure.com\/ \\\n  -p 4000:4000 \\\n  docker.litellm.ai\/berriai\/litellm:main-latest \\\n  --config \/app\/config.yaml --detailed_debug\n```\n#### Step 2 — Call it with the OpenAI client[](https:\/\/docs.litellm.ai\/docs\/#step-2--call-it-with-the-openai-client \"Direct link to Step 2 — Call it with the OpenAI client\")\n```\nimport openai\n\nclient = openai.OpenAI(api_key=\"anything\", base_url=\"http:\/\/0.0.0.0:4000\")\n\nresponse = client.chat.completions.create(\n  model=\"gpt-3.5-turbo\",\n  messages=[{\"role\": \"user\", \"content\": \"Write a short poem\"}]\n)\nprint(response.choices[0].message.content)\n```\n👉 [Full proxy quickstart with Docker →](https:\/\/docs.litellm.ai\/docs\/proxy\/docker_quick_start)\n\nDebugging tool\n\nUse [**`\/utils\/transform_request`**](https:\/\/docs.litellm.ai\/docs\/utils\/transform_request) to inspect exactly what LiteLLM sends to any provider — useful for debugging prompt formatting, header issues, and provider-specific parameters.\n\n🔗 [Interactive API explorer (Swagger) →](https:\/\/litellm-api.up.railway.app\/)\n***\n## Agent & MCP Gateway[](https:\/\/docs.litellm.ai\/docs\/#agent--mcp-gateway \"Direct link to Agent & MCP Gateway\")\nLiteLLM is a unified gateway for **LLMs, agents, and MCP** — you don't need a separate agent or MCP gateway. One endpoint for 100+ models, A2A agents, and MCP tools.\n\n[🔗A2A AgentsAdd and invoke A2A agents via the LiteLLM gateway.](https:\/\/docs.litellm.ai\/docs\/a2a)\n\n[🛠️MCP GatewayCentral MCP endpoint with per-key access control.](https:\/\/docs.litellm.ai\/docs\/mcp)\n***\n## What to Explore Next[](https:\/\/docs.litellm.ai\/docs\/#what-to-explore-next \"Direct link to What to Explore Next\")\n[🔀Routing & Load BalancingLoad balance across deployments and set automatic fallbacks.](https:\/\/docs.litellm.ai\/docs\/routing-load-balancing)\n\n[🔑Virtual KeysManage access, budgets, and rate limits per team or user.](https:\/\/docs.litellm.ai\/docs\/proxy\/virtual_keys)\n\n[📊Spend TrackingTrack costs per key, team, and user across all providers.](https:\/\/docs.litellm.ai\/docs\/proxy\/cost_tracking)\n\n[🛡️GuardrailsAdd content filtering, PII masking, and safety checks.](https:\/\/docs.litellm.ai\/docs\/proxy\/guardrails\/quick_start)\n\n[📡ObservabilityIntegrate with Langfuse, MLflow, Helicone, and more.](https:\/\/docs.litellm.ai\/docs\/observability\/agentops_integration)\n\n[🏭EnterpriseSSO\/SAML, audit logs, and advanced security for production.](https:\/\/docs.litellm.ai\/docs\/enterprise)\n\n[Nextcompletion()](https:\/\/docs.litellm.ai\/docs\/completion\/input)\n\n- [Installation](https:\/\/docs.litellm.ai\/docs\/#installation)\n- [Quick Start](https:\/\/docs.litellm.ai\/docs\/#quick-start)\n  - [Response Format](https:\/\/docs.litellm.ai\/docs\/#response-format)\n- [New to LiteLLM?](https:\/\/docs.litellm.ai\/docs\/#new-to-litellm)\n- [Choose Your Path](https:\/\/docs.litellm.ai\/docs\/#choose-your-path)\n- [LiteLLM Python SDK](https:\/\/docs.litellm.ai\/docs\/#litellm-python-sdk)\n  - [Streaming](https:\/\/docs.litellm.ai\/docs\/#streaming)\n  - [Exception Handling](https:\/\/docs.litellm.ai\/docs\/#exception-handling)\n  - [Logging & Observability](https:\/\/docs.litellm.ai\/docs\/#logging--observability)\n  - [Track Costs & Usage](https:\/\/docs.litellm.ai\/docs\/#track-costs--usage)\n- [LiteLLM Proxy Server (LLM Gateway)](https:\/\/docs.litellm.ai\/docs\/#litellm-proxy-server-llm-gateway)\n- [Agent & MCP Gateway](https:\/\/docs.litellm.ai\/docs\/#agent--mcp-gateway)\n- [What to Explore Next](https:\/\/docs.litellm.ai\/docs\/#what-to-explore-next)\n\n🚅\n\nLiteLLM Enterprise\n\nSSO\/SAML, audit logs, spend tracking, multi-team management, and guardrails — built for production.\n\n[Learn more →](https:\/\/docs.litellm.ai\/docs\/enterprise)\n\nDocs\n\n- [Getting Started](https:\/\/docs.litellm.ai\/docs\/)\n\nCommunity\n\n- [Discord](https:\/\/discord.com\/invite\/wuPM9dRgDw)\n- [Twitter](https:\/\/twitter.com\/LiteLLM)\n\nMore\n\n- [GitHub](https:\/\/github.com\/BerriAI\/litellm\/)\n\nCopyright © 2026 liteLLM","attrs_readable_markdown":"**LiteLLM** is an open-source library that gives you a single, unified interface to call 100+ LLMs — OpenAI, Anthropic, Vertex AI, Bedrock, and more — using the OpenAI format.\n\n- Call any provider using the same `completion()` interface — no re-learning the API for each one\n- Consistent output format regardless of which provider or model you use\n- Built-in retry \/ fallback logic across multiple deployments via the [Router](https:\/\/docs.litellm.ai\/docs\/routing)\n- Self-hosted [LLM Gateway (Proxy)](https:\/\/docs.litellm.ai\/docs\/simple_proxy) with virtual keys, cost tracking, and an admin UI\n\n[![PyPI](https:\/\/img.shields.io\/pypi\/v\/litellm.svg)](https:\/\/pypi.org\/project\/litellm\/) [![GitHub Stars](https:\/\/img.shields.io\/github\/stars\/BerriAI\/litellm?style=social)](https:\/\/github.com\/BerriAI\/litellm)\n***\n## Installation[](https:\/\/docs.litellm.ai\/docs\/#installation \"Direct link to Installation\")\n```\npip install litellm\n```\nTo run the full Proxy Server (LLM Gateway):\n```\npip install 'litellm[proxy]'\n```\n***\n## Quick Start[](https:\/\/docs.litellm.ai\/docs\/#quick-start \"Direct link to Quick Start\")\nMake your first LLM call using the provider of your choice:\n\n- OpenAI\n- Anthropic\n- Vertex AI\n- Bedrock\n- Ollama\n- Azure OpenAI\n```\nfrom litellm import completion\nimport os\n\nos.environ[\"OPENAI_API_KEY\"] = \"your-api-key\"\n\nresponse = completion(\n  model=\"openai\/gpt-4o\",\n  messages=[{\"role\": \"user\", \"content\": \"Hello, how are you?\"}]\n)\nprint(response.choices[0].message.content)\n```\n\nEvery response follows the OpenAI Chat Completions format, regardless of provider. ✅\n\n### Response Format[](https:\/\/docs.litellm.ai\/docs\/#response-format \"Direct link to Response Format\")\nNon-streaming responses return a `ModelResponse` object:\n```\n{\n  \"id\": \"chatcmpl-abc123\",\n  \"object\": \"chat.completion\",\n  \"created\": 1677858242,\n  \"model\": \"gpt-4o\",\n  \"choices\": [\n    {\n      \"index\": 0,\n      \"message\": {\n        \"role\": \"assistant\",\n        \"content\": \"Hello! I'm doing well, thanks for asking.\"\n      },\n      \"finish_reason\": \"stop\"\n    }\n  ],\n  \"usage\": {\n    \"prompt_tokens\": 13,\n    \"completion_tokens\": 12,\n    \"total_tokens\": 25\n  }\n}\n```\nStreaming responses (`stream=True`) yield `ModelResponseStream` chunks:\n```\n{\n  \"id\": \"chatcmpl-abc123\",\n  \"object\": \"chat.completion.chunk\",\n  \"created\": 1677858242,\n  \"model\": \"gpt-4o\",\n  \"choices\": [\n    {\n      \"index\": 0,\n      \"delta\": {\n        \"role\": \"assistant\",\n        \"content\": \"Hello\"\n      },\n      \"finish_reason\": null\n    }\n  ]\n}\n```\n📖 [Full output format reference →](https:\/\/docs.litellm.ai\/docs\/completion\/output)\n\nOpen in Colab\n\n[![Open In Colab](https:\/\/colab.research.google.com\/assets\/colab-badge.svg)](https:\/\/colab.research.google.com\/github\/BerriAI\/litellm\/blob\/main\/cookbook\/liteLLM_Getting_Started.ipynb)\n***\n## New to LiteLLM?[](https:\/\/docs.litellm.ai\/docs\/#new-to-litellm \"Direct link to New to LiteLLM?\")\n**Want to get started fast?** Head to [Tutorials](https:\/\/docs.litellm.ai\/docs\/tutorials) for step-by-step walkthroughs — AI coding tools, agent SDKs, proxy setup, and more.\n\n**Need to understand a specific feature?** Check [Guides](https:\/\/docs.litellm.ai\/docs\/guides) for streaming, function calling, prompt caching, and other how-tos.\n***\n## Choose Your Path[](https:\/\/docs.litellm.ai\/docs\/#choose-your-path \"Direct link to Choose Your Path\")\n***\n## LiteLLM Python SDK[](https:\/\/docs.litellm.ai\/docs\/#litellm-python-sdk \"Direct link to LiteLLM Python SDK\")\n### Streaming[](https:\/\/docs.litellm.ai\/docs\/#streaming \"Direct link to Streaming\")\nAdd `stream=True` to receive chunks as they are generated:\n```\nfrom litellm import completion\nimport os\n\nos.environ[\"OPENAI_API_KEY\"] = \"your-api-key\"\n\nfor chunk in completion(\n  model=\"openai\/gpt-4o\",\n  messages=[{\"role\": \"user\", \"content\": \"Write a short poem\"}],\n  stream=True,\n):\n    print(chunk.choices[0].delta.content or \"\", end=\"\")\n```\n### Exception Handling[](https:\/\/docs.litellm.ai\/docs\/#exception-handling \"Direct link to Exception Handling\")\nLiteLLM maps every provider's errors to the OpenAI exception types — your existing error handling works out of the box:\n```\nimport litellm\n\ntry:\n    litellm.completion(\n      model=\"anthropic\/claude-instant-1\",\n      messages=[{\"role\": \"user\", \"content\": \"Hey!\"}]\n    )\nexcept litellm.AuthenticationError as e:\n    print(f\"Bad API key: {e}\")\nexcept litellm.RateLimitError as e:\n    print(f\"Rate limited: {e}\")\nexcept litellm.APIError as e:\n    print(f\"API error: {e}\")\n```\n### Logging & Observability[](https:\/\/docs.litellm.ai\/docs\/#logging--observability \"Direct link to Logging & Observability\")\nSend input\/output to Langfuse, MLflow, Helicone, Lunary, and more with a single line:\n```\nimport litellm\n\nlitellm.success_callback = [\"langfuse\", \"mlflow\", \"helicone\"]\n\nresponse = litellm.completion(\n  model=\"gpt-4o\",\n  messages=[{\"role\": \"user\", \"content\": \"Hi!\"}]\n)\n```\n📖 [See all observability integrations →](https:\/\/docs.litellm.ai\/docs\/observability\/agentops_integration)\n\n### Track Costs & Usage[](https:\/\/docs.litellm.ai\/docs\/#track-costs--usage \"Direct link to Track Costs & Usage\")\nUse a callback to capture cost per response:\n```\nimport litellm\n\ndef track_cost(kwargs, completion_response, start_time, end_time):\n    print(\"Cost:\", kwargs.get(\"response_cost\", 0))\n\nlitellm.success_callback = [track_cost]\n\nlitellm.completion(\n  model=\"gpt-4o\",\n  messages=[{\"role\": \"user\", \"content\": \"Hello!\"}],\n  stream=True\n)\n```\n📖 [Custom callback docs →](https:\/\/docs.litellm.ai\/docs\/observability\/custom_callback)\n***\n## LiteLLM Proxy Server (LLM Gateway)[](https:\/\/docs.litellm.ai\/docs\/#litellm-proxy-server-llm-gateway \"Direct link to LiteLLM Proxy Server (LLM Gateway)\")\nThe proxy is a self-hosted OpenAI-compatible gateway. Any client that works with OpenAI works with the proxy — no code changes needed.\n\n![LiteLLM Proxy Dashboard](https:\/\/github.com\/BerriAI\/litellm\/assets\/29436595\/47c97d5e-b9be-4839-b28c-43d7f4f10033)\n\n#### Step 1 — Start the proxy[](https:\/\/docs.litellm.ai\/docs\/#step-1--start-the-proxy \"Direct link to Step 1 — Start the proxy\")\n- pip\n- Docker\n```\nlitellm --model huggingface\/bigcode\/starcoder\n# Proxy running on http:\/\/0.0.0.0:4000\n```\n#### Step 2 — Call it with the OpenAI client[](https:\/\/docs.litellm.ai\/docs\/#step-2--call-it-with-the-openai-client \"Direct link to Step 2 — Call it with the OpenAI client\")\n```\nimport openai\n\nclient = openai.OpenAI(api_key=\"anything\", base_url=\"http:\/\/0.0.0.0:4000\")\n\nresponse = client.chat.completions.create(\n  model=\"gpt-3.5-turbo\",\n  messages=[{\"role\": \"user\", \"content\": \"Write a short poem\"}]\n)\nprint(response.choices[0].message.content)\n```\n👉 [Full proxy quickstart with Docker →](https:\/\/docs.litellm.ai\/docs\/proxy\/docker_quick_start)\n\nDebugging tool\n\nUse [**`\/utils\/transform_request`**](https:\/\/docs.litellm.ai\/docs\/utils\/transform_request) to inspect exactly what LiteLLM sends to any provider — useful for debugging prompt formatting, header issues, and provider-specific parameters.\n\n🔗 [Interactive API explorer (Swagger) →](https:\/\/litellm-api.up.railway.app\/)\n***\n## Agent & MCP Gateway[](https:\/\/docs.litellm.ai\/docs\/#agent--mcp-gateway \"Direct link to Agent & MCP Gateway\")\nLiteLLM is a unified gateway for **LLMs, agents, and MCP** — you don't need a separate agent or MCP gateway. One endpoint for 100+ models, A2A agents, and MCP tools.\n***\n## What to Explore Next[](https:\/\/docs.litellm.ai\/docs\/#what-to-explore-next \"Direct link to What to Explore Next\")\n[🔀Routing & Load BalancingLoad balance across deployments and set automatic fallbacks.](https:\/\/docs.litellm.ai\/docs\/routing-load-balancing)\n\n[🔑Virtual KeysManage access, budgets, and rate limits per team or user.](https:\/\/docs.litellm.ai\/docs\/proxy\/virtual_keys)\n\n[📊Spend TrackingTrack costs per key, team, and user across all providers.](https:\/\/docs.litellm.ai\/docs\/proxy\/cost_tracking)\n\n[🛡️GuardrailsAdd content filtering, PII masking, and safety checks.](https:\/\/docs.litellm.ai\/docs\/proxy\/guardrails\/quick_start)\n\n[📡ObservabilityIntegrate with Langfuse, MLflow, Helicone, and more.](https:\/\/docs.litellm.ai\/docs\/observability\/agentops_integration)\n\n[🏭EnterpriseSSO\/SAML, audit logs, and advanced security for production.](https:\/\/docs.litellm.ai\/docs\/enterprise)","meta_canonical":null}

3. Robots.txt Check

Query:

Response:

4. Spam/Ban Check

Query:

Response:

5. Seen Status Check

ℹ️ Skipped - page is already crawled

📄

INDEXABLE

✅

CRAWLED

3 days ago

🤖

ROBOTS ALLOWED

Page Info Filters

Filter	Status	Condition	Details
HTTP status	PASS	`download_http_code = 200`	HTTP 200
Age cutoff	PASS	`download_stamp > now() - 6 MONTH`	0.1 months ago
History drop	PASS	`isNull(history_drop_reason)`	No drop reason
Spam/ban	PASS	`fh_dont_index != 1 AND ml_spam_score = 0`	ml_spam_score=0
Canonical	PASS	`meta_canonical IS NULL OR = '' OR = src_unparsed`	Not set

Page Details

Property	Value
URL	https://docs.litellm.ai/docs/
Last Crawled	2026-04-06 13:05:20 (3 days ago)
First Indexed	2023-08-15 00:44:53 (2 years ago)
HTTP Status Code	200
Meta Title	Getting Started \| liteLLM
Meta Description	LiteLLM is an open-source library that gives you a single, unified interface to call 100+ LLMs — OpenAI, Anthropic, Vertex AI, Bedrock, and more — using the OpenAI format.
Meta Canonical	null
Boilerpipe Text	LiteLLM is an open-source library that gives you a single, unified interface to call 100+ LLMs — OpenAI, Anthropic, Vertex AI, Bedrock, and more — using the OpenAI format. Call any provider using the same completion() interface — no re-learning the API for each one Consistent output format regardless of which provider or model you use Built-in retry / fallback logic across multiple deployments via the Router Self-hosted LLM Gateway (Proxy) with virtual keys, cost tracking, and an admin UI Installation pip install litellm To run the full Proxy Server (LLM Gateway): pip install 'litellm[proxy]' Quick Start Make your first LLM call using the provider of your choice: OpenAI Anthropic Vertex AI Bedrock Ollama Azure OpenAI from litellm import completion import os os . environ [ "OPENAI_API_KEY" ] = "your-api-key" response = completion ( model = "openai/gpt-4o" , messages = [ { "role" : "user" , "content" : "Hello, how are you?" } ] ) print ( response . choices [ 0 ] . message . content ) Every response follows the OpenAI Chat Completions format, regardless of provider. ✅ Response Format Non-streaming responses return a ModelResponse object: { "id" : "chatcmpl-abc123" , "object" : "chat.completion" , "created" : 1677858242 , "model" : "gpt-4o" , "choices" : [ { "index" : 0 , "message" : { "role" : "assistant" , "content" : "Hello! I'm doing well, thanks for asking." } , "finish_reason" : "stop" } ] , "usage" : { "prompt_tokens" : 13 , "completion_tokens" : 12 , "total_tokens" : 25 } } Streaming responses ( stream=True ) yield ModelResponseStream chunks: { "id" : "chatcmpl-abc123" , "object" : "chat.completion.chunk" , "created" : 1677858242 , "model" : "gpt-4o" , "choices" : [ { "index" : 0 , "delta" : { "role" : "assistant" , "content" : "Hello" } , "finish_reason" : null } ] } 📖 Full output format reference → Open in Colab New to LiteLLM? Want to get started fast? Head to Tutorials for step-by-step walkthroughs — AI coding tools, agent SDKs, proxy setup, and more. Need to understand a specific feature? Check Guides for streaming, function calling, prompt caching, and other how-tos. Choose Your Path LiteLLM Python SDK Streaming Add stream=True to receive chunks as they are generated: from litellm import completion import os os . environ [ "OPENAI_API_KEY" ] = "your-api-key" for chunk in completion ( model = "openai/gpt-4o" , messages = [ { "role" : "user" , "content" : "Write a short poem" } ] , stream = True , ) : print ( chunk . choices [ 0 ] . delta . content or "" , end = "" ) Exception Handling LiteLLM maps every provider's errors to the OpenAI exception types — your existing error handling works out of the box: import litellm try : litellm . completion ( model = "anthropic/claude-instant-1" , messages = [ { "role" : "user" , "content" : "Hey!" } ] ) except litellm . AuthenticationError as e : print ( f"Bad API key: { e } " ) except litellm . RateLimitError as e : print ( f"Rate limited: { e } " ) except litellm . APIError as e : print ( f"API error: { e } " ) Logging & Observability Send input/output to Langfuse, MLflow, Helicone, Lunary, and more with a single line: import litellm litellm . success_callback = [ "langfuse" , "mlflow" , "helicone" ] response = litellm . completion ( model = "gpt-4o" , messages = [ { "role" : "user" , "content" : "Hi!" } ] ) 📖 See all observability integrations → Track Costs & Usage Use a callback to capture cost per response: import litellm def track_cost ( kwargs , completion_response , start_time , end_time ) : print ( "Cost:" , kwargs . get ( "response_cost" , 0 ) ) litellm . success_callback = [ track_cost ] litellm . completion ( model = "gpt-4o" , messages = [ { "role" : "user" , "content" : "Hello!" } ] , stream = True ) 📖 Custom callback docs → LiteLLM Proxy Server (LLM Gateway) The proxy is a self-hosted OpenAI-compatible gateway. Any client that works with OpenAI works with the proxy — no code changes needed. Step 1 — Start the proxy pip Docker litellm --model huggingface/bigcode/starcoder # Proxy running on http://0.0.0.0:4000 Step 2 — Call it with the OpenAI client import openai client = openai . OpenAI ( api_key = "anything" , base_url = "http://0.0.0.0:4000" ) response = client . chat . completions . create ( model = "gpt-3.5-turbo" , messages = [ { "role" : "user" , "content" : "Write a short poem" } ] ) print ( response . choices [ 0 ] . message . content ) 👉 Full proxy quickstart with Docker → Debugging tool Use /utils/transform_request to inspect exactly what LiteLLM sends to any provider — useful for debugging prompt formatting, header issues, and provider-specific parameters. 🔗 Interactive API explorer (Swagger) → Agent & MCP Gateway LiteLLM is a unified gateway for LLMs, agents, and MCP — you don't need a separate agent or MCP gateway. One endpoint for 100+ models, A2A agents, and MCP tools. What to Explore Next 🔀 Routing & Load Balancing Load balance across deployments and set automatic fallbacks. 🔑 Virtual Keys Manage access, budgets, and rate limits per team or user. 📊 Spend Tracking Track costs per key, team, and user across all providers. 🛡️ Guardrails Add content filtering, PII masking, and safety checks. 📡 Observability Integrate with Langfuse, MLflow, Helicone, and more. 🏭 Enterprise SSO/SAML, audit logs, and advanced security for production.
Markdown	[Skip to main content](https://docs.litellm.ai/docs/#__docusaurus_skipToContent_fallback) [🚅 LiteLLM](https://docs.litellm.ai/) [Docs](https://docs.litellm.ai/docs/)[Learn](https://docs.litellm.ai/docs/learn)[Integrations](https://docs.litellm.ai/docs/integrations/)[Enterprise](https://docs.litellm.ai/docs/enterprise)[Changelog](https://docs.litellm.ai/release_notes)[Blog](https://docs.litellm.ai/blog) - [Get Started]() - [Quickstart](https://docs.litellm.ai/docs/) - [Models & Pricing](https://models.litellm.ai/) - [Changelog](https://docs.litellm.ai/release_notes) - [LiteLLM Python SDK](https://docs.litellm.ai/docs/#litellm-python-sdk) - [LiteLLM AI Gateway (Proxy)](https://docs.litellm.ai/docs/simple_proxy) - [Supported Endpoints](https://docs.litellm.ai/docs/supported_endpoints) - [Supported Models & Providers](https://docs.litellm.ai/docs/providers) - [Routing & Load Balancing](https://docs.litellm.ai/docs/routing-load-balancing) - [Load Testing](https://docs.litellm.ai/docs/benchmarks) - [Contributing](https://docs.litellm.ai/docs/extras/contributing_code) - [Extras](https://docs.litellm.ai/docs/sdk_custom_pricing) - [Troubleshooting](https://docs.litellm.ai/docs/troubleshoot/ui_issues) - Get Started - Quickstart On this page # Getting Started ![](https://docs.litellm.ai/assets/ideal-img/hero.1927c06.640.png) LiteLLM is an open-source library that gives you a single, unified interface to call 100+ LLMs — OpenAI, Anthropic, Vertex AI, Bedrock, and more — using the OpenAI format. - Call any provider using the same `completion()` interface — no re-learning the API for each one - Consistent output format regardless of which provider or model you use - Built-in retry / fallback logic across multiple deployments via the [Router](https://docs.litellm.ai/docs/routing) - Self-hosted [LLM Gateway (Proxy)](https://docs.litellm.ai/docs/simple_proxy) with virtual keys, cost tracking, and an admin UI [![PyPI](https://img.shields.io/pypi/v/litellm.svg)](https://pypi.org/project/litellm/) [![GitHub Stars](https://img.shields.io/github/stars/BerriAI/litellm?style=social)](https://github.com/BerriAI/litellm) * ## Installation[](https://docs.litellm.ai/docs/#installation "Direct link to Installation") ``` pip install litellm ``` To run the full Proxy Server (LLM Gateway): ``` pip install 'litellm[proxy]' ``` * ## Quick Start[](https://docs.litellm.ai/docs/#quick-start "Direct link to Quick Start") Make your first LLM call using the provider of your choice: - OpenAI - Anthropic - Vertex AI - Bedrock - Ollama - Azure OpenAI ``` from litellm import completion import os os.environ["OPENAI_API_KEY"] = "your-api-key" response = completion( model="openai/gpt-4o", messages=[{"role": "user", "content": "Hello, how are you?"}] ) print(response.choices[0].message.content) ``` ``` from litellm import completion import os os.environ["ANTHROPIC_API_KEY"] = "your-api-key" response = completion( model="anthropic/claude-3-5-sonnet-20241022", messages=[{"role": "user", "content": "Hello, how are you?"}] ) print(response.choices[0].message.content) ``` ``` from litellm import completion import os # auth: run 'gcloud auth application-default login' os.environ["VERTEXAI_PROJECT"] = "your-project-id" os.environ["VERTEXAI_LOCATION"] = "us-central1" response = completion( model="vertex_ai/gemini-1.5-pro", messages=[{"role": "user", "content": "Hello, how are you?"}] ) print(response.choices[0].message.content) ``` ``` from litellm import completion import os os.environ["AWS_ACCESS_KEY_ID"] = "your-key" os.environ["AWS_SECRET_ACCESS_KEY"] = "your-secret" os.environ["AWS_REGION_NAME"] = "us-east-1" response = completion( model="bedrock/anthropic.claude-haiku-4-5-20251001:0", messages=[{"role": "user", "content": "Hello, how are you?"}] ) print(response.choices[0].message.content) ``` ``` from litellm import completion response = completion( model="ollama/llama3", messages=[{"role": "user", "content": "Hello, how are you?"}], api_base="http://localhost:11434" ) print(response.choices[0].message.content) ``` ``` from litellm import completion import os os.environ["AZURE_API_KEY"] = "your-key" os.environ["AZURE_API_BASE"] = "https://your-resource.openai.azure.com" os.environ["AZURE_API_VERSION"] = "2024-02-01" response = completion( model="azure/your-deployment-name", messages=[{"role": "user", "content": "Hello, how are you?"}] ) print(response.choices[0].message.content) ``` Every response follows the OpenAI Chat Completions format, regardless of provider. ✅ ### Response Format[](https://docs.litellm.ai/docs/#response-format "Direct link to Response Format") Non-streaming responses return a `ModelResponse` object: ``` { "id": "chatcmpl-abc123", "object": "chat.completion", "created": 1677858242, "model": "gpt-4o", "choices": [ { "index": 0, "message": { "role": "assistant", "content": "Hello! I'm doing well, thanks for asking." }, "finish_reason": "stop" } ], "usage": { "prompt_tokens": 13, "completion_tokens": 12, "total_tokens": 25 } } ``` Streaming responses (`stream=True`) yield `ModelResponseStream` chunks: ``` { "id": "chatcmpl-abc123", "object": "chat.completion.chunk", "created": 1677858242, "model": "gpt-4o", "choices": [ { "index": 0, "delta": { "role": "assistant", "content": "Hello" }, "finish_reason": null } ] } ``` 📖 [Full output format reference →](https://docs.litellm.ai/docs/completion/output) Open in Colab [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/BerriAI/litellm/blob/main/cookbook/liteLLM_Getting_Started.ipynb) * ## New to LiteLLM?[](https://docs.litellm.ai/docs/#new-to-litellm "Direct link to New to LiteLLM?") Want to get started fast? Head to [Tutorials](https://docs.litellm.ai/docs/tutorials) for step-by-step walkthroughs — AI coding tools, agent SDKs, proxy setup, and more. Need to understand a specific feature? Check [Guides](https://docs.litellm.ai/docs/guides) for streaming, function calling, prompt caching, and other how-tos. * ## Choose Your Path[](https://docs.litellm.ai/docs/#choose-your-path "Direct link to Choose Your Path") [🐍Python SDKIntegrate LiteLLM directly into your Python application. Drop-in replacement for the OpenAI client.completion(), embedding(), image\_generation() and more Router with retry, fallback, and load balancing OpenAI-compatible exceptions across all providers Observability callbacks (Langfuse, MLflow, Helicone…)](https://docs.litellm.ai/docs/#litellm-python-sdk) [🖥️Proxy Server (LLM Gateway)Self-hosted gateway for platform teams managing LLM access across an organization.Virtual keys with per-key/team/user budgets Centralized logging, guardrails, and caching Admin UI for monitoring and management Drop-in replacement for any OpenAI-compatible client](https://docs.litellm.ai/docs/#litellm-proxy-server-llm-gateway) * ## LiteLLM Python SDK[](https://docs.litellm.ai/docs/#litellm-python-sdk "Direct link to LiteLLM Python SDK") ### Streaming[](https://docs.litellm.ai/docs/#streaming "Direct link to Streaming") Add `stream=True` to receive chunks as they are generated: ``` from litellm import completion import os os.environ["OPENAI_API_KEY"] = "your-api-key" for chunk in completion( model="openai/gpt-4o", messages=[{"role": "user", "content": "Write a short poem"}], stream=True, ): print(chunk.choices[0].delta.content or "", end="") ``` ### Exception Handling[](https://docs.litellm.ai/docs/#exception-handling "Direct link to Exception Handling") LiteLLM maps every provider's errors to the OpenAI exception types — your existing error handling works out of the box: ``` import litellm try: litellm.completion( model="anthropic/claude-instant-1", messages=[{"role": "user", "content": "Hey!"}] ) except litellm.AuthenticationError as e: print(f"Bad API key: {e}") except litellm.RateLimitError as e: print(f"Rate limited: {e}") except litellm.APIError as e: print(f"API error: {e}") ``` ### Logging & Observability[](https://docs.litellm.ai/docs/#logging--observability "Direct link to Logging & Observability") Send input/output to Langfuse, MLflow, Helicone, Lunary, and more with a single line: ``` import litellm litellm.success_callback = ["langfuse", "mlflow", "helicone"] response = litellm.completion( model="gpt-4o", messages=[{"role": "user", "content": "Hi!"}] ) ``` 📖 [See all observability integrations →](https://docs.litellm.ai/docs/observability/agentops_integration) ### Track Costs & Usage[](https://docs.litellm.ai/docs/#track-costs--usage "Direct link to Track Costs & Usage") Use a callback to capture cost per response: ``` import litellm def track_cost(kwargs, completion_response, start_time, end_time): print("Cost:", kwargs.get("response_cost", 0)) litellm.success_callback = [track_cost] litellm.completion( model="gpt-4o", messages=[{"role": "user", "content": "Hello!"}], stream=True ) ``` 📖 [Custom callback docs →](https://docs.litellm.ai/docs/observability/custom_callback) * ## LiteLLM Proxy Server (LLM Gateway)[](https://docs.litellm.ai/docs/#litellm-proxy-server-llm-gateway "Direct link to LiteLLM Proxy Server (LLM Gateway)") The proxy is a self-hosted OpenAI-compatible gateway. Any client that works with OpenAI works with the proxy — no code changes needed. ![LiteLLM Proxy Dashboard](https://github.com/BerriAI/litellm/assets/29436595/47c97d5e-b9be-4839-b28c-43d7f4f10033) #### Step 1 — Start the proxy[](https://docs.litellm.ai/docs/#step-1--start-the-proxy "Direct link to Step 1 — Start the proxy") - pip - Docker ``` litellm --model huggingface/bigcode/starcoder # Proxy running on http://0.0.0.0:4000 ``` litellm\_config.yaml ``` model_list: - model_name: gpt-3.5-turbo litellm_params: model: azure/your-deployment api_base: os.environ/AZURE_API_BASE api_key: os.environ/AZURE_API_KEY api_version: "2023-07-01-preview" ``` ``` docker run \ -v $(pwd)/litellm_config.yaml:/app/config.yaml \ -e AZURE_API_KEY=your-key \ -e AZURE_API_BASE=https://your-resource.openai.azure.com/ \ -p 4000:4000 \ docker.litellm.ai/berriai/litellm:main-latest \ --config /app/config.yaml --detailed_debug ``` #### Step 2 — Call it with the OpenAI client[](https://docs.litellm.ai/docs/#step-2--call-it-with-the-openai-client "Direct link to Step 2 — Call it with the OpenAI client") ``` import openai client = openai.OpenAI(api_key="anything", base_url="http://0.0.0.0:4000") response = client.chat.completions.create( model="gpt-3.5-turbo", messages=[{"role": "user", "content": "Write a short poem"}] ) print(response.choices[0].message.content) ``` 👉 [Full proxy quickstart with Docker →](https://docs.litellm.ai/docs/proxy/docker_quick_start) Debugging tool Use [`/utils/transform_request`](https://docs.litellm.ai/docs/utils/transform_request) to inspect exactly what LiteLLM sends to any provider — useful for debugging prompt formatting, header issues, and provider-specific parameters. 🔗 [Interactive API explorer (Swagger) →](https://litellm-api.up.railway.app/) * ## Agent & MCP Gateway[](https://docs.litellm.ai/docs/#agent--mcp-gateway "Direct link to Agent & MCP Gateway") LiteLLM is a unified gateway for LLMs, agents, and MCP — you don't need a separate agent or MCP gateway. One endpoint for 100+ models, A2A agents, and MCP tools. [🔗A2A AgentsAdd and invoke A2A agents via the LiteLLM gateway.](https://docs.litellm.ai/docs/a2a) [🛠️MCP GatewayCentral MCP endpoint with per-key access control.](https://docs.litellm.ai/docs/mcp) * ## What to Explore Next[](https://docs.litellm.ai/docs/#what-to-explore-next "Direct link to What to Explore Next") [🔀Routing & Load BalancingLoad balance across deployments and set automatic fallbacks.](https://docs.litellm.ai/docs/routing-load-balancing) [🔑Virtual KeysManage access, budgets, and rate limits per team or user.](https://docs.litellm.ai/docs/proxy/virtual_keys) [📊Spend TrackingTrack costs per key, team, and user across all providers.](https://docs.litellm.ai/docs/proxy/cost_tracking) [🛡️GuardrailsAdd content filtering, PII masking, and safety checks.](https://docs.litellm.ai/docs/proxy/guardrails/quick_start) [📡ObservabilityIntegrate with Langfuse, MLflow, Helicone, and more.](https://docs.litellm.ai/docs/observability/agentops_integration) [🏭EnterpriseSSO/SAML, audit logs, and advanced security for production.](https://docs.litellm.ai/docs/enterprise) [Nextcompletion()](https://docs.litellm.ai/docs/completion/input) - [Installation](https://docs.litellm.ai/docs/#installation) - [Quick Start](https://docs.litellm.ai/docs/#quick-start) - [Response Format](https://docs.litellm.ai/docs/#response-format) - [New to LiteLLM?](https://docs.litellm.ai/docs/#new-to-litellm) - [Choose Your Path](https://docs.litellm.ai/docs/#choose-your-path) - [LiteLLM Python SDK](https://docs.litellm.ai/docs/#litellm-python-sdk) - [Streaming](https://docs.litellm.ai/docs/#streaming) - [Exception Handling](https://docs.litellm.ai/docs/#exception-handling) - [Logging & Observability](https://docs.litellm.ai/docs/#logging--observability) - [Track Costs & Usage](https://docs.litellm.ai/docs/#track-costs--usage) - [LiteLLM Proxy Server (LLM Gateway)](https://docs.litellm.ai/docs/#litellm-proxy-server-llm-gateway) - [Agent & MCP Gateway](https://docs.litellm.ai/docs/#agent--mcp-gateway) - [What to Explore Next](https://docs.litellm.ai/docs/#what-to-explore-next) 🚅 LiteLLM Enterprise SSO/SAML, audit logs, spend tracking, multi-team management, and guardrails — built for production. [Learn more →](https://docs.litellm.ai/docs/enterprise) Docs - [Getting Started](https://docs.litellm.ai/docs/) Community - [Discord](https://discord.com/invite/wuPM9dRgDw) - [Twitter](https://twitter.com/LiteLLM) More - [GitHub](https://github.com/BerriAI/litellm/) Copyright © 2026 liteLLM
Readable Markdown	LiteLLM is an open-source library that gives you a single, unified interface to call 100+ LLMs — OpenAI, Anthropic, Vertex AI, Bedrock, and more — using the OpenAI format. - Call any provider using the same `completion()` interface — no re-learning the API for each one - Consistent output format regardless of which provider or model you use - Built-in retry / fallback logic across multiple deployments via the [Router](https://docs.litellm.ai/docs/routing) - Self-hosted [LLM Gateway (Proxy)](https://docs.litellm.ai/docs/simple_proxy) with virtual keys, cost tracking, and an admin UI [![PyPI](https://img.shields.io/pypi/v/litellm.svg)](https://pypi.org/project/litellm/) [![GitHub Stars](https://img.shields.io/github/stars/BerriAI/litellm?style=social)](https://github.com/BerriAI/litellm) * ## Installation[](https://docs.litellm.ai/docs/#installation "Direct link to Installation") ``` pip install litellm ``` To run the full Proxy Server (LLM Gateway): ``` pip install 'litellm[proxy]' ``` * ## Quick Start[](https://docs.litellm.ai/docs/#quick-start "Direct link to Quick Start") Make your first LLM call using the provider of your choice: - OpenAI - Anthropic - Vertex AI - Bedrock - Ollama - Azure OpenAI ``` from litellm import completion import os os.environ["OPENAI_API_KEY"] = "your-api-key" response = completion( model="openai/gpt-4o", messages=[{"role": "user", "content": "Hello, how are you?"}] ) print(response.choices[0].message.content) ``` Every response follows the OpenAI Chat Completions format, regardless of provider. ✅ ### Response Format[](https://docs.litellm.ai/docs/#response-format "Direct link to Response Format") Non-streaming responses return a `ModelResponse` object: ``` { "id": "chatcmpl-abc123", "object": "chat.completion", "created": 1677858242, "model": "gpt-4o", "choices": [ { "index": 0, "message": { "role": "assistant", "content": "Hello! I'm doing well, thanks for asking." }, "finish_reason": "stop" } ], "usage": { "prompt_tokens": 13, "completion_tokens": 12, "total_tokens": 25 } } ``` Streaming responses (`stream=True`) yield `ModelResponseStream` chunks: ``` { "id": "chatcmpl-abc123", "object": "chat.completion.chunk", "created": 1677858242, "model": "gpt-4o", "choices": [ { "index": 0, "delta": { "role": "assistant", "content": "Hello" }, "finish_reason": null } ] } ``` 📖 [Full output format reference →](https://docs.litellm.ai/docs/completion/output) Open in Colab [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/BerriAI/litellm/blob/main/cookbook/liteLLM_Getting_Started.ipynb) * ## New to LiteLLM?[](https://docs.litellm.ai/docs/#new-to-litellm "Direct link to New to LiteLLM?") Want to get started fast? Head to [Tutorials](https://docs.litellm.ai/docs/tutorials) for step-by-step walkthroughs — AI coding tools, agent SDKs, proxy setup, and more. Need to understand a specific feature? Check [Guides](https://docs.litellm.ai/docs/guides) for streaming, function calling, prompt caching, and other how-tos. * ## Choose Your Path[](https://docs.litellm.ai/docs/#choose-your-path "Direct link to Choose Your Path") * ## LiteLLM Python SDK[](https://docs.litellm.ai/docs/#litellm-python-sdk "Direct link to LiteLLM Python SDK") ### Streaming[](https://docs.litellm.ai/docs/#streaming "Direct link to Streaming") Add `stream=True` to receive chunks as they are generated: ``` from litellm import completion import os os.environ["OPENAI_API_KEY"] = "your-api-key" for chunk in completion( model="openai/gpt-4o", messages=[{"role": "user", "content": "Write a short poem"}], stream=True, ): print(chunk.choices[0].delta.content or "", end="") ``` ### Exception Handling[](https://docs.litellm.ai/docs/#exception-handling "Direct link to Exception Handling") LiteLLM maps every provider's errors to the OpenAI exception types — your existing error handling works out of the box: ``` import litellm try: litellm.completion( model="anthropic/claude-instant-1", messages=[{"role": "user", "content": "Hey!"}] ) except litellm.AuthenticationError as e: print(f"Bad API key: {e}") except litellm.RateLimitError as e: print(f"Rate limited: {e}") except litellm.APIError as e: print(f"API error: {e}") ``` ### Logging & Observability[](https://docs.litellm.ai/docs/#logging--observability "Direct link to Logging & Observability") Send input/output to Langfuse, MLflow, Helicone, Lunary, and more with a single line: ``` import litellm litellm.success_callback = ["langfuse", "mlflow", "helicone"] response = litellm.completion( model="gpt-4o", messages=[{"role": "user", "content": "Hi!"}] ) ``` 📖 [See all observability integrations →](https://docs.litellm.ai/docs/observability/agentops_integration) ### Track Costs & Usage[](https://docs.litellm.ai/docs/#track-costs--usage "Direct link to Track Costs & Usage") Use a callback to capture cost per response: ``` import litellm def track_cost(kwargs, completion_response, start_time, end_time): print("Cost:", kwargs.get("response_cost", 0)) litellm.success_callback = [track_cost] litellm.completion( model="gpt-4o", messages=[{"role": "user", "content": "Hello!"}], stream=True ) ``` 📖 [Custom callback docs →](https://docs.litellm.ai/docs/observability/custom_callback) * ## LiteLLM Proxy Server (LLM Gateway)[](https://docs.litellm.ai/docs/#litellm-proxy-server-llm-gateway "Direct link to LiteLLM Proxy Server (LLM Gateway)") The proxy is a self-hosted OpenAI-compatible gateway. Any client that works with OpenAI works with the proxy — no code changes needed. ![LiteLLM Proxy Dashboard](https://github.com/BerriAI/litellm/assets/29436595/47c97d5e-b9be-4839-b28c-43d7f4f10033) #### Step 1 — Start the proxy[](https://docs.litellm.ai/docs/#step-1--start-the-proxy "Direct link to Step 1 — Start the proxy") - pip - Docker ``` litellm --model huggingface/bigcode/starcoder # Proxy running on http://0.0.0.0:4000 ``` #### Step 2 — Call it with the OpenAI client[](https://docs.litellm.ai/docs/#step-2--call-it-with-the-openai-client "Direct link to Step 2 — Call it with the OpenAI client") ``` import openai client = openai.OpenAI(api_key="anything", base_url="http://0.0.0.0:4000") response = client.chat.completions.create( model="gpt-3.5-turbo", messages=[{"role": "user", "content": "Write a short poem"}] ) print(response.choices[0].message.content) ``` 👉 [Full proxy quickstart with Docker →](https://docs.litellm.ai/docs/proxy/docker_quick_start) Debugging tool Use [`/utils/transform_request`](https://docs.litellm.ai/docs/utils/transform_request) to inspect exactly what LiteLLM sends to any provider — useful for debugging prompt formatting, header issues, and provider-specific parameters. 🔗 [Interactive API explorer (Swagger) →](https://litellm-api.up.railway.app/) * ## Agent & MCP Gateway[](https://docs.litellm.ai/docs/#agent--mcp-gateway "Direct link to Agent & MCP Gateway") LiteLLM is a unified gateway for LLMs, agents, and MCP — you don't need a separate agent or MCP gateway. One endpoint for 100+ models, A2A agents, and MCP tools. * ## What to Explore Next[](https://docs.litellm.ai/docs/#what-to-explore-next "Direct link to What to Explore Next") [🔀Routing & Load BalancingLoad balance across deployments and set automatic fallbacks.](https://docs.litellm.ai/docs/routing-load-balancing) [🔑Virtual KeysManage access, budgets, and rate limits per team or user.](https://docs.litellm.ai/docs/proxy/virtual_keys) [📊Spend TrackingTrack costs per key, team, and user across all providers.](https://docs.litellm.ai/docs/proxy/cost_tracking) [🛡️GuardrailsAdd content filtering, PII masking, and safety checks.](https://docs.litellm.ai/docs/proxy/guardrails/quick_start) [📡ObservabilityIntegrate with Langfuse, MLflow, Helicone, and more.](https://docs.litellm.ai/docs/observability/agentops_integration) [🏭EnterpriseSSO/SAML, audit logs, and advanced security for production.](https://docs.litellm.ai/docs/enterprise)
Shard	11 (laksa)
Root Hash	4050687676692930411
Unparsed URL	ai,litellm!docs,/docs/ s443