Lituano · Edge Rewrite

🚅 LiteLLM
Developer	BerriAI
Release	2023
Written in	Python and TypeScript
Type	LLM gateway
License	MIT License
Website	www.litellm.ai
Repository	https://github.com/BerriAI/litellm

LiteLLM is an open-source large language model (LLM) gateway developed by BerriAI. It provides a unified OpenAI-compatible application programming interface (API) for accessing and routing requests to different model providers. LiteLLM can be deployed as a self-hosted proxy server or used as a Python library within applications.

Functionality

[edit]

LiteLLM provides a common interface for models and services from providers including OpenAI, Anthropic, Google Gemini, Amazon Bedrock, Microsoft Azure, and Hugging Face. It translates requests and responses between provider APIs into a consistent OpenAI-compatible format, allowing applications to switch between models without separately implementing each provider's interface.^[1]^[2]

The software also provides model routing, load balancing, fallbacks, authentication, rate limiting, usage monitoring, and cost controls. Its proxy server can act as a centralized API gateway through which multiple applications or users access AI models.^[2]

LiteLLM also supports the Model Context Protocol (MCP), an open protocol developed by Anthropic for connecting AI applications to external tools and data sources.^[3]

Security

[edit]

In March 2026, malicious versions of LiteLLM were distributed through the Python Package Index (PyPI) as part of a software supply-chain attack. TechCrunch reported that AI recruiting company Mercor was among the organizations affected by the compromise.^[4]

Compatibility

[edit]

Supported models

[edit]

LiteLLM supports more than 100 large language models through a common OpenAI-compatible interface. Notable supported model families include:^[2]

Compatible AI coding software

[edit]

LiteLLM can be used as an inference gateway for AI-assisted software development tools, routing their model requests through the LiteLLM proxy. Documented integrations include:^[5]

References

[edit]

↑ MSV, Janakiram (May 15, 2025). "LiteLLM: An open-source gateway for unified LLM access". InfoWorld. Retrieved September 9, 2026.
1 2 3 "LiteLLM - Getting Started". LiteLLM. BerriAI. Retrieved September 9, 2026.
↑ "Introducing the Model Context Protocol". Anthropic. November 25, 2024. Retrieved September 9, 2026.
↑ Singh, Jagmeet (March 31, 2026). "Mercor says it was hit by cyberattack tied to compromise of open source LiteLLM project". TechCrunch. Retrieved September 9, 2026.
↑ "LiteLLM Proxy CLI". GitHub. BerriAI. Retrieved September 10, 2026.
↑ "Configure LiteLLM Provider". Cline. Retrieved September 10, 2026.

External links

[edit]

Official website
LiteLLM on GitHub

v t e Large language models (LLMs)
List of LLMs AI Companies Benchmarks List of chatbots Foundation model Generative AI
Concepts	Language model NLP NLG Computational linguistics Foundation model Small language model Reasoning model Generative pre-trained transformer Transformer Attention KV cache Context window Tokenization Word embedding Parameter Hyperparameter Autoregression Mixture of experts (MoE) Inference Model compression Knowledge distillation Speculative decoding PagedAttention Neural scaling law Multimodality Open weights
Training, prompting, and alignment	Self-supervised learning Supervised learning Fine-tuning Instruction tuning RLHF Constitutional AI AI alignment AI safety Mechanistic interpretability Prompt engineering In-context learning Chain-of-thought prompting RAG Prompt injection Adversarial machine learning Hallucination Stochastic parrot Glitch token Generative engine optimization
Models	Apertus BERT BLOOM Chinchilla Claude DBRX DeepSeek-LLM Gemini Gemma GloVe GLM GPT GPT-J PanGu Granite Hy Inkling Jais LaMDA Laguna S Llama MiMo Minerva Mistral and Mixtral Muse Spark Muse Glimmer Nemotron PaLM Phi Qwen Seq2seq T5 Vicuna Word2vec XLNet
Chatbots and assistants	Amazon Q ChatGPT Character.ai Claude Claude Code DeepSeek Doubao Ernie Bot Gemini Grok Kimi Kimi Code Lumo Microsoft Copilot Meta AI Perplexity AI Sparrow You.com
Agents, coding, and applications	AI agent Intelligent agent AutoGPT CrewAI LangChain Manus Model Context Protocol Agent2Agent OpenAI Codex Vibe coding Code generation Question answering Machine translation Text summarization Chatbot Virtual assistant
Software	PyTorch TensorFlow Hugging Face llama.cpp LM Studio Ollama SGLang TensorRT-LLM vLLM ONNX OpenVINO Vector database ChromaDB Deep learning software Open-source AI software
Hardware and infrastructure	AI data center CUDA GPU High Bandwidth Memory Neural processing unit TPU
Benchmarks, evaluation, and detection	Language model benchmark MMLU Humanity's Last Exam LMArena LLM-as-a-Judge Perplexity metric GPTZero Artificial intelligence content detection Undetectable.ai
Datasets and data	Data set Text corpus Common Crawl The Pile Web scraping Synthetic data Training, validation, and test data sets
Organizations	AI21 Labs Alibaba Anthropic Baidu Cohere DeepSeek EleutherAI Google DeepMind Hugging Face Meta AI Microsoft AI Mistral AI MiniMax Moonshot AI Nvidia OpenAI OpenRouter Sarvam AI StepFun Technology Innovation Institute Thinking Machines Lab xAI
People	Sam Altman Dario Amodei Yoshua Bengio Aidan Gomez Demis Hassabis Geoffrey Hinton Andrej Karpathy Yann LeCun Percy Liang Liang Wenfeng Christopher D. Manning Arthur Mensch Mira Murati Alec Radford Noam Shazeer Ilya Sutskever Ashish Vaswani Andrew Ng
Social, economic, and governance	AI boom AI bubble AI slop AI anthropomorphism AI arms race Chatbot psychosis Competition Copyright Deaths linked to chatbots Dependency (GAID) Environmental impact Regulation Ethics Existential risk In education In healthcare Workplace impact
Category:Large language models

v t e AI-assisted software development
Concepts and techniques	Automated program repair Automatic programming Code generation Code review Intelligent code completion Program synthesis AI-assisted testing Static program analysis Vibe coding
Coding assistants	Amazon Q Developer ChatGPT Cody Google Gemini GitHub Copilot Qodo Tabnine
Development environments	Cursor Google AI Studio Google Antigravity Zed
Software development agents	Amp Claude Code Cline Codex Codex CLI Devin goose Grok Build Hermes Agent Kilo Code Kimi Code Muse Code Oh My Py OpenCode OpenHands Perplexity Computer Pi Replit Agent
Models and model families	Claude DeepSeek Coder Gemini GLM IBM Granite Kimi Nemotron Qwen Tencent Hy
Inference software	Llama.cpp LM Studio Ollama vLLM
Frameworks	CrewAI Theia AI LangChain
Agent harness List of AI-assisted software development tools List of large language models OpenRouter

Generative AI

Concepts

Models

Text	Amazon Nova Character.ai Claude Command DeepSeek Doubao Ernie EXAONE Gemini Gemma GLM GPT ChatGPT Grok IBM Granite Kimi MAI Microsoft Copilot Mistral MiniMax Muse Spark Nemotron Perplexity Solar Poe HKChat Qwen Tencent Hy Xiaomi MiMo You.com
Image	Adobe Firefly Flux GPT Image Ideogram Midjourney Nano Banana Recraft Seedream Stable Diffusion
Video	Dream Machine Genie world model MiniMax H3 Kling AI LTX Runway Gen Seedance Sora Veo
Speech	15.ai Eleven Gemini Speech MiniMax Speech
Music	Eleven Music Endel MiniMax Music Riffusion Suno Udio

Products

Coding tools	Claude Code Codex Cursor Devin AI GitHub Copilot Google Antigravity Replit
Agents	AutoGPT ChatGPT agent Claude Cowork Gemini Spark Manus MiniMax Agent OpenClaw

Applications

Companies

Controversies

Functionality

Security

Compatibility

Supported models

Compatible AI coding software

See also

References

External links