LiteLLM¶

In this article

• LiteLLM. Main Features
• Deployment Features
• Getting Started After LiteLLM Deployment
• Connection and Initial Setup
• Ordering a Server with LiteLLM via API

LiteLLM. Main Features¶

• Unified API (OpenAI Compatibility). To switch from gpt-4o to claude-3-5-sonnet or a local llama-3, you only need to change a single line (the model parameter) without rewriting your application logic.
• Massive Provider Ecosystem. Supports over 100 providers out of the box: OpenAI, Azure OpenAI, Anthropic, Google Gemini, AWS Bedrock, Cohere, HuggingFace, Ollama, vLLM, Groq, and many others.
• Cost and Budget Management (Cost Tracking). Built-in precise calculation of the cost of each request in dollars (or tokens). You can set strict spending limits (budgets) for individual users, API keys, or teams to avoid unexpected bills.
• Fault Tolerance (Fallbacks and Retries). If the primary API is unavailable, returns an error, or hits a rate limit, LiteLLM can automatically retry or seamlessly switch to a fallback model or a different provider without the user noticing.
• Load Balancing. Allows you to distribute requests among multiple API keys from the same provider or different deployments of the same model to effectively bypass rate limits.
• Observability and Logging. Seamless integration with popular monitoring and debugging tools (Langfuse, Prometheus, Datadog, Arize Phoenix, Helicone) to track latency, errors, and costs in real time.
• Asynchrony and Streaming. Full support for asynchronous calls (asyncio) and response streaming for all supported providers, which is critical for creating fast and responsive chat interfaces.

Deployment Features¶

Getting Started After LiteLLM Deployment¶

After payment, a notification regarding the server's readiness will be sent to the email address provided during registration. It will contain the VPS IP address, as well as the login and password for connection. Customers can manage their hardware in the server and API control panel — Invapi.

Authorization data can be found in the Config >> Tags tab of the server control panel or in the sent email:

• Web interface control panel link: in the webpanel tag;
• Login: admin;
• Password: for the administrator - sent via email to your email address upon server delivery;

Connection and Initial Setup¶

After clicking the link from the webpanel tag, the login window will open.

After entering your login and password, you will be redirected to the LiteLLM main management screen.

You can now start working with LiteLLM. Please note that adding additional users is only available after configuring the email delivery service.

Ordering a Server with LiteLLM via API¶

To install this software using the API, follow these instructions.