LiteLLM Integration Guide

This guide will walk you through integrating LiteLLM with APIpie, enabling you to access a wide range of language models through a unified OpenAI-compatible interface.

What is LiteLLM?

LiteLLM is a powerful library and proxy service that provides a unified interface for working with multiple LLM providers including:

Unified API Interface: Call any LLM API using the familiar OpenAI format
Consistent Response Format: Get standardized responses regardless of the underlying provider
Fallback Routing: Configure backup models in case of API failures or rate limits
Cost Tracking & Budgeting: Monitor spend across providers and set limits
Proxy Server: Deploy a gateway to manage all your LLM API calls

By integrating LiteLLM with APIpie, you can leverage APIpie's powerful models while maintaining compatibility with other providers through a single consistent interface.

Integration Steps

1. Create an APIpie Account

Register here: APIpie Registration
Complete the sign-up process.

2. Add Credit

Add Credit: APIpie Subscription
Add credits to your account to enable API access.

3. Generate an API Key

API Key Management: APIpie API Keys
Create a new API key for use with LiteLLM.

4. Install LiteLLM

pip install litellm

For the proxy server:

pip install 'litellm[proxy]'

5. Configure LiteLLM for APIpie

There are two main ways to use LiteLLM with APIpie:

A. Direct Integration (Python Library)

import os
import litellm
from litellm import completion

# Set your APIpie API key
os.environ["APIPIE_API_KEY"] = "your-apipie-api-key"

# Use APIpie models with LiteLLM
response = completion(
    model="apipie/gpt-4o-mini",  # Use APIpie's gpt-4o-mini model
    messages=[{"role": "user", "content": "Hello, how are you?"}]
)

print(response)

B. Configure LiteLLM Proxy with APIpie

Create a config.yaml file:

model_list:
  - model_name: gpt-4
    litellm_params:
      model: apipie/gpt-4
      api_key: your-apipie-api-key
      api_base: "https://apipie.ai/v1"
  
  - model_name: apipie/claude-3-sonnet
    litellm_params:
      model: claude-3-sonnet-20240229
      api_key: your-apipie-api-key
      api_base: "https://apipie.ai/v1"
  
  - model_name: apipie/*
    litellm_params:
      model: apipie/*
      api_key: your-apipie-api-key
      api_base: "https://apipie.ai/v1"

Then start the proxy:

litellm --config config.yaml

Key Features

Provider-Agnostic Interface: Switch between models from different providers without changing your code
Fallback Routing: Configure fallback models if your primary model fails
Consistent Response Format: Get standardized responses in OpenAI format
Cost Management: Track and limit spending across different providers
Streaming Support: Stream responses from all supported models
Logging & Monitoring: Track usage, errors, and performance

Example Workflows

Application Type	What LiteLLM Helps You Build
Multi-Provider Applications	Apps that can use any LLM provider through a single API
High-Reliability Services	Systems with automatic fallback to backup models
Cost-Optimized Applications	Apps that route to the most cost-efficient provider
Enterprise LLM Gateways	Central API gateways with access controls and monitoring
Multi-Model Agents	Agents that use specialized models for different tasks

Using LiteLLM with APIpie

Basic Chat Completion

import os
from litellm import completion

# Set your APIpie API key
os.environ["APIPIE_API_KEY"] = "your-apipie-api-key"

# Use any APIpie model
response = completion(
    model="apipie/gpt-4o",  # APIpie's GPT-4o
    messages=[
        {"role": "system", "content": "You are a helpful assistant."},
        {"role": "user", "content": "Write a short poem about AI."}
    ]
)

print(response.choices[0].message.content)

Streaming Responses

import os
from litellm import completion

# Set your APIpie API key
os.environ["APIPIE_API_KEY"] = "your-apipie-api-key"

# Stream the response
response = completion(
    model="apipie/gpt-4o",
    messages=[{"role": "user", "content": "Write a short story about space exploration."}],
    stream=True
)

# Process the streaming response
for chunk in response:
    content = chunk.choices[0].delta.content
    if content:
        print(content, end="", flush=True)

Model Fallbacks with Router

import os
from litellm import Router

# Set your APIpie API key
os.environ["APIPIE_API_KEY"] = "your-apipie-api-key"

# Configure models with fallbacks
model_list = [
    {
        "model_name": "gpt-4",
        "litellm_params": {
            "model": "apipie/gpt-4",
            "api_key": os.environ["APIPIE_API_KEY"],
            "api_base": "https://apipie.ai/v1"
        }
    },
    {
        "model_name": "apipie/claude-alternative",
        "litellm_params": {
            "model": "apipie/claude-3-sonnet-20240229",
            "api_key": os.environ["APIPIE_API_KEY"],
            "api_base": "https://apipie.ai/v1"
        }
    },
    {
        "model_name": "apipie/*",
        "litellm_params": {
            "model": "apipie/*",
            "api_key": os.environ["APIPIE_API_KEY"],
            "api_base": "https://apipie.ai/v1"
        }
    }
]

# Initialize the router with fallback options
router = Router(
    model_list=model_list,
    fallbacks=[
        {"gpt-4": ["claude-alternative"]}  # If gpt-4 fails, try claude
    ]
)

# Use the router for completions
response = router.completion(
    model="gpt-4",
    messages=[{"role": "user", "content": "Explain quantum computing in simple terms."}]
)

print(response.choices[0].message.content)

Async Completions

import os
import asyncio
from litellm import acompletion

# Set your APIpie API key
os.environ["APIPIE_API_KEY"] = "your-apipie-api-key"

async def get_completion():
    response = await acompletion(
        model="apipie/gpt-4o",
        messages=[{"role": "user", "content": "What are the benefits of renewable energy?"}]
    )
    return response.choices[0].message.content

# Run the async function
result = asyncio.run(get_completion())
print(result)

Using Embeddings

import os
from litellm import embedding

# Set your APIpie API key
os.environ["APIPIE_API_KEY"] = "your-apipie-api-key"

# Generate embeddings
response = embedding(
    model="apipie/text-embedding-3-large",
    input=["Renewable energy is the future of sustainable living."]
)

print(f"Embedding dimension: {len(response.data[0].embedding)}")
print(f"First few values: {response.data[0].embedding[:5]}")

Setting up the LiteLLM Proxy with APIpie

The LiteLLM proxy serves as a gateway between your applications and various LLM providers, including APIpie.

Step 1: Create a Configuration File

Create a config.yaml file:

model_list:
  - model_name: gpt-4
    litellm_params:
      model: apipie/gpt-4
      api_key: your-apipie-api-key
      api_base: "https://apipie.ai/v1"
  
  - model_name: apipie/claude-3-sonnet
    litellm_params:
      model: claude-3-sonnet-20240229
      api_key: your-apipie-api-key
      api_base: "https://apipie.ai/v1"
  
  - model_name: apipie/*
    litellm_params:
      model: apipie/*
      api_key: your-apipie-api-key
      api_base: "https://apipie.ai/v1"

# Optional configurations
router_settings:
  routing_strategy: "simple-shuffle"  # or "usage-based", "latency-based"

# Set up the proxy server
litellm_settings:
  success_callback: ["prometheus"]  # For metrics tracking
  drop_params: true  # Drop unsupported parameters

# Set up API keys for proxy users (optional)
api_keys:
  - key: "sk-1234"
    aliases: ["team-1"]
    metadata:
      team: "research"
    spend: 0
    max_budget: 100  # $100 budget

Step 2: Start the Proxy Server

litellm --config config.yaml

The proxy will start on http://localhost:4000 by default.

Step 3: Use the Proxy in Your Applications

import openai

client = openai.OpenAI(
    api_key="sk-1234",  # Your proxy API key
    base_url="http://localhost:4000/v1"  # Your proxy URL
)

response = client.chat.completions.create(
    model="gpt-4",  # This will be routed to APIpie/gpt-4 based on your config
    messages=[{"role": "user", "content": "Explain the theory of relativity."}]
)

print(response.choices[0].message.content)

Troubleshooting & FAQ

How do I handle environment variables securely?
Store your API keys in environment variables and avoid hardcoding them in your code. For production, use a secure secret management system.
How can I monitor costs across providers?
LiteLLM's proxy server includes spend tracking functionality. You can also integrate with observability tools like Langfuse or Helicone.
Can I set rate limits on API usage?
Yes, the LiteLLM proxy allows setting rate limits per user, team, or model.
Does LiteLLM support function calling or tool use?
Yes, when using OpenAI-compatible models through APIpie that support function calling, those capabilities are preserved.
How do I handle provider-specific parameters?
Use the litellm_params field in your configuration to specify provider-specific parameters.
Can I deploy the proxy on cloud platforms?
Yes, the LiteLLM proxy can be deployed on platforms like Render, Railway, AWS, GCP, or Azure.

For more information, see the LiteLLM documentation or the GitHub repository.

Support

If you encounter any issues during the integration process, please reach out on APIpie Discord or LiteLLM Discord for assistance.

What is LiteLLM?​

Integration Steps​

1. Create an APIpie Account​

2. Add Credit​

3. Generate an API Key​

4. Install LiteLLM​

5. Configure LiteLLM for APIpie​

A. Direct Integration (Python Library)​

B. Configure LiteLLM Proxy with APIpie​

Key Features​

Example Workflows​

Using LiteLLM with APIpie​

Basic Chat Completion​

Streaming Responses​

Model Fallbacks with Router​

Async Completions​

Using Embeddings​

Setting up the LiteLLM Proxy with APIpie​

Step 1: Create a Configuration File​

Step 2: Start the Proxy Server​

Step 3: Use the Proxy in Your Applications​

Troubleshooting & FAQ​

Support​