Self-host LiteLLM Proxy on a VPS
One OpenAI-format gateway in front of 100+ model providers, with keys, budgets, and spend tracking.
levelrail-cli templates deploy litellm --name my-litellmSpecification
- Status
- Listed
- Category
- AI
- Services
- db, litellm
- Ports
- 4000
- Volumes
- 1
- Memory
- about 1 GiB
- arm64
- image published
- amd64
- image published
- Settings to fill in
- none
- License
- not yet recorded
About LiteLLM Proxy
One OpenAI-format gateway in front of 100+ model providers, with keys, budgets, and spend tracking. Runs 2 containers (db, litellm), with litellm listening on port 4000; data is kept in named volumes.
What it deploys
LiteLLM Proxy runs as 2 services in a single Compose app, listening on port 4000, with 1 named volume mount.
| Service | Image | Tag | Ports | Volumes |
|---|---|---|---|---|
| db | postgres | 16-alpine | none | litellm_db_data -> /var/lib/postgresql/data |
| litellm | ghcr.io/berriai/litellm | main-stable | 4000 | none |
Before you install
- CPU architecture. arm64 listed. Every image lists linux/arm64 in its registry manifest. That is a manifest fact, not proof the app runs on Arm.
- Listed. It is in the catalog. Back up the data volume before changing an image tag.
- Host ports. It publishes port 4000 directly on the host, so open them in the VPS firewall only if you want that.
Settings and environment
You do not have to fill in anything. 3 secrets are generated at deploy and stored encrypted. Values are never shown here.
| Service | Variable | Value |
|---|---|---|
| db | POSTGRES_DB | Preset by the template |
| db | POSTGRES_PASSWORD | Generated at deploy |
| db | POSTGRES_USER | Preset by the template |
| litellm | DATABASE_URL | Generated at deploy |
| litellm | LITELLM_MASTER_KEY | Generated at deploy |
| litellm | STORE_MODEL_IN_DB | Preset by the template |
Install LiteLLM Proxy on a VPS
-
Install Levelrail on a fresh VPS
Run this as root on a Linux server (amd64 or arm64) with ports 80 and 443 reachable. The installer prints a setup link.
curl -fsSL https://levelrail.com/install.sh | sudo sh -
Sign in
Open the setup link, choose a password, then log the CLI in. The CLI install command is
curl -fsSL https://levelrail.com/install-cli.sh | shif the machine does not have it. See Getting started.levelrail-cli login --device -
Read the template
Shows the entry, including its full compose.yaml.
levelrail-cli templates get litellm -
Deploy it
This deploys the template compose file as an app. The app name defaults to the template id; --name sets another.
levelrail-cli templates deploy litellm --name my-litellm
Without a domain the app gets an sslip.io address. You can also deploy it from the dashboard: Apps, New app, Browse templates, then LiteLLM Proxy.
Docs and upstream
More in AI
- llama.cpp ServerThe llama.cpp HTTP server: run quantized GGUF models on plain CPUs with an OpenAI-compatible API.
- Lobe ChatA modern, open-source AI chat framework supporting many model providers.
- LocalAIA drop-in OpenAI-compatible API that runs language, image, and speech models on your own hardware.
- LightRAGGraph based retrieval augmented generation server with a web UI and API. Runs on local file storage; point it at a model provider with the LLM and embedding settings.
- LibreChatA self-hosted ChatGPT-style interface for many model providers, with agents, search, and multi-user login.
- LangfuseTrace, evaluate, and debug LLM applications with prompt management and cost analytics.