Self-host Promptfoo on a VPS
Test and compare prompts, models and agents with a local web UI for browsing evaluation results. Runs the viewer with results stored on a volume.
levelrail-cli templates deploy promptfoo --name my-promptfooSpecification
- Status
- Listed
- Category
- AI
- Services
- promptfoo
- Ports
- 3000
- Volumes
- 1
- Memory
- about 256 MiB
- arm64
- unknown
- amd64
- unknown
- Settings to fill in
- none
- License
- Permissive license, MIT
- Listing
- Community template
About Promptfoo
Test and compare prompts, models and agents with a local web UI for browsing evaluation results. Runs the viewer with results stored on a volume.
What it deploys
Promptfoo runs as one service in a single Compose app, listening on port 3000, with 1 named volume mount.
| Service | Image | Tag | Ports | Volumes |
|---|---|---|---|---|
| promptfoo | ghcr.io/promptfoo/promptfoo | 0.125.1 | 3000 | data -> /home/promptfoo/.promptfoo |
Before you install
- CPU architecture. arm64 unknown. The scan could not read a manifest or the tag is chosen at install time. This is not a claim either way.
- Listed. It is in the catalog. Back up the data volume before changing an image tag.
Settings and environment
The catalog lists no environment variables for this template.
First run
- Open the address to browse evals. Run evaluations from the CLI with this server as the share target.
Install Promptfoo on a VPS
-
Install Levelrail on a fresh VPS
Run this as root on a Linux server (amd64 or arm64) with ports 80 and 443 reachable. The installer prints a setup link.
curl -fsSL https://levelrail.com/install.sh | sudo sh -
Sign in
Open the setup link, choose a password, then log the CLI in. The CLI install command is
curl -fsSL https://levelrail.com/install-cli.sh | shif the machine does not have it. See Getting started.levelrail-cli login --device -
Read the template
Shows the entry, including its full compose.yaml.
levelrail-cli templates get promptfoo -
Deploy it
This deploys the template compose file as an app. The app name defaults to the template id; --name sets another.
levelrail-cli templates deploy promptfoo --name my-promptfoo
Without a domain the app gets an sslip.io address. You can also deploy it from the dashboard: Apps, New app, Browse templates, then Promptfoo.
Docs and upstream
More in AI
- Paperless-AIAdds automatic tagging, titles and document chat to a Paperless-ngx server using your choice of AI provider. MIT licensed. A setup wizard collects the Paperless address and token.
- SillyTavernA local chat front end for language models with characters, lorem books and extensions, bringing your own model API. AGPL-3.0 licensed. Sign in with the admin user and the generated password.
- OpenClawA self-hosted personal AI agent gateway that connects chat apps and tools to the model provider you choose. Recommended container limit is 2 GiB. Measured use is 0.9 to 1.3 GiB, so a Levelrail VM needs at least 4096 MiB. The gateway trusts the ingress proxy through gateway.trustedProxies, written once to openclaw.json in the data volume.
- Open WebUIA self-hosted chat interface for running local large language models through Ollama.
- Open NotebookAn open source, self-hosted alternative to NotebookLM for researching with your own sources, with podcasts and chat. Add model provider keys in the app after install.
- Ollama: Qwen 2.5 7BAlibaba's Qwen 2.5 7B model, served locally through Ollama's HTTP API.