Self-host Promptfoo on a VPS

Test and compare prompts, models and agents with a local web UI for browsing evaluation results. Runs the viewer with results stored on a volume.

levelrail-cli templates deploy promptfoo --name my-promptfoo

Specification

Status
Listed
Category
AI
Services
promptfoo
Ports
3000
Volumes
1
Memory
about 256 MiB
arm64
unknown
amd64
unknown
Settings to fill in
none
Listing
Community template

About Promptfoo

Test and compare prompts, models and agents with a local web UI for browsing evaluation results. Runs the viewer with results stored on a volume.

What it deploys

Promptfoo runs as one service in a single Compose app, listening on port 3000, with 1 named volume mount.

ServiceImageTagPortsVolumes
promptfoo ghcr.io/promptfoo/promptfoo 0.125.1 3000 data -> /home/promptfoo/.promptfoo

Before you install

  • CPU architecture. arm64 unknown. The scan could not read a manifest or the tag is chosen at install time. This is not a claim either way.
  • Listed. It is in the catalog. Back up the data volume before changing an image tag.

Settings and environment

The catalog lists no environment variables for this template.

First run

  1. Open the address to browse evals. Run evaluations from the CLI with this server as the share target.

Install Promptfoo on a VPS

  1. Install Levelrail on a fresh VPS

    Run this as root on a Linux server (amd64 or arm64) with ports 80 and 443 reachable. The installer prints a setup link.

    curl -fsSL https://levelrail.com/install.sh | sudo sh
  2. Sign in

    Open the setup link, choose a password, then log the CLI in. The CLI install command is curl -fsSL https://levelrail.com/install-cli.sh | sh if the machine does not have it. See Getting started.

    levelrail-cli login --device
  3. Read the template

    Shows the entry, including its full compose.yaml.

    levelrail-cli templates get promptfoo
  4. Deploy it

    This deploys the template compose file as an app. The app name defaults to the template id; --name sets another.

    levelrail-cli templates deploy promptfoo --name my-promptfoo

Without a domain the app gets an sslip.io address. You can also deploy it from the dashboard: Apps, New app, Browse templates, then Promptfoo.

More in AI

All AI templates