Operate

Deployment

Arbr runs as one container on one port, with the gateway, admin API, and dashboard together, like a LiteLLM proxy or an MLflow tracking server. Put TLS in front of it and you're in production.

The deployment model

One standalone instance per organisation. Everything is served on a single port (default 4100): the gateway at /v1/*, the admin API at /api/*, the dashboard at /, and /health. State lives in MongoDB.

Docker Compose

The repository ships compose files for both a demo and a production profile. For production, layer the prod overlay:

docker compose -f docker-compose.yml -f docker-compose.prod.yml up -d

The production profile fails closed: with NODE_ENV=production it requires ARBR_ADMIN_KEY and ARBR_ENCRYPTION_KEY to be set, disables demo seeding, binds the app to loopback (so only your reverse proxy reaches it), and forces gateway API-key auth on.

Production checklist

• Set ARBR_ADMIN_KEY and ARBR_ENCRYPTION_KEY to strong random values.
• Set SEED_ON_BOOT=false (seeding wipes request records).
• Terminate TLS in front and keep port 4100 private.
• Choose an ARBR_AUTH_MODE. Use oidc or trusted-header for per-user accountability.

TLS & reverse proxy

Run nginx (with certbot) or an AWS ALB terminating 443 and proxying to 127.0.0.1:4100. Streaming needs buffering disabled on the SSE path:

location /v1/ {
    proxy_pass http://127.0.0.1:4100;
    proxy_http_version 1.1;
    proxy_set_header Connection "";
    proxy_buffering off;                 # required for SSE streaming
    proxy_read_timeout 300s;
    add_header X-Accel-Buffering no;
}

Behind an ALB, set the idle timeout to at least 300s so long streams aren't cut.

Deploy on GCP

The docs include a step-by-step guide for a single Compute Engine VM running the production Docker Compose profile. It walks through an e2-medium on Ubuntu, Docker with nginx and certbot, the app behind nginx with a Let's Encrypt certificate, and port 4100 blocked at the VPC firewall so only nginx is public. A gated image-based deploy (ops/deploy.sh) pulls a published, CI-green image, health-checks /health, and rolls back automatically on failure.

Configuration

Environment variables cover infrastructure and secrets. Runtime behaviour is managed in the dashboard and stored in MongoDB.

Key environment variables

Provider keys (OPENAI_API_KEY and friends) are covered under Providers. Routing mode, Require API keys, budgets, default model, rules, and the AI policy are runtime settings. They live in MongoDB and are managed in the dashboard, not in the environment.