Documentation

The AI gateway and control plane

Route every model request through one endpoint. See where the money goes, prove a cheaper model holds up on your own traffic, then roll the change out with human approval and measure what you actually saved.

Arbr is a self-hosted model optimisation and governance control plane for teams running AI in production. Applications send their requests through one OpenAI-compatible gateway. Arbr logs and costs every call, points out where a premium model is doing a cheap model's job, checks candidate models against your real traffic, and keeps every routing change explicit, reversible, and measurable.

It runs on its own or in front of infrastructure you already operate, including LiteLLM. It's open source under the MIT license.

The optimisation lifecycle

Arbr is organised around a loop that turns request-level data into a verified outcome. Nothing reroutes silently, and every step leaves evidence.

The operating boundary

This is the rule that makes Arbr safe to put in the request path:

Quickstart

You don't need API keys to explore. Arbr ships with a demo mode that seeds realistic data, so every dashboard, the recommendation engine, and the routing controls work right away.

Option A: Docker (recommended)

git clone https://github.com/project-arbr/arbr-control-plane
cd arbr-control-plane
cp .env.example .env
docker compose up

Open http://localhost:4100 and the dashboard loads with seeded demo data. Go to Recommendations → Recompute to see the engine run.

Option B: Local (Node + MongoDB)

git clone https://github.com/project-arbr/arbr-control-plane
cd arbr-control-plane
npm run setup   # install deps + seed built-in models + demo request records
npm run dev     # server on :4100, dashboard on :5173

Add a provider key

The demo works with zero keys. To make real calls, open the Models page and paste a key for any provider. It goes live immediately, with no restart. You can also set it in the environment before starting:

# .env
OPENAI_API_KEY=sk-...

Environment variables take precedence over dashboard-stored keys. See Providers & models for the full list.

Make your first call

Once a provider is live, hit the gateway. Passing model: "auto" lets the router decide:

curl -X POST http://localhost:4100/v1/chat \
  -H 'Content-Type: application/json' \
  -d '{
    "application": "support-chat",
    "model": "auto",
    "messages": [{ "role": "user", "content": "Summarise: my card was declined." }]
  }'

The response tells you which model actually served the call and why. See Gateway.

Where Arbr fits

• In front of LiteLLM. Arbr is an upstream that connects to LiteLLM, not a replacement for it. You keep LiteLLM's provider breadth and add routing, evidence, budgets, and governance on top.
• As a standalone gateway. Point any OpenAI-compatible client at Arbr and it works unchanged. Change the base URL and keep the SDK.
• Under human control. Arbr never reroutes silently. A person approves every routing change and can undo it in seconds.
• Measuring real savings. Each record stores both the model requested and the model served, so savings are measured after the fact rather than estimated up front.

Next steps