Operate

Budgets & governance

Policy that acts before inference. Cost caps, rate limits, API-key attribution, and application kill switches sit in the data path. They stay effective no matter which provider or model a developer requests.

Budgets & caps

A cap limits spend over a rolling window. Each cap has a dimension, a period, a limit, and an action:

Downgrade and block are enforced in the data path, before routing. A breached block cap returns 429 and outranks even an explicit model pin. Gateway enforcement covers global, application, and provider caps, and other dimensions are tracked for visibility. Manage caps from the dashboard or the admin API:

curl -X POST http://localhost:4100/api/caps \
  -H 'Authorization: Bearer $ARBR_ADMIN_KEY' \
  -H 'Content-Type: application/json' \
  -d '{ "dimension": "application", "value": "support-chat",
        "period": "month", "limit": 500, "action": "downgrade" }'

Gateway API keys

Data-plane keys (prefix ab_) authenticate application traffic. Each key:

• binds attribution, so the application and department tied to the key override whatever a request body claims
• can enforce a per-key requests-per-minute rate limit
• is shown once at creation, then stored only as a SHA-256 hash

Turn on Require API keys to reject anonymous gateway calls. Keys support expiry and one-click rotation.

Admin access

The dashboard and admin API are separate from the data plane. Choose how administrators authenticate with ARBR_AUTH_MODE:

Under OIDC and trusted-header, each admin action is attributed to a real person. Roles are viewer, operator, and administrator. Every mutation is written to an immutable audit log.

Recommendations & evidence

Arbr's recommendation engine finds where a premium model is repeatedly handling a cheap task, scoped to the application whose traffic justifies the change. A recommendation is a costed suggestion, not an action. Accepting one is quality-gated:

1. The engine surfaces a candidate substitution from your own traffic.
2. You freeze a representative sample into an evaluation dataset.
3. The candidate is replayed against the sample and judged before anything changes.
4. On approval, Arbr creates a disabled routing rule. You enable it deliberately, and can disable it in seconds.

Each accepted change records how it was approved (acceptedVia) and whether quality was proven (qualityGate: passed, overridden, or ungated), so a downgrade rolled out on evidence is easy to tell apart from one waved through.

Application kill switch & privacy

Each application has a kill switch. Flip it and that application's gateway traffic is rejected immediately, without touching anyone else. On the data side, request payload capture is off by default in production, PII masking is applied at log time, and request records are purged on a configurable retention window (30 days by default).