Control every AI request.

Route each task to the right model. See where every token goes. Set the limits before costs, failures, or provider risk become operational problems.

Deploy ARBR ↗ See the integration

One OpenAI-compatible endpoint. Self-hosted. Provider-neutral.

POST /v1/chat/completions model:auto

CONNECT

Normalize access across every provider.

every major provider+ LiteLLM · self-hosted1 endpoint

SEE

Understand every request before routing.

taskTypedifficultycost

ROUTE

Choose the right model for the task.

ruleAI policyguardrail

GOVERN

Enforce limits before execution.

budgetrate limitkill switch

SELECTED PROVIDEROpenAI

gpt-4o-mini

184mslatency

$0.0024cost

63%saved

One control layer across the models your teams already use.

ANTHROPICOPENAIGEMINIBEDROCKDEEPSEEKGROQ

Platform capabilities

One control plane, six jobs.

Every request and response transits ARBR, so the same seat routes, governs, observes, evaluates and ships new models, all from one place.

01 · Route

Right model, right cost

One OpenAI-compatible endpoint over every provider. Rules, cost guardrails, difficulty-aware selection and fallback pick the model per request.

02 · Govern

Policy before inference

Budgets, rate limits, output guardrails, prompt-injection checks and kill switches act in the request path, not after the fact.

03 · Observe

See every request

Cost, latency, tokens and routing decisions as structured events, attributed by app, team, model and user. Local dashboards or OpenTelemetry.

04 · Evaluate

Quality, not just cost

Score a sample of live traffic with an LLM judge, then route to the cheapest model proven to clear the bar.

05 · Deploy

Ship models safely

Canary or shadow a new model on real traffic, with instant rollback and regression gates that block it until evals pass.

Roadmap06 · Capture

Traffic into training data

Turn the traffic ARBR already logs into fine-tuning-ready datasets. Not shipped yet.

System architecture

One gateway between your apps and every model.

Point any application at a single OpenAI-compatible endpoint. ARBR classifies, routes, meters and governs each request, then delivers it back, across every major provider plus any OpenAI-compatible or self-hosted endpoint. Pricing and benchmark data spans 3,000+ models.

ClientsPoint any app at one OpenAI-compatible endpoint.

Control planeClassify, route, meter and govern every request.

DeliversResponse, analytics and recommendations back out.

ProvidersOne layer over every major provider and any OpenAI-compatible endpoint.

SYSTEM ARCHITECTURE LIVE

MODEL PROVIDERS

Every major provider · 3,000+ models priced · one endpoint

Frontier labs

AnthropicOpenAIGoogle Gemini

Cloud

Amazon BedrockAzure OpenAIVertex AI

Fast · open

GroqDeepSeekMoonshotxAIMistral

Via gateway

LiteLLMNVIDIA NIM

Operational visibility

Know what your AI estate is doing.

Measure spend, performance and reliability by provider, model, application, workflow, team and user. Every call becomes a structured event you can query in local dashboards or export to OpenTelemetry (Datadog, Grafana, Prometheus).

Overview

Usage, cost and system health across your AI estate.

Last 30 days

Total requests

1.82M

↑ 14.2%

Total cost

$6,281

↓ 18.4%

Avg. latency

612ms

↓ 9.6%

Realised savings

$4,392

↑ 22.1%

Requests and cost

Spend by provider

$6.2ktotal spend

ApplicationModelTaskDecisionCost

support-copilotgemini-flash-litesummarisationguardrail$0.0018

sales-assistantclaude-sonnetanalysisrule$0.0321

invoice-extractgpt-4o-miniextractioncache$0.0000

Integrate once

Change the base URL. Keep the stack.

ARBR is OpenAI-compatible, so your SDK, agent framework and chat UI keep working. All requests immediately gain routing, logging, authentication and budget enforcement.

✓Streaming chat completions

✓OpenAI-compatible embeddings

✓Realtime voice proxy

✓JavaScript and Python SDKs

# Keep using the OpenAI SDK
from openai import OpenAI

client = OpenAI(
    base_url="https://arbr.yourcompany.com/v1",
    api_key="ab_your_gateway_key"
)

response = client.chat.completions.create(
    model="auto",
    messages=[{
        "role": "user",
        "content": "Summarise this ticket"
    }]
)

print(response.choices[0].message.content)

// Keep using the OpenAI SDK
import OpenAI from "openai";

const client = new OpenAI({
  baseURL: "https://arbr.yourcompany.com/v1",
  apiKey: "ab_your_gateway_key"
});

const response = await client.chat.completions.create({
  model: "auto",
  messages: [{ role: "user", content: "Summarise this ticket" }]
});

console.log(response.choices[0].message.content);

curl https://arbr.yourcompany.com/v1/chat/completions \
  -H "Authorization: Bearer ab_your_gateway_key" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "auto",
    "messages": [{
      "role": "user",
      "content": "Summarise this ticket"
    }]
  }'

Put one operational layer across every model.

Deploy ARBR, point your existing clients at the gateway and start building a governed view of your AI estate from the first request.

View project on GitHub ↗Read deployment docs