Providers & models · menu
Core
Providers & models
Eight providers are built in, plus any OpenAI-compatible endpoint, including a LiteLLM proxy. Connect a key once and its models become routable. Arbr never hands provider credentials to client applications.
Built-in providers
| Provider | ID | Env var | Default model |
|---|---|---|---|
| Anthropic | anthropic | ANTHROPIC_API_KEY | claude-haiku-4-5 |
| OpenAI | openai | OPENAI_API_KEY | gpt-4o-mini |
| Google Gemini | gemini | GEMINI_API_KEY | gemini-2.5-flash |
| Amazon Bedrock | bedrock-nova | AWS_ACCESS_KEY_ID + secret | us.amazon.nova-lite-v1:0 |
| DeepSeek | deepseek | DEEPSEEK_API_KEY | deepseek-chat |
| Moonshot AI | moonshot | MOONSHOT_API_KEY | moonshot-v1-8k |
| xAI (Grok) | xai | XAI_API_KEY | grok-3-mini |
| Groq | groq | GROQ_API_KEY | llama-3.3-70b-versatile |
Connecting a provider
Two ways, and you can mix them:
- Dashboard. Open the Models page, pick a provider, paste a key, and it goes live immediately with no restart. Keys are stored encrypted at rest, shown only masked, and never returned to the browser. See Setting up routing for the step-by-step.
- Environment. Set the provider's env var before starting. Environment variables take precedence over dashboard-stored keys, which is the recommended path for production.
With zero provider keys configured, Arbr runs in demo mode: every dashboard, the recommendation engine, and the routing controls work on seeded data. Only live /v1 calls are disabled until you add a key.
LiteLLM & custom OpenAI-compatible providers
Arbr sits in front of LiteLLM. It's an upstream Arbr connects to, not a replacement. You keep LiteLLM's provider breadth and layer Arbr's routing, evidence, budgets, and governance on top. Connect it from the dashboard (Discover → Import models), or point the openai provider at any OpenAI-compatible base URL:
OPENAI_BASE_URL=https://your-litellm-host/v1
OPENAI_API_KEY=sk-litellm-...The same mechanism connects vLLM, Ollama, NVIDIA NIM, or any other OpenAI-compatible server. Requests, tool calls, and streaming pass through the full chain unchanged.
The model registry
Arbr ships 29 built-in models with pricing, tier, and capability metadata. The registry drives cost tracking, recommendations, guardrail downgrades, and routing. A model without a pricing entry still works as a pass-through. Its cost is logged as $0 and left out of savings claims until you add pricing.
A sample of the registry
| Model | Provider | Tier | $ / 1M in → out |
|---|---|---|---|
claude-opus-4-8 | anthropic | premium | $5 → $25 |
claude-sonnet-4-6 | anthropic | mid | $3 → $15 |
claude-haiku-4-5 | anthropic | light | $1 → $5 |
gpt-4o | openai | premium | $2.50 → $10 |
gpt-4o-mini | openai | light | $0.15 → $0.60 |
gemini-2.5-pro | gemini | premium | $1.25 → $10 |
gemini-2.5-flash | gemini | light | $0.30 → $2.50 |
nova-lite-v1:0 | bedrock-nova | light | $0.06 → $0.24 |
deepseek-chat | deepseek | light | $0.27 → $1.10 |
grok-3-mini | xai | light | $0.30 → $0.50 |
llama-3.3-70b-versatile | groq | mid | $0.59 → $0.79 |
Add or edit models from the dashboard, the admin API, or the seed. Built-in pricing updates on re-seed without touching your custom entries.
Tool-call support
Not every provider supports function calling through Arbr today. GET /v1/models returns a toolCallSupported flag per model, so a client knows whether to enable its function-calling UI. It's true for OpenAI, DeepSeek, Groq, xAI, Moonshot, LiteLLM, and Amazon Nova. It's false for Gemini, Anthropic, and non-Nova Bedrock models. If your agent needs tools, pin a tool-capable model rather than using auto.
Provider-specific quirks, like Opus rejecting temperature, Gemini's thinking-variant JSON, and Bedrock IAM permissions, are documented per provider in the providers reference on GitHub ↗.