Provider management

Arbitex Gateway supports nine built-in AI providers plus custom bring-your-own-endpoint (BYOE) configurations. This guide covers adding and configuring providers, setting up fallback chains for resilience, understanding health monitor states, and managing provider availability.

For the technical reference on routing modes, fallback chain behavior, and health monitor state transitions, see Routing.

Supported providers

Provider	Protocol	Notes
Anthropic	Native API	Claude model family
OpenAI	Native API	GPT model family
Google Gemini	Native API	Gemini model family
Azure OpenAI	Azure-specific API	Requires custom endpoint URL + deployment ID
AWS Bedrock	AWS SDK	Requires IAM credentials, region, and model ARN
Groq	OpenAI-compatible	High-throughput inference; OpenAI-compatible API format
Mistral	Native API	Mistral model family
Cohere	Native API	Command model family
Ollama	OpenAI-compatible	Self-hosted models; requires accessible Ollama base URL

Provider credentials are stored encrypted, per-organization. A credential configured for one org is never accessible to any other org.

Adding a provider

Standard providers (Anthropic, OpenAI, Google Gemini, Groq, Mistral, Cohere)

Navigate to Settings > Providers > Add Provider.
Select the provider from the list.
Enter the API key.
Click Save. Arbitex immediately tests the credential by listing available models. If the test fails, a validation error is shown — verify the key in your provider’s console before retrying.
After a successful save, the provider appears in the provider list with its available models.

Azure OpenAI

Azure OpenAI requires additional configuration beyond the API key:

Field	Description
API key	Azure OpenAI resource API key
Endpoint URL	Your Azure resource endpoint (e.g., `https://myresource.openai.azure.com`)
Deployment ID	The name of your Azure deployment (e.g., `gpt-4o`)
API version	Azure API version string (e.g., `2025-02-01-preview`)

Each Azure deployment appears as a separate provider entry with its own (provider, deployment_id) pair in the model catalog.

AWS Bedrock

AWS Bedrock requires IAM credentials:

Field	Description
Access key ID	AWS IAM access key
Secret access key	AWS IAM secret key
Region	AWS region where the model is available (e.g., `us-east-1`)
Model ARN	The Bedrock model ID (e.g., `anthropic.claude-3-5-sonnet-20241022-v2:0`)

For production, use an IAM role with minimal permissions (bedrock:InvokeModel on the specific model ARN). Avoid using root credentials or admin IAM keys.

Ollama (self-hosted)

Field	Description
Base URL	Ollama server URL accessible from Arbitex Cloud (e.g., `https://ollama.internal.example.com`)
Model name	The model name as registered in Ollama (e.g., `llama3.2`)

Ollama deployments must be reachable from Arbitex Cloud over HTTPS. If the Ollama instance is inside a private network, use a reverse proxy with TLS termination.

Bring your own endpoint (BYOE)

Any OpenAI-compatible API can be configured as a custom endpoint:

Navigate to Settings > Providers > Add Custom Endpoint.
Enter:
- Name — display name for this endpoint
- Base URL — the API base URL (requests will be sent to {base_url}/v1/chat/completions)
- API key (optional) — included in the Authorization: Bearer header
Arbitex tests the endpoint with a lightweight model-list request. If your endpoint does not implement /v1/models, disable the test check.

Custom endpoints appear in the model catalog as a custom provider. They support fallback chains and health monitoring in the same way as built-in providers.

Model catalog

After adding a provider, its available models are listed in the model catalog (Settings > Providers > [Provider] > Models). Each model entry shows:

Model ID — the identifier used in API requests and fallback chain configuration
Context window — token limit (input + output)
Input / output cost — per-million-token pricing (for billing and cost-optimized routing)
Capabilities — whether the model supports streaming, function calling, vision, etc.
Status — current availability

The catalog is updated when the provider adapter ships a new model. Models not in the catalog cannot be routed to — routing a request to an unlisted (provider, model_id) pair returns a 400 error.

Fallback chain configuration

Each model can have a fallback chain — an ordered list of alternative (provider, model) pairs. When the primary fails (5xx error or timeout), the gateway automatically routes to the next entry in the chain.

Setting up a fallback chain

Navigate to Settings > Routing > Fallback Chains and click the model you want to configure. Or use the API:

PUT https://api.arbitex.ai/api/providers/fallback/gpt-4o
Authorization: Bearer $ADMIN_TOKEN
Content-Type: application/json

{
  "chain": [
    {"provider": "openai", "model_id": "gpt-4o"},
    {"provider": "anthropic", "model_id": "claude-sonnet-4-20250514"},
    {"provider": "google_gemini", "model_id": "gemini-1.5-pro"}
  ]
}

The first entry in the chain is the primary. Subsequent entries are fallbacks in priority order.

Fallback chain design principles

Cross-provider diversity — do not use multiple models from the same provider as your chain. A provider outage disables all of them simultaneously.
Capability parity — ensure fallback models support the same features your application uses (streaming, function calling, vision). A fallback to a model that doesn’t support streaming will fail if the request uses streaming.
Cost awareness — fallback chains route to the next entry regardless of cost. If your primary is a low-cost model and the fallback is a premium model, a provider outage will increase your costs. Monitor fallback activation rates in the audit log.
Kill switch awareness — plan chains with the assumption that any entry may be kill-switched at any time. A kill-switched entry is silently skipped, not an error.

Verifying a fallback chain

GET https://api.arbitex.ai/api/providers/fallback/gpt-4o
Authorization: Bearer $ADMIN_TOKEN

{
  "model_id": "gpt-4o",
  "chain": [
    {"provider": "openai", "model_id": "gpt-4o"},
    {"provider": "anthropic", "model_id": "claude-sonnet-4-20250514"}
  ]
}

Health monitoring

The gateway monitors every (provider, model) pair in the catalog every 30 seconds. Health status is available per-pair:

GET https://api.arbitex.ai/api/providers/{provider}/models/{model_id}/health
Authorization: Bearer $ADMIN_TOKEN

{
  "provider": "openai",
  "model_id": "gpt-4o",
  "status": "healthy",
  "monitor_state": "Active",
  "last_checked": "2026-03-10T17:20:00Z"
}

Health monitor states

State	Meaning	Traffic behavior
`Active`	Normal operation	Requests route to this pair
`Disengaged`	Failure threshold exceeded	Requests skip this pair and use fallback
`Testing`	Lockout period elapsed (300s), testing recovery	One test request allowed through

The health monitor is automatic. You do not need to manually re-enable a pair after it goes Disengaged — after 300 seconds, the monitor transitions to Testing and allows a single test request. If the test succeeds, the monitor returns to Active. If it fails, the lockout restarts.

Monitoring health in the admin portal

Navigate to Settings > Providers. Each provider entry shows its current health status badge. Click a provider to see per-model health status.

A Disengaged badge on a model means the gateway is currently routing around that model. If this is unexpected, check the provider’s status page and the Arbitex audit log for failure events.

Disabling a provider (kill switch)

To disable a provider or model immediately — for an incident, security event, or maintenance — use the kill switch.

Via the admin portal

Navigate to Settings > Providers > [Provider Name]. Click the Disable toggle. A confirmation dialog requires a reason before activation.

Via the API

POST https://api.arbitex.ai/api/admin/kill-switch
Authorization: Bearer $ADMIN_TOKEN
Content-Type: application/json

{
  "provider": "openai",
  "model_id": "gpt-4o",
  "enabled": false,
  "reason": "Provider incident — disabling pending investigation"
}

Omit model_id to disable all models for a provider:

{
  "provider": "openai",
  "enabled": false,
  "reason": "Provider-wide incident — all openai traffic disabled"
}

Kill switch vs health monitor

	Kill switch	Health monitor
Trigger	Manual — admin	Automatic — failures
Recovery	Manual — admin re-enables	Automatic after 300s
Persists across restarts	Yes	No

Both can be active simultaneously. Re-enabling the kill switch does not override the health monitor — both must be in normal state before traffic resumes.

For a full operational guide including blast radius assessment and recovery procedures, see Kill switch — operational guide.

Updating provider credentials

To rotate an API key:

Navigate to Settings > Providers > [Provider Name] > Edit.
Enter the new API key in the API key field.
Click Save. Arbitex tests the new key before saving. If the test fails, the old key remains in use.

The old key is replaced immediately upon save. Any in-flight requests that were authenticated with the old key complete normally (they have already been authenticated at the provider). New requests use the new key.

Removing a provider

Navigate to Settings > Providers > [Provider Name] > Remove. This removes the provider and all its models from the catalog. Any fallback chains that include models from this provider will have those entries removed from the chain.

Removing a provider does not affect the audit log — historical records of requests to the removed provider are retained.

Troubleshooting

Provider health shows Disengaged unexpectedly

The health monitor disengages a pair after a series of failures. Check:

Provider status page for reported incidents.
Audit log for 5xx errors from that provider: GET /api/admin/audit-log?provider=<provider>&status=error&limit=50.
Provider API key — if the key was rotated at the provider level but not updated in Arbitex, all requests will fail with 401 errors.

New model not appearing in the catalog

The catalog is updated when the adapter ships. If a provider has released a new model but it is not in the catalog:

Verify the adapter version in Settings > System matches the latest release.
If the model is listed in the provider’s API but not in Arbitex, submit a model request or use a BYOE custom endpoint as a workaround.

Requests failing with 400 — unknown model

The (provider, model_id) pair is not in the catalog. Check:

Spelling of the model ID — provider model IDs are case-sensitive and version-pinned (e.g., claude-sonnet-4-20250514 not claude-sonnet).
Whether the model is available in the region or tier of your provider account.
Whether the provider entry in Arbitex has been configured correctly.