Skip to content

Emergency Controls API

The Emergency Controls system provides instant, Redis-backed enforcement of emergency states across the Arbitex Platform. The EmergencyMiddleware intercepts every HTTP request and returns 503 Service Unavailable when any emergency key is active for the request’s org, feature, or outpost.

For model/provider-level kill switches (disabling individual models or providers), see the Kill Switch API. The emergency controls documented here operate at a higher level — org isolation, feature-wide shutdown, and credential revocation.

All endpoints require admin authentication (Authorization: Bearer $ARBITEX_API_KEY) unless noted otherwise.

Base URL: https://gateway.arbitex.ai


Emergency keys are stored in Redis. The EmergencyMiddleware (ASGI middleware) checks these keys on every inbound HTTP request, with an in-process cache (5-second TTL) to avoid per-request Redis round-trips.

Fail behavior: The middleware checks Redis first, then falls back to the platform database (system_config table). If both Redis and the database are unavailable, the middleware fails open — requests are allowed through. The enforcement toggle defaults to enabled (fail-closed) when the database is unavailable.

Response format: When an emergency key is active, the middleware returns:

HTTP/1.1 503 Service Unavailable
Content-Type: application/json
Retry-After: 30
{"detail": "Organization is isolated due to emergency controls", "code": "emergency_org_isolated"}

The code field identifies which emergency control triggered the block. The Retry-After: 30 header signals clients to back off for 30 seconds.


The middleware checks four Redis key families. A fifth family (audit_freeze) is read by the audit sink only and is not enforced in the request path.

Redis Key Pattern Scope Error Code Description
emergency:org_isolated:{org_id} Per-org emergency_org_isolated Isolates an organization — all API requests for the org are blocked
emergency:credentials_revoked:{org_id} Per-org emergency_credentials_revoked Blocks all requests for an org whose credentials have been revoked
emergency:killswitch:{feature} Per-feature emergency_killswitch_active Disables a feature by path prefix (see feature map below)
emergency:outpost_disconnected:{outpost_id} Per-outpost emergency_outpost_disconnected Severs an outpost’s connection to the platform
emergency:audit_freeze:{scope} Per-org or global Freezes audit log writes; read by audit sink, not enforced in middleware

The emergency:killswitch:{feature} key maps request paths to feature identifiers:

Path Prefix Feature Key
/api/v1/chat chat
/api/v1/admin admin
/internal/dlp dlp
/api/v1/mcp mcp
/api/v1/oauth oauth
/api/v1/scim scim
/api/v1/saml saml

Setting emergency:killswitch:chat in Redis disables all chat completions. Setting emergency:killswitch:admin disables the entire admin API surface.

The following paths are never checked by the emergency middleware:

  • /internal/health* — health probes
  • /internal/metrics — Prometheus scrape
  • /docs — API documentation
  • /openapi.json — OpenAPI spec
  • /redoc — ReDoc viewer

The emergency_enforcement.enabled system config key controls whether the middleware is active. When set to false (or "disabled", "0"), all emergency checks are skipped. The toggle value is cached for 5 seconds. Default: true (enforcement enabled).


The middleware resolves the request’s identity from ASGI scope state:

Check Source
Org ID scope["state"]["org_id"] or scope["state"]["tenant_id"] (set by auth middleware)
Outpost ID scope["state"]["outpost_id"] or X-Outpost-Id request header
Feature Request path matched against the feature map

Per-org checks (org_isolated, credentials_revoked) run first. Feature kill-switch checks run next. Outpost disconnection checks run last. The first match short-circuits — no further checks are performed.


Sync quarantine records from Hybrid Outpost

Section titled “Sync quarantine records from Hybrid Outpost”
POST /v1/internal/outpost-quarantine-sync

Receives batched quarantine records from Hybrid Outpost instances. The outpost queues quarantine events locally (SQLite) and drains them to the platform via this endpoint during periodic sync cycles. This enables central quarantine management even when outpost-to-platform connectivity is intermittent.

Authentication: mTLS client certificate (internal endpoint). The caller’s Common Name (CN) is extracted from the client certificate for audit purposes.

Request body

Field Type Required Description
records OutpostQuarantineRecord[] Yes Array of quarantine records (1–500 per request)

OutpostQuarantineRecord

Field Type Required Description
org_id UUID Yes Organization that owns the quarantined content
content_ref string | null No Reference to the original content (e.g., object store key)
scan_result object Yes DLP scan result that triggered quarantine
source string Yes Quarantine source identifier (max 32 chars, e.g., "outpost_proxy", "email_relay")
channel string | null No Channel that produced the content (max 64 chars, e.g., "ai_gateway", "email", "file_upload")
item_metadata object | null No Arbitrary metadata (user agent, original filename, etc.)
user_id UUID | null No User who generated the content, if known
client_id UUID | null No OAuth client that generated the content, if applicable

Request

Terminal window
curl -s -X POST https://gateway.arbitex.ai/v1/internal/outpost-quarantine-sync \
--cert /etc/arbitex/certs/outpost.pem \
--key /etc/arbitex/certs/outpost-key.pem \
--cacert /etc/arbitex/certs/ca.pem \
-H "Content-Type: application/json" \
-d '{
"records": [
{
"org_id": "550e8400-e29b-41d4-a716-446655440000",
"source": "outpost_proxy",
"channel": "ai_gateway",
"scan_result": {
"entities_found": ["ssn", "credit_card"],
"action": "quarantine",
"confidence": 0.97
},
"content_ref": "s3://quarantine/2026/04/06/abc123.enc",
"item_metadata": {
"model": "gpt-4o",
"user_agent": "python-requests/2.31"
}
}
]
}'

Response 200 OK

{
"created": 1,
"errors": []
}

Response fields

Field Type Description
created int Number of records successfully persisted to the central quarantine database
errors array Per-record errors for any records that failed. Each entry contains index (position in the input array), org_id, and error (description)

Partial success example — 3 records submitted, 1 fails:

{
"created": 2,
"errors": [
{
"index": 1,
"org_id": "550e8400-e29b-41d4-a716-446655440000",
"error": "Duplicate content_ref"
}
]
}

Error responses

Status Description
403 Invalid or missing mTLS client certificate
422 Request body validation failure (empty records array, exceeds 500 records, missing required fields)

The quarantine sync follows a two-path architecture:

  1. Real-time proxy path: The outpost’s gateway proxy evaluates content against the DLP pipeline. When a policy action is quarantine, the content is blocked immediately and a QuarantineItem is written to the outpost’s local SQLite database.

  2. Async sync path: A background task on the outpost periodically drains the local SQLite queue and sends batched records to the platform via POST /v1/internal/outpost-quarantine-sync. On success, the local records are marked as synced. On failure (network interruption), records remain queued for the next sync cycle.

This design ensures quarantine enforcement is immediate (no dependency on platform connectivity) while central quarantine management is eventually consistent.