Emergency Controls API
The Emergency Controls system provides instant, Redis-backed enforcement of emergency states across the Arbitex Platform. The EmergencyMiddleware intercepts every HTTP request and returns 503 Service Unavailable when any emergency key is active for the request’s org, feature, or outpost.
For model/provider-level kill switches (disabling individual models or providers), see the Kill Switch API. The emergency controls documented here operate at a higher level — org isolation, feature-wide shutdown, and credential revocation.
All endpoints require admin authentication (Authorization: Bearer $ARBITEX_API_KEY) unless noted otherwise.
Base URL: https://gateway.arbitex.ai
How emergency controls work
Section titled “How emergency controls work”Emergency keys are stored in Redis. The EmergencyMiddleware (ASGI middleware) checks these keys on every inbound HTTP request, with an in-process cache (5-second TTL) to avoid per-request Redis round-trips.
Fail behavior: The middleware checks Redis first, then falls back to the platform database (system_config table). If both Redis and the database are unavailable, the middleware fails open — requests are allowed through. The enforcement toggle defaults to enabled (fail-closed) when the database is unavailable.
Response format: When an emergency key is active, the middleware returns:
HTTP/1.1 503 Service UnavailableContent-Type: application/jsonRetry-After: 30
{"detail": "Organization is isolated due to emergency controls", "code": "emergency_org_isolated"}The code field identifies which emergency control triggered the block. The Retry-After: 30 header signals clients to back off for 30 seconds.
Emergency key families
Section titled “Emergency key families”The middleware checks four Redis key families. A fifth family (audit_freeze) is read by the audit sink only and is not enforced in the request path.
| Redis Key Pattern | Scope | Error Code | Description |
|---|---|---|---|
emergency:org_isolated:{org_id} |
Per-org | emergency_org_isolated |
Isolates an organization — all API requests for the org are blocked |
emergency:credentials_revoked:{org_id} |
Per-org | emergency_credentials_revoked |
Blocks all requests for an org whose credentials have been revoked |
emergency:killswitch:{feature} |
Per-feature | emergency_killswitch_active |
Disables a feature by path prefix (see feature map below) |
emergency:outpost_disconnected:{outpost_id} |
Per-outpost | emergency_outpost_disconnected |
Severs an outpost’s connection to the platform |
emergency:audit_freeze:{scope} |
Per-org or global | — | Freezes audit log writes; read by audit sink, not enforced in middleware |
Feature kill-switch key mapping
Section titled “Feature kill-switch key mapping”The emergency:killswitch:{feature} key maps request paths to feature identifiers:
| Path Prefix | Feature Key |
|---|---|
/api/v1/chat |
chat |
/api/v1/admin |
admin |
/internal/dlp |
dlp |
/api/v1/mcp |
mcp |
/api/v1/oauth |
oauth |
/api/v1/scim |
scim |
/api/v1/saml |
saml |
Setting emergency:killswitch:chat in Redis disables all chat completions. Setting emergency:killswitch:admin disables the entire admin API surface.
Exempt paths
Section titled “Exempt paths”The following paths are never checked by the emergency middleware:
/internal/health*— health probes/internal/metrics— Prometheus scrape/docs— API documentation/openapi.json— OpenAPI spec/redoc— ReDoc viewer
Enforcement toggle
Section titled “Enforcement toggle”The emergency_enforcement.enabled system config key controls whether the middleware is active. When set to false (or "disabled", "0"), all emergency checks are skipped. The toggle value is cached for 5 seconds. Default: true (enforcement enabled).
Identity resolution
Section titled “Identity resolution”The middleware resolves the request’s identity from ASGI scope state:
| Check | Source |
|---|---|
| Org ID | scope["state"]["org_id"] or scope["state"]["tenant_id"] (set by auth middleware) |
| Outpost ID | scope["state"]["outpost_id"] or X-Outpost-Id request header |
| Feature | Request path matched against the feature map |
Per-org checks (org_isolated, credentials_revoked) run first. Feature kill-switch checks run next. Outpost disconnection checks run last. The first match short-circuits — no further checks are performed.
Quarantine ingest endpoint
Section titled “Quarantine ingest endpoint”Sync quarantine records from Hybrid Outpost
Section titled “Sync quarantine records from Hybrid Outpost”POST /v1/internal/outpost-quarantine-syncReceives batched quarantine records from Hybrid Outpost instances. The outpost queues quarantine events locally (SQLite) and drains them to the platform via this endpoint during periodic sync cycles. This enables central quarantine management even when outpost-to-platform connectivity is intermittent.
Authentication: mTLS client certificate (internal endpoint). The caller’s Common Name (CN) is extracted from the client certificate for audit purposes.
Request body
| Field | Type | Required | Description |
|---|---|---|---|
records |
OutpostQuarantineRecord[] |
Yes | Array of quarantine records (1–500 per request) |
OutpostQuarantineRecord
| Field | Type | Required | Description |
|---|---|---|---|
org_id |
UUID |
Yes | Organization that owns the quarantined content |
content_ref |
string | null |
No | Reference to the original content (e.g., object store key) |
scan_result |
object |
Yes | DLP scan result that triggered quarantine |
source |
string |
Yes | Quarantine source identifier (max 32 chars, e.g., "outpost_proxy", "email_relay") |
channel |
string | null |
No | Channel that produced the content (max 64 chars, e.g., "ai_gateway", "email", "file_upload") |
item_metadata |
object | null |
No | Arbitrary metadata (user agent, original filename, etc.) |
user_id |
UUID | null |
No | User who generated the content, if known |
client_id |
UUID | null |
No | OAuth client that generated the content, if applicable |
Request
curl -s -X POST https://gateway.arbitex.ai/v1/internal/outpost-quarantine-sync \ --cert /etc/arbitex/certs/outpost.pem \ --key /etc/arbitex/certs/outpost-key.pem \ --cacert /etc/arbitex/certs/ca.pem \ -H "Content-Type: application/json" \ -d '{ "records": [ { "org_id": "550e8400-e29b-41d4-a716-446655440000", "source": "outpost_proxy", "channel": "ai_gateway", "scan_result": { "entities_found": ["ssn", "credit_card"], "action": "quarantine", "confidence": 0.97 }, "content_ref": "s3://quarantine/2026/04/06/abc123.enc", "item_metadata": { "model": "gpt-4o", "user_agent": "python-requests/2.31" } } ] }'Response 200 OK
{ "created": 1, "errors": []}Response fields
| Field | Type | Description |
|---|---|---|
created |
int |
Number of records successfully persisted to the central quarantine database |
errors |
array |
Per-record errors for any records that failed. Each entry contains index (position in the input array), org_id, and error (description) |
Partial success example — 3 records submitted, 1 fails:
{ "created": 2, "errors": [ { "index": 1, "org_id": "550e8400-e29b-41d4-a716-446655440000", "error": "Duplicate content_ref" } ]}Error responses
| Status | Description |
|---|---|
403 |
Invalid or missing mTLS client certificate |
422 |
Request body validation failure (empty records array, exceeds 500 records, missing required fields) |
Quarantine flow: outpost to platform
Section titled “Quarantine flow: outpost to platform”The quarantine sync follows a two-path architecture:
-
Real-time proxy path: The outpost’s gateway proxy evaluates content against the DLP pipeline. When a policy action is
quarantine, the content is blocked immediately and aQuarantineItemis written to the outpost’s local SQLite database. -
Async sync path: A background task on the outpost periodically drains the local SQLite queue and sends batched records to the platform via
POST /v1/internal/outpost-quarantine-sync. On success, the local records are marked as synced. On failure (network interruption), records remain queued for the next sync cycle.
This design ensures quarantine enforcement is immediate (no dependency on platform connectivity) while central quarantine management is eventually consistent.
See also
Section titled “See also”- Kill Switch API — model/provider-level disable/enable
- Security Operations — incident response playbooks that use emergency controls
- Email DLP API — email quarantine management
- Outpost Architecture — outpost deployment and sync architecture