Provider Safety Signals

Provider safety signals are safety metadata returned by AI providers alongside their model responses. When a provider flags content for hate speech, sexual content, self-harm, or violence, that metadata is available in the API response. Arbitex’s SafetySignalReader parses these provider-specific formats, normalizes them into a canonical SafetySignal representation, and populates provider_safety_signals, provider_safety_categories, and provider_safety_max_severity on the request context.

SafetySignalReader is active in the output processing path (intake_pipeline.py). It runs during response processing, reads provider safety metadata from the raw API response, and makes the normalized signals available for downstream policy evaluation. Failures are caught and logged at DEBUG level (fail-open) — a provider response that cannot be parsed does not block the request.

Enabling provider safety signals

Provider safety signal parsing is gated on the provider_safety_signals_enabled system config toggle.

Config key	Type	Default	Description
provider_safety_signals_enabled	boolean	true	Enable parsing of provider safety metadata from API responses

To update via the admin API:

curl -X PUT https://platform.arbitex.ai/api/v1/admin/config/provider_safety_signals_enabled \
  -H "Authorization: Bearer $ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"value": true}'

When disabled, the SafetySignalReader.parse() method returns an empty list without inspecting the response body.

Supported providers

OpenAI

OpenAI returns safety metadata in a content_filter_results dict. On standard OpenAI API responses, this appears on each choice object. On Azure OpenAI, it may appear at the top level of the response.

{
  "choices": [
    {
      "content_filter_results": {
        "hate": {"filtered": false, "severity": "low"},
        "sexual": {"filtered": false, "severity": "none"},
        "self_harm": {"filtered": false, "severity": "none"},
        "violence": {"filtered": true, "severity": "high"}
      }
    }
  ]
}

A signal is emitted when filtered is true OR when severity is explicitly set to any value. The parser checks choice-level first, then falls back to top-level (Azure format).

Anthropic

Anthropic’s current API does not expose structured per-category safety metadata. The only reliable signal is:

{
  "stop_reason": "content_filtered"
}

When detected, a single signal is emitted with category="safety_filtered" and severity="high". No per-category breakdown is available from the Anthropic API.

Google (Gemini)

Google Gemini returns safety ratings as a list on each candidate:

{
  "candidates": [
    {
      "safety_ratings": [
        {
          "category": "HARM_CATEGORY_HATE_SPEECH",
          "probability": "LOW"
        },
        {
          "category": "HARM_CATEGORY_SEXUALLY_EXPLICIT",
          "probability": "NEGLIGIBLE"
        }
      ]
    }
  ]
}

Only signals with probability LOW or higher are emitted. NEGLIGIBLE is treated as severity="none" and skipped. Unknown categories are mapped to "violence" as a conservative fallback.

SafetySignal fields

Each parsed signal is represented as a SafetySignal with the following fields:

Field	Type	Description
category	string	Arbitex canonical category (see below)
severity	string	Normalized severity: none, low, medium, or high
provider_category	string	Original provider term (e.g., "HARM_CATEGORY_HATE_SPEECH", "hate", "content_filtered")
provider	string	Provider slug: openai, anthropic, or google
raw_data	dict	Original signal dict preserved for audit trail

Canonical categories

Arbitex Category	Description
hate	Hate speech and harassment
sexual	Sexually explicit content
self_harm	Self-harm related content
violence	Violent content
safety_filtered	Anthropic catch-all (no per-category breakdown available)

Category mapping

OpenAI → Arbitex

OpenAI Key	Arbitex Category
hate	hate
sexual	sexual
self_harm	self_harm
violence	violence

Google Gemini → Arbitex

Google Category	Arbitex Category
HARM_CATEGORY_HATE_SPEECH	hate
HARM_CATEGORY_HARASSMENT	hate
HARM_CATEGORY_SEXUALLY_EXPLICIT	sexual
HARM_CATEGORY_DANGEROUS_CONTENT	violence
HARM_CATEGORY_CIVIC_INTEGRITY	violence

Severity levels

Severity values are ordered: none < low < medium < high.

The max_severity() utility computes the highest severity across all signals in a response. This can be used by downstream components to make aggregate policy decisions — for example, blocking any response with a maximum severity of high.

Severity	Meaning
none	No safety concern detected
low	Minor safety signal detected
medium	Moderate safety concern
high	Content was filtered or flagged at highest severity

See also

• DLP Overview
• Provider Management