Content Detection

Content detection extends the DLP pipeline with behavioral content controls — profanity filtering, language enforcement, and code exfiltration prevention. Each detector is independently toggled, org-configurable, and policy-integrated. All run as DLPDetector ABC implementations registered in AsyncScanEngine.

How content detection integrates with the pipeline

Detectors run as part of the DLP pipeline scan on each request and response. Findings from these detectors appear in audit events alongside PII and credential findings. Each detector has an independent enable/disable toggle and an admin UI panel.

Staff portal provides two override mechanisms:

• Floor enforcement: minimum action an org must apply. If an org sets allow but the floor is redact, redact takes effect.
• Hard-block override: forces block regardless of org configuration. Useful for regulated industry deployments.

Profanity detector

Detects profanity and explicit language using a curated keyword word list with Levenshtein distance tolerance. The tolerance catches l33tspeak and character substitution evasion — for example, f*ck, sh!t, phuck.

Default behavior: disabled.

Admin UI: Navigate to Admin → Content Detection → Profanity.

• Toggle: enable or disable profanity detection.
• Action: allow (log only), redact (replace matched terms with [PROFANITY]), or block (deny the request).
• Custom word list: add or remove terms per org.

Config keys

Config key	Type	Default	Description
profanity_detection_enabled	boolean	false	Enable profanity detection on all request/response text
profanity_action	enum	block	Action when profanity is detected: allow, redact, or block

Programmatic toggle

curl -X PUT https://platform.arbitex.ai/api/v1/admin/config/profanity_detection_enabled -H "Authorization: Bearer $ADMIN_TOKEN" -H "Content-Type: application/json" -d '{"value": true}'

Policy integration

Profanity detection fires the configured action directly — no separate policy rule is required. To use it in a policy rule instead, set profanity_action to allow (log only) and create a policy rule conditioned on DLP finding type profanity.

Language detector

Detects the primary language of request text using lingua-language-detector v2.x (Apache-2.0, Rust-backed). Identifies 75 languages with ~4–7ms latency. Detection accuracy: 74% on single words, 89% on word pairs, higher on sentence-length text.

Use cases: enforce English-only policies, allow a specific set of languages per org, flag unexpected language switching.

Default behavior: disabled.

Admin UI: Navigate to Admin → Content Detection → Language.

• Toggle: enable or disable language detection.
• Allowlist mode: specify ISO 639-1 codes permitted (e.g., en, de, fr).
• Blocklist mode: specify codes to block.
• If neither list is set and the detector is enabled, detection runs but no action is taken (log only).

Config keys

Config key	Type	Default	Description
language_detection_enabled	boolean	false	Enable language detection on request text

Policy condition

The language condition accepts an ISO 639-1 code string. Example — block non-English requests:

{
  "name": "Enforce English-only",
  "action": "block",
  "message": "This service only accepts requests in English.",
  "conditions": {
    "language_not": ["en"]
  }
}

Short inputs (fewer than ~10 characters) may produce unreliable language detections. The detector returns the highest-confidence language — confidence below the internal threshold returns no detection.

Code snippet detector

Detects code snippets in request and response text using a two-tier approach:

• Tier 1 (regex): fenced code blocks (triple backtick with language tag), and standalone syntax markers (def , function , class , import , #include, SELECT ... FROM, <?php, #!/).
• Tier 2 (heuristic): token density analysis — symbol density, bracket nesting depth, semicolons per line, camelCase/snake_case identifier frequency. Catches obfuscated or unfenced code.

Use cases: prevent code exfiltration in regulated environments, block model responses that return executable code to unauthorized users.

Default behavior: disabled.

Admin UI: Navigate to Admin → Content Detection → Code Snippets.

• Toggle: enable or disable code snippet detection.
• Language filter: optionally restrict detection to specific programming languages (e.g., detect only Python and JavaScript).
• Action: allow (log) or block.

Config keys

Config key	Type	Default	Description
code_detection_enabled	boolean	false	Enable code snippet detection on request/response text
code_detection_action	enum	block	Action when code is detected: allow or block

Programmatic toggle

curl -X PUT https://platform.arbitex.ai/api/v1/admin/config/code_detection_enabled -H "Authorization: Bearer $ADMIN_TOKEN" -H "Content-Type: application/json" -d '{"value": true}'

Policy condition

Code findings appear as code_snippet in dlp_findings. Example — block code in responses for non-developers:

{
  "name": "Block code in response for non-developers",
  "action": "block",
  "message": "Code responses are restricted in your organization.",
  "conditions": {
    "dlp_findings": ["code_snippet"],
    "groups_exclude": ["engineering", "red-team"]
  }
}

Pre-submit safety classifier

A lightweight input-direction classifier that runs before the LLM call. Complements provider safety signals, which are output-direction only. Classifies for: hate/discrimination, sexual content, violence, self-harm.

Default behavior: disabled.

Config key	Type	Default	Description
safety_classifier_enabled	boolean	false	Enable the pre-submit safety classifier on input text

Admin UI: Navigate to Admin → Content Detection → Pre-Submit Classifier.

• Toggle: enable or disable.
• Action per safety category: allow, flag, or block.

For output-direction safety (model responses), see Provider Safety Signals.

Staff portal overrides

Staff can set floor enforcement and hard-block overrides for any org via the staff portal (Staff → System Config).

• Floor enforcement: minimum action an org must apply. If an org sets allow but the floor is redact, redact takes effect.
• Hard-block override: forces block regardless of org configuration. Useful for regulated industry deployments.

Floor enforcement config keys: profanity_detection_enabled, language_detection_enabled, code_detection_enabled, safety_classifier_enabled.

Hard-block variants: profanity_hard_block, code_hard_block.

See also

• Tool Call Inspection
• Provider Safety Signals
• DLP Overview
• Policy Rule Reference
• Audit Log