DLP Redaction Visibility

DLP Redaction Visibility

Arbitex enforces Data Loss Prevention (DLP) policies at every stage of the AI pipeline. When a policy modifies your prompt or blocks a response, the platform delivers enforcement events over the Server-Sent Events (SSE) stream and may modify persisted message content with [REDACTED] tokens.

This guide covers the four DLP pipeline stages, the SSE event types each stage can produce, the difference between SSE-transient metadata and persisted data, and the behavioral differences between platform and Outpost deployments.

---

The DLP Pipeline Stages

Content passes through four enforcement stages on every request:

Stage	When	What it does	SSE event produced
Input NER scan	Before the LLM call	Full NER scan on user input; substitutes detected entities with [REDACTED] tokens	input_redacted
Streaming regex	During output streaming	Fast pattern matching; redacts common PII/PHI formats in real-time as tokens arrive	None — content modified inline
Output finalization	After stream ends	Full NER scan on complete model output; blocks response if policy threshold exceeded	output_blocked
Post-stream audit	After stream ends	Catches entities the streaming regex missed; corrects output content in place	dlp_correction

All four stages run on every chat request. Each stage can independently produce SSE events and modify message content.

---

Input Redaction — input_redacted Event

When it appears: The input NER scan detected one or more entities in your prompt and substituted them with [REDACTED] tokens before the prompt was sent to the AI model.

The event is emitted before the LLM stream begins.

What the event contains

{
  "type": "input_redacted",
  "original_length": 245,
  "redacted_count": 2,
  "entities": [
    {"entity_type": "ssn", "action": "redact", "confidence": 0.95},
    {"entity_type": "credit_card", "action": "redact", "confidence": 0.99}
  ],
  "policy_name": "dlp-pipeline"
}

• original_length — character length of the original prompt before redaction
• redacted_count — number of entities substituted
• entities — array of detected entities with type, action, and confidence score
• policy_name — either "dlp-pipeline" for DLP-level redaction or the name of the matching policy from the policy engine (defaults to "policy-engine")

What the model received

The message content stored in the database and visible in the chat interface reflects what was actually sent to the model — with [REDACTED] tokens in place of the removed content. The entities list and policy_name are SSE-transient and are not persisted.

---

Output Blocking — output_blocked Event

When it appears: The output finalization scan (full NER on the complete model response) determined the response exceeded a policy block threshold. The response is suppressed and not shown to the user.

The event is emitted after the stream ends and the post-stream scan completes.

What the event contains

{
  "type": "output_blocked",
  "message": "Response blocked by output policy"
}

The output_blocked event contains only type and message. There is no policy_name field in the SSE event — policy metadata for blocked responses is recorded in the DLP audit log and is accessible to administrators only.

---

Post-Stream Correction — dlp_correction Event

When it appears: After the full stream completed, the post-stream audit stage detected an entity that the streaming regex missed. The output content is corrected in place with a [REDACTED] token substitution.

One dlp_correction event is emitted per entity caught by the post-stream scan.

What the event contains

{
  "type": "dlp_correction",
  "redaction": {"entity_type": "email_address", "confidence": 0.92}
}

• entity_type — the type of entity the post-stream NER scan caught
• confidence — confidence score from the NER model

Post-stream corrections are a normal part of the pipeline. Streaming regex prioritizes speed; post-stream NER applies full entity recognition for completeness. The corrected [REDACTED] token is persisted in the message content.

---

[REDACTED] Tokens in Message Content

When DLP redacts an entity from a message, the matched text is replaced with a literal redaction token. The exact format depends on whether the request is processed by the platform or by an Outpost proxy.

Platform format

The platform replaces matched text with the plain string:

[REDACTED]

No entity type suffix is included in the token itself. Entity type information is carried in the SSE event payload and the DLP audit log.

Outpost format

The Outpost proxy uses an entity-typed format:

[REDACTED:credit_card]
[REDACTED:ssn]
[REDACTED:email_address]

The entity type suffix is embedded directly in the token. This allows downstream systems and users to distinguish what category of data was removed without consulting the audit log.

Persistence behavior

[REDACTED] tokens that appear in message content are persisted in the database. If the conversation is reloaded, the tokens remain in the message exactly as stored. The DLP metadata that accompanies them — entity counts, entity types, policy name, and banner state — is SSE-transient and is not stored. That metadata is available only during the live session and only through the DLP audit log after the fact.

---

DLP in Compare and Summarize Modes

In Compare mode, each model stream is checked independently through all four pipeline stages. An input_redacted event applies to the shared prompt sent to all models; output_blocked and dlp_correction events are per-stream and may differ between models in the comparison.

In Summarize mode, both the source model responses and the synthesizer output are subject to DLP enforcement. The synthesizer output runs through its own output finalization and post-stream audit.

---

SSE-Transient vs. Persisted Data

Data	Persisted?	Survives page reload?
[REDACTED] tokens in message content	Yes — stored in the messages table	Yes
Entity counts and types from input_redacted	No — SSE only	No
Policy name from input_redacted	No — SSE only	No
Block message from output_blocked	No — SSE only	No
Entity type and confidence from dlp_correction	No — SSE only	No
Banner display state (which banner is shown)	No — SSE only	No
Full DLP event details	Yes — DLP events table	Yes (admin access)

If you need DLP event history — entity types, policy triggers, confidence scores, timestamps — your administrator can access the full audit log through the Admin Dashboard.

---

Outpost-Specific Behavior

When requests are processed through an Outpost proxy, DLP enforcement has several behavioral differences from the platform.

Buffered response for REDACT action

When the DLP pipeline determines a REDACT action is needed on the output, Outpost buffers the full response stream, applies redaction across the complete content, and returns the entire response as a single non-streaming JSON response. The client receives one complete response rather than a token stream. This differs from the platform, which redacts inline during streaming.

X-DLP-Action response header

Outpost adds an X-DLP-Action response header when DLP enforcement modifies the response:

X-DLP-Action: REDACT

This header is present on the HTTP response and can be inspected by intermediate proxies, logging systems, or client applications.

Entity-typed redaction format

As noted above, Outpost uses [REDACTED:{entity_type}] rather than plain [REDACTED]. This format is consistent across all DLP actions taken by the Outpost proxy and is embedded in the persisted message content when using Outpost.

---

Admin Perspective

How DLP policies trigger these behaviors

DLP policies define action maps that control what happens when an entity is detected. The DLPAction enum defines four actions:

Action	Priority	Behavior
log_only	0	Record the detection in the audit log; do not modify content
prompt	1	Issue a governance challenge before proceeding
redact	2	Replace matched text with a [REDACTED] token
block	3	Suppress the entire response

When multiple detectors match the same content, the action with the highest priority applies.

DLP sensitivity levels and confidence routing

The DLP service routes detections through confidence thresholds that interact with the configured dlp_sensitivity setting:

Confidence range	Sensitivity setting	Routing outcome
> 0.70	Any	hard_block
0.35 – 0.70	high	soft_block with audit flag
0.35 – 0.70	standard	pass with elevated audit flag
< 0.35	Any	pass

Detections below 0.35 confidence are passed through without action but may still appear in audit logs at the elevated flag level depending on policy configuration.

DLP match object

The DLP pipeline records the following fields for each detection in the audit log:

Field	Type	Description
detector_name	string	Detector that made the match: "regex", "ner", "gliner", or "llm"
entity_type	string	Entity category: "credit_card", "ssn", "api_key", etc.
matched_text	string	The original text that was matched
start	integer	Start character offset in the source string
end	integer	End character offset in the source string
confidence	float	Confidence score from 0.0 to 1.0

Monitoring DLP events in the Admin Dashboard

The Admin Dashboard DLP events view shows aggregate metrics and individual event records:

• Total events — all DLP detections across all actions
• Blocks — events where the block action was applied
• Redactions — events where the redact action was applied
• Cancels — requests cancelled due to governance challenges

Events are assigned a severity level based on the action and entity type:

Severity	Condition
LOW	log_only action
MEDIUM	redact action
HIGH	block action
CRITICAL	Multiple blocks in a session or api_key entity type detected

Full event details including entity type, confidence score, detector name, policy name, and character offsets are available in the DLP events table and exportable to your SIEM via the audit API.

---

Related Documentation

• DLP Pipeline Configuration — configuring action maps, sensitivity levels, and entity type rules
• Policy Engine — how the policy engine interacts with DLP enforcement
• Outpost Deployment — DLP enforcement in on-premises and hybrid deployments