IR-3: DLP Bypass Detected

A DLP bypass occurs when the 5-tier DLP pipeline is weakened or disabled — either intentionally (by an unauthorized actor) or accidentally (by a misconfiguration). The result is sensitive content passing through the platform without being scanned, redacted, or blocked. This is automatically CRITICAL severity because the impact window is unknown until the audit trail is analyzed.

For DLP pipeline architecture, see DLP Pipeline. For DLP configuration reference, see DLP Pipeline Configuration.

---

1. Detection

DLP bypass detection signals:

Signal	Source	Confidence
scan_level changed to disabled or log_only in audit trail	dlp.config_updated audit event	HIGH
DLP rule deleted or action changed to log	dlp.rule_deleted or dlp.rule_updated audit event	HIGH
Zero DLP detections during normal traffic volume	Monitoring — detection rate drops to 0	HIGH
SIEM alert: sensitive content pattern in proxy response logs	External SIEM / Prometheus alert	HIGH
Customer report: PII visible in responses that should have been redacted	Customer out-of-band report	HIGH
Unexpected is_enabled: false value on org DLP config	Admin spot-check	MEDIUM

Check current DLP scan_level for all orgs:

curl -s "https://api.arbitex.ai/api/v1/admin/dlp-rules?include_config=true" \
  -H "Authorization: Bearer $STAFF_TOKEN" | jq '.orgs[] | {org_id, scan_level, is_enabled}'

Check recent DLP config audit events:

curl -s "https://api.arbitex.ai/api/v1/admin/audit-logs/?action=dlp.config_updated&created_after=<window_iso>&limit=200" \
  -H "Authorization: Bearer $STAFF_TOKEN" | jq '.events[] | {timestamp, action, actor_id, org_id, metadata}'

Silent bypass detection — check detection rate in monitoring:

# Prometheus query for DLP detection rate (run against your Grafana/Prometheus instance)
# arbitex_dlp_detections_total should not be 0 during periods of non-trivial traffic
curl -s "https://<prometheus_host>/api/v1/query?query=rate(arbitex_dlp_detections_total[5m])"

---

2. Severity classification

This incident defaults to CRITICAL. There is no lower initial severity — any confirmed bypass of the 5-tier DLP pipeline is CRITICAL because:

• The blast radius is unknown until the audit window is analyzed
• Sensitive content may have already passed through
• Regulatory consequences begin at the moment the bypass was active, not the moment it was discovered

Condition	Severity	Action
Bypass confirmed, no traffic during window	CRITICAL	Restore enforcement, verify
Bypass confirmed, traffic present, no sensitive content detected	CRITICAL	Restore enforcement, full audit
Bypass confirmed, sensitive content in traffic logs	CRITICAL	Restore enforcement, notify affected orgs
Bypass confirmed, exfiltration via AI responses detected	CRITICAL	Restore enforcement, notify legal

Do not downgrade severity during the incident — only at close, once full scope is confirmed.

---

3. Immediate actions (first 15 minutes)

Step	Action	Command / UI
1	Declare incident	POST /api/staff/incident/declare
2	Freeze audit log	POST /api/staff/emergency/audit/freeze
3	Capture current DLP state snapshot	GET all org DLP configs
4	Lock DLP configuration via feature kill switch	POST /api/staff/emergency/killswitch
5	Revert scan_level to full where bypassed	PUT /api/v1/admin/dlp-rules per affected org
6	Restore deleted rules from backup or re-create	Platform admin portal or API
7	Verify DLP enforcement is active	Send test content

Step 1 — Declare incident:

curl -s -X POST "https://api.arbitex.ai/api/staff/incident/declare" \
  -H "Authorization: Bearer $STAFF_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "title": "DLP bypass detected — scan_level disabled or rules removed",
    "severity": "CRITICAL",
    "playbook": "IR-3"
  }' | jq '{incident_id: .id, ic_expires: .ic_window_expires}'

Step 2 — Freeze audit log immediately:

curl -s -X POST "https://api.arbitex.ai/api/staff/emergency/audit/freeze" \
  -H "Authorization: Bearer $STAFF_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"incident_id": "<incident_id>", "reason": "IR-3: Preserving evidence before DLP config restoration"}'

Step 3 — Capture current DLP state (before changing anything):

curl -s "https://api.arbitex.ai/api/v1/admin/dlp-rules?include_config=true&include_rules=true" \
  -H "Authorization: Bearer $STAFF_TOKEN" > "incident-<incident_id>-dlp-state-at-detection.json"

Save this file. It is forensic evidence of the state at detection time.

Step 4 — Lock DLP configuration:

curl -s -X POST "https://api.arbitex.ai/api/staff/emergency/killswitch" \
  -H "Authorization: Bearer $STAFF_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "feature": "dlp_config_write",
    "incident_id": "<incident_id>",
    "reason": "IR-3: DLP bypass active — locking config writes during investigation and restoration"
  }'

While this lock is active, no user or API call can modify DLP configuration. Restoration steps are performed by the IC using direct admin endpoints that bypass the lock.

---

4. Investigation

Reconstruct the exact sequence of changes, identify who made them, and determine the impact window.

Identify when the bypass began:

# Find the earliest dlp.config_updated or dlp.rule_deleted event that weakened enforcement
curl -s "https://api.arbitex.ai/api/v1/admin/audit-logs/?action=dlp.config_updated&created_after=<72h_ago_iso>&limit=500" \
  -H "Authorization: Bearer $STAFF_TOKEN" \
  | jq 'sort_by(.timestamp) | .[] | {timestamp, action, actor_id, org_id, metadata}'

Identify who made the changes:

# For each dlp change event, look up the actor
curl -s "https://api.arbitex.ai/api/v1/admin/audit-logs/?user_id=<actor_uuid>&action=dlp.config_updated&created_after=<bypass_start_iso>" \
  -H "Authorization: Bearer $STAFF_TOKEN" | jq '.events'

Investigate whether the actor was:

• A legitimate admin making an accidental change
• An account operating outside its normal pattern (possible compromise — cross-reference with IR-1 or IR-2)
• An API key that should not have had DLP admin permissions

Check if scan_level was changed per-org or globally:

# Per-org scope — check each org's scan_level history
curl -s "https://api.arbitex.ai/api/v1/admin/audit-logs/?action=dlp.config_updated&created_after=<bypass_start_iso>" \
  -H "Authorization: Bearer $STAFF_TOKEN" | jq '.events[] | {org_id: .metadata.org_id, old_value: .metadata.previous, new_value: .metadata.current}'

Determine what traffic passed through the bypass window:

# Count gateway requests during the bypass window for affected orgs
curl -s "https://api.arbitex.ai/api/v1/admin/audit-logs/?action=gateway.request&org_id=<org_uuid>&created_after=<bypass_start_iso>&created_before=<bypass_end_iso>" \
  -H "Authorization: Bearer $STAFF_TOKEN" | jq '{total: .total, events: [.events[] | {timestamp, metadata}]}'

Check which DLP rules were deleted or weakened:

# Get rule deletion events
curl -s "https://api.arbitex.ai/api/v1/admin/audit-logs/?action=dlp.rule_deleted&org_id=<org_uuid>&created_after=<bypass_start_iso>" \
  -H "Authorization: Bearer $STAFF_TOKEN" | jq '.events[] | {rule_id: .metadata.rule_id, rule_name: .metadata.name, action_was: .metadata.action}'

# Get rule action changes to 'log' (weakening without deletion)
curl -s "https://api.arbitex.ai/api/v1/admin/audit-logs/?action=dlp.rule_updated&org_id=<org_uuid>&created_after=<bypass_start_iso>" \
  -H "Authorization: Bearer $STAFF_TOKEN" | jq '.events[] | select(.metadata.changes.action == "log") | {rule_id: .metadata.rule_id, was: .metadata.previous.action, now: .metadata.current.action}'

Verify HMAC audit chain integrity:

curl -s -X POST "https://api.arbitex.ai/api/v1/admin/audit/verify" \
  -H "Authorization: Bearer $STAFF_TOKEN" \
  | jq '{valid, total_entries, errors}'

A chain break during this window (valid: false) is a strong indicator of intentional log tampering — escalate immediately.

---

5. Containment

Containment for a DLP bypass has two phases: stop any further bypass (done via the kill switch in step 3), and limit impact for any sensitive content already passed.

Restore scan_level to full for affected orgs:

# For each affected org
curl -s -X PUT "https://api.arbitex.ai/api/v1/admin/dlp-rules" \
  -H "Authorization: Bearer $STAFF_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "org_id": "<org_uuid>",
    "is_enabled": true,
    "scan_level": "full"
  }'

Restore deleted rules:

If specific rules were deleted, re-create them from the DLP config snapshot taken at step 3 (which shows what was present at detection), cross-referenced with the pre-bypass state from audit history.

# Re-create a deleted rule
curl -s -X POST "https://api.arbitex.ai/api/v1/admin/dlp-rules" \
  -H "Authorization: Bearer $STAFF_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "org_id": "<org_uuid>",
    "entity_type": "<entity_type>",
    "action": "block",
    "scan_level": "full",
    "name": "<rule_name>"
  }'

Revert rules changed to log action back to block:

curl -s -X PUT "https://api.arbitex.ai/api/v1/admin/dlp-rules/<rule_id>" \
  -H "Authorization: Bearer $STAFF_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"action": "block"}'

If the actor is a compromised account — contain the account:

If the DLP changes were made by a compromised account, follow IR-1: Compromised Tenant Account or IR-2: Compromised API Key in parallel.

---

6. Recovery

Validate DLP pipeline is functioning correctly:

Send test content that is known to trigger each tier of the DLP pipeline and confirm detection fires.

# Test Tier 1 (pattern match) — use a synthetic test SSN pattern
curl -s -X POST "https://api.arbitex.ai/v1/chat/completions" \
  -H "Authorization: Bearer $TEST_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-4o",
    "messages": [{"role": "user", "content": "Here is a test: 123-45-6789"}]
  }' | jq '{blocked: (.error.code == "dlp_block"), status: .error.type}'

A correctly functioning pipeline returns "dlp_block" for content matching an active block rule. If it does not, DLP is not enforcing correctly — do not lift the kill switch.

Confirm all rules restored:

curl -s "https://api.arbitex.ai/api/v1/admin/dlp-rules?org_id=<org_uuid>&include_rules=true" \
  -H "Authorization: Bearer $STAFF_TOKEN" | jq '{scan_level: .scan_level, is_enabled: .is_enabled, rule_count: (.rules | length)}'

Compare rule count against the pre-bypass count from the audit trail.

Lift DLP config lock (only after DLP is verified active):

curl -s -X POST "https://api.arbitex.ai/api/staff/emergency/killswitch" \
  -H "Authorization: Bearer $STAFF_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "feature": "dlp_config_write",
    "action": "restore",
    "incident_id": "<incident_id>",
    "reason": "IR-3: DLP enforcement verified active, config lock lifted"
  }'

Verify platform health:

curl -s "https://api.arbitex.ai/readyz" | jq '.'
curl -s "https://api.arbitex.ai/health/deep" | jq '{status, db, redis, providers}'

Tighten DLP admin permissions post-recovery:

Review which roles and API keys had permission to modify DLP configuration. If the bypass was enabled by a key or account with broader permissions than necessary, restrict DLP config write to admin-only roles and document this in the post-incident review.

---

7. Communication

Internal escalation (immediate)

Notify the Arbitex security team and management within 15 minutes of declaring CRITICAL:

[CRITICAL INCIDENT IR-3] DLP bypass detected

Incident ID: [INCIDENT_ID]
Detection time: [ISO_TIMESTAMP]
Bypass start (estimated): [BYPASS_START_ISO]
Affected orgs: [ORG_LIST or "under investigation"]
Root cause (preliminary): [scan_level change / rule deletion / unknown]
Actor (preliminary): [actor_id or "under investigation"]
Current status: DLP enforcement restored / restoration in progress
Data at risk: Under assessment

IC: [IC_NAME]
Next update in: 30 minutes

Customer notification (if traffic passed during bypass window)

For each org with traffic during the bypass window, notify the org’s security contact:

Subject: [SECURITY NOTICE] Temporary DLP policy gap — your organization

We are writing to inform you of a security incident affecting your Arbitex organization.

During the period [BYPASS_START] to [BYPASS_END], the DLP enforcement policy for
your organization was misconfigured, resulting in content not being scanned against
your active DLP rules.

What this means:
- Content submitted through the API during this window was not scanned or filtered
- No content was shared externally by Arbitex; this affects in-transit inspection only
- Your conversation history and audit log are complete and unmodified

What we have done:
- Detected the configuration error at [DETECTION_TIME]
- Restored DLP enforcement at [RESTORATION_TIME]
- Conducted HMAC chain verification confirming audit log integrity
- Are conducting a full investigation

What you should do:
- Review your API audit log for this time window (export attached)
- If you submitted content you consider sensitive during this window, please let us know
- Contact [email protected] with any questions

Incident reference: [INCIDENT_ID]

Legal notification (if sensitive content confirmed in bypass window)

If review of traffic during the bypass window reveals PII, PHI, financial data, or other regulated content passed through without DLP scanning:

• Notify Arbitex legal immediately
• Provide: affected org list, estimated record count, content categories, jurisdictions
• Begin breach notification timeline clock per applicable regulations (GDPR: 72 hours, HIPAA: 60 days, state laws vary)

Audit export for affected customers:

curl -s -X POST "https://api.arbitex.ai/api/v1/admin/audit/export" \
  -H "Authorization: Bearer $STAFF_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "org_id": "<org_uuid>",
    "start_date": "<bypass_start_date>",
    "end_date": "<restoration_complete_date>",
    "format": "json",
    "include_metadata": true
  }' -o "incident-<incident_id>-org-<org_uuid>-audit-export.json"

---

8. Post-incident

Evidence preservation checklist:

Item	Status
DLP state snapshot at detection saved to secure storage	[ ]
Full audit export (bypass window, all affected orgs) saved	[ ]
HMAC chain verification result recorded	[ ]
Bypass start and end timestamps confirmed from audit trail	[ ]
All traffic volume during bypass window quantified	[ ]
Content risk assessment (sensitive content present: yes/no) completed	[ ]
Incident timeline document created	[ ]
Affected customer notifications sent and logged	[ ]

Close incident:

curl -s -X POST "https://api.arbitex.ai/api/staff/incident/<incident_id>/close" \
  -H "Authorization: Bearer $STAFF_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "resolution": "DLP enforcement restored, all rules verified, permissions tightened",
    "root_cause": "<describe: accidental misconfiguration / compromised account / insider / unknown>",
    "bypass_duration_seconds": <seconds>,
    "traffic_during_bypass": <request_count>,
    "sensitive_content_confirmed": false
  }'

Lessons learned template:

• What change caused the bypass and was it intentional or accidental?
• How long was the pipeline in a weakened state before detection?
• Was silent bypass detection (monitoring for zero detections during normal traffic) already in place? If not, add it.
• Did the HMAC chain remain intact? If broken, investigate tampered audit events separately.
• Was the DLP admin permission scope appropriately narrow? What changes are needed?
• Should DLP scan_level: disabled be blocked entirely via a policy guard rather than just logged?

Related playbooks:

• If bypass was caused by a compromised account: follow IR-1: Compromised Tenant Account
• If bypass was caused by a compromised API key: follow IR-2: Compromised API Key
• If HMAC chain verification failed: follow IR-7: HMAC Chain Break Detected