IR-12: Insider Threat
Severity: CRITICAL
Scope: A current or former Arbitex staff member (support, engineering, or admin role) is suspected of malicious activity — unauthorized access to customer data, policy manipulation, evidence tampering, or data exfiltration. This is the most sensitive IR scenario in this library. It requires strict evidence chain preservation, covert investigation, and legal coordination before any visible action is taken against the suspected staff member.
IC role required for: audit freeze, bulk credential revocation, tenant isolation, emergency kill switch.
Related documentation: Security Operations — Audit Log Verification — Audit Chain Integrity — Credential Management
1. Detection
Section titled “1. Detection”Insider threat is detected through anomalous patterns in staff access logs, audit trail analysis, and external reports. Detection signals are often subtle and require correlation across multiple sources.
| Signal | Source | Notes |
|---|---|---|
| Staff member accessing customer data outside of any open support ticket | Staff audit trail | Compare staff access events against open support ticket timestamps |
| Admin operations performed outside normal business hours without incident context | Audit events | Check for admin. prefix events without a correlated incident ID |
| HMAC chain break correlating with staff session | POST /api/v1/admin/audit/verify |
Chain break during a known staff session window requires immediate investigation |
| Data appearing outside authorized channels | External report, DLP event | Customer data found in unauthorized location |
| Whistleblower report | HR, security team, anonymous channel | High trust — investigate immediately |
| Impossible travel on staff account | Staff auth log | Two logins from distant locations within an implausible time window |
| New FIDO/WebAuthn key registered outside of onboarding | Staff auth audit | Key registration without IT change ticket is anomalous |
| DLP rule modifications that loosen enforcement without change record | Audit trail | Policy weakening by a specific staff account without a tracked change request |
| Large audit exports performed by staff member outside incident context | audit.export events |
Bulk data collection by staff is a high-signal indicator |
2. Severity classification
Section titled “2. Severity classification”| Condition | Severity |
|---|---|
| Single anomalous access event — no confirmed unauthorized activity | MEDIUM — investigate covertly before escalating |
| Pattern of anomalous access across multiple customers or time periods | HIGH |
| Confirmed unauthorized access to customer data | CRITICAL |
| Evidence of policy manipulation, DLP rule changes, or audit tampering | CRITICAL |
| Confirmed data exfiltration | CRITICAL |
Do not escalate to visible action (account suspension, FIDO revocation) until CRITICAL is confirmed or legal has approved action. Premature visible action destroys the opportunity for covert evidence collection and may create legal complications. Consult legal before any action the staff member could observe.
3. Immediate actions (first 15 minutes)
Section titled “3. Immediate actions (first 15 minutes)”CRITICAL: Do not alert the suspected staff member. Limit knowledge of the investigation to the minimum personnel required — incident commander, one additional admin-role staff member, and legal. Do not use normal team communication channels (Slack, email threads) that the suspected staff member may have access to.
| Step | Action | Who |
|---|---|---|
| 1 | Declare incident with restricted scope — admin + legal only | IC (admin role) |
| 2 | Freeze audit log immediately to prevent evidence destruction | IC |
| 3 | Begin covert audit export of the suspect’s actions | IC |
| 4 | Verify HMAC chain integrity for the investigation window | IC |
| 5 | Contact legal before any visible action against the staff member | IC + Legal |
| 6 | Do not suspend, revoke, or alert the suspect until legal approves | IC |
Step 1 — Declare incident (restricted):
curl -s -X POST "https://api.arbitex.ai/api/staff/incident/declare" \ -H "Authorization: Bearer $STAFF_TOKEN" \ -H "Content-Type: application/json" \ -d '{ "title": "IR-12: Insider threat investigation — RESTRICTED", "severity": "CRITICAL", "playbook": "IR-12", "restricted": true }' | jq '{incident_id: .id, ic_expires: .ic_window_expires}'The restricted: true flag limits incident visibility to admin role only. Save the incident_id — do not share it in non-secure channels.
Step 2 — Freeze audit log immediately:
This is the single most important action. A staff member with engineering or admin access may be able to manipulate audit records if they detect the investigation. Freeze before they can act.
curl -s -X POST "https://api.arbitex.ai/api/staff/emergency/audit/freeze" \ -H "Authorization: Bearer $STAFF_TOKEN" \ -H "Content-Type: application/json" \ -d '{ "incident_id": "<incident_id>", "reason": "IR-12: Insider threat investigation — evidence preservation" }'A frozen audit log is readable but write operations are suspended. Freeze as early as possible.
Step 3 — Begin covert audit export:
Export the suspect’s complete audit trail before any action is taken. This is the primary evidence base.
curl -s -X POST "https://api.arbitex.ai/api/v1/admin/audit/export" \ -H "Authorization: Bearer $STAFF_TOKEN" \ -H "Content-Type: application/json" \ -d '{ "user_id": "<suspect_staff_user_id>", "start_date": "<investigation_window_start_date>", "end_date": "<today>", "format": "json", "include_metadata": true }' -o "ir12-<incident_id>-staff-audit-RESTRICTED.json"Store the export in a secure, access-restricted location — not a shared team drive.
Step 4 — Verify HMAC chain integrity:
curl -s -X POST "https://api.arbitex.ai/api/v1/admin/audit/verify" \ -H "Authorization: Bearer $STAFF_TOKEN" \ | jq '{valid, total_entries, errors}'If valid: false, the HMAC chain has been broken. The HMAC chain uses HMAC-SHA256 with a GENESIS sentinel (64 zero hex characters) anchoring the first entry. Any break indicates a deletion or modification of audit records — this is itself evidence of tampering and must be documented precisely.
4. Investigation
Section titled “4. Investigation”Collect complete evidence before taking any visible action. The investigation must be covert while the staff member remains active.
Build a complete timeline of the suspect’s staff actions:
# All actions by the suspect staff account — unlimited windowcurl -s "https://api.arbitex.ai/api/v1/admin/audit-logs/?user_id=<suspect_id>&limit=2000&created_after=<90_days_ago_iso>" \ -H "Authorization: Bearer $STAFF_TOKEN" \ | jq '.events | sort_by(.timestamp) | .[] | {timestamp, action, org_id, target_id, source_ip, metadata}'Check for unauthorized customer data access:
# All customer data access events (gateway requests, DLP scans, audit exports) by the suspectcurl -s "https://api.arbitex.ai/api/v1/admin/audit-logs/?user_id=<suspect_id>&action=customer.data_access&created_after=<window_start_iso>" \ -H "Authorization: Bearer $STAFF_TOKEN"
# Cross-reference with open support tickets — any access without a correlated ticket_id is unauthorizedcurl -s "https://api.arbitex.ai/api/v1/admin/audit-logs/?user_id=<suspect_id>&action=audit.export&created_after=<window_start_iso>" \ -H "Authorization: Bearer $STAFF_TOKEN" | jq '.events[] | {timestamp, org_id, metadata}'Check for policy and DLP rule modifications:
# Policy changes made by the suspectcurl -s "https://api.arbitex.ai/api/v1/admin/audit-logs/?user_id=<suspect_id>&action=policy.updated&created_after=<window_start_iso>" \ -H "Authorization: Bearer $STAFF_TOKEN" | jq '.events'
# DLP rule changes — check /api/v1/admin/dlp-rules modificationscurl -s "https://api.arbitex.ai/api/v1/admin/audit-logs/?user_id=<suspect_id>&action=dlp.rule_updated&created_after=<window_start_iso>" \ -H "Authorization: Bearer $STAFF_TOKEN" | jq '.events'If DLP rule changes are found that loosen enforcement, capture the before and after state. These may need to be reversed during containment.
Check for credential creation or rotation outside normal procedures:
# Credentials created or rotated by the suspectcurl -s "https://api.arbitex.ai/api/v1/admin/audit-logs/?user_id=<suspect_id>&action=credential.created&created_after=<window_start_iso>" \ -H "Authorization: Bearer $STAFF_TOKEN" | jq '.events[] | {timestamp, action, target_id, org_id}'Credentials created by a staff member outside of a documented provisioning workflow are a high-confidence backdoor indicator.
Check FIDO/WebAuthn key registration history:
# FIDO key events for the suspect's staff accountcurl -s "https://api.arbitex.ai/api/v1/admin/audit-logs/?user_id=<suspect_id>&action=fido.key_registered&created_after=<window_start_iso>" \ -H "Authorization: Bearer $STAFF_TOKEN" | jq '.events'FIDO key registration events outside of onboarding or a documented key replacement are anomalous. Multiple registered keys may indicate the suspect has registered a device unknown to IT.
Check for impossible travel on the staff account:
# Login events for the suspect — look for multiple IPs within short windowscurl -s "https://api.arbitex.ai/api/v1/admin/audit-logs/?user_id=<suspect_id>&action=staff.auth.login&created_after=<window_start_iso>" \ -H "Authorization: Bearer $STAFF_TOKEN" \ | jq '.events[] | {timestamp, source_ip, metadata.geoip_country}'5. Containment
Section titled “5. Containment”Only execute containment actions after legal approves visible action against the staff member. All containment steps are immediately observable — once started, the staff member will know an investigation is active.
| Action | When required | Command |
|---|---|---|
| Suspend staff account | Legal approval obtained | Staff UI — admin action (no API equivalent for staff account ops) |
| Revoke all FIDO/WebAuthn keys | Account suspension | Staff UI — FIDO key management |
| Terminate all active sessions | Account suspension | DELETE /api/v1/admin/users/{user_id}/sessions |
| Revoke API credentials held by the staff member | Suspect has direct API credentials | DELETE /api/v1/admin/credentials/{id} |
| Rotate any shared secrets the suspect had access to | Confirmed or suspected knowledge of shared secrets | Per secret type — see section 5 procedures |
| Isolate tenants accessed without authorization | Confirmed unauthorized customer data access | POST /api/staff/emergency/tenant/isolate |
Terminate all active sessions for the suspended staff account:
curl -s -X DELETE "https://api.arbitex.ai/api/v1/admin/users/<suspect_user_id>/sessions" \ -H "Authorization: Bearer $STAFF_TOKEN"Revoke any API credentials held by the staff member:
# List credentials owned by the suspectcurl -s "https://api.arbitex.ai/api/v1/admin/credentials?owner_id=<suspect_user_id>" \ -H "Authorization: Bearer $STAFF_TOKEN" | jq '.credentials[] | {id, type, created_at}'
# Revoke each credentialcurl -s -X DELETE "https://api.arbitex.ai/api/v1/admin/credentials/<credential_id>" \ -H "Authorization: Bearer $STAFF_TOKEN"Rotate any shared secrets the suspect had access to:
If the staff member had knowledge of shared operational secrets (database passwords, HMAC signing keys, Redis passwords, Key Vault access credentials), those must be rotated through the appropriate operational procedures. Consult with infrastructure team to identify which secrets were in scope for the suspect’s role and access level.
Isolate tenants where unauthorized access is confirmed:
# For each org where unauthorized access is confirmedcurl -s -X POST "https://api.arbitex.ai/api/staff/emergency/tenant/isolate" \ -H "Authorization: Bearer $STAFF_TOKEN" \ -H "Content-Type: application/json" \ -d '{ "org_id": "<org_uuid>", "incident_id": "<incident_id>", "reason": "IR-12: Insider threat — unauthorized staff access confirmed, tenant isolated pending customer notification" }'6. Recovery
Section titled “6. Recovery”Recovery addresses both platform integrity (reverting unauthorized changes) and operational continuity (rotating secrets, verifying chain integrity).
| Step | Action | Owner | Verification |
|---|---|---|---|
| 1 | Audit all actions taken by the suspect during investigation window | IC + Engineering | Complete action inventory |
| 2 | Reverse unauthorized policy changes | Engineering | Policy state matches last approved configuration |
| 3 | Reverse unauthorized DLP rule changes | Engineering | DLP rules match last approved state |
| 4 | Revoke any backdoor credentials created by the suspect | Engineering | Credential audit clean |
| 5 | Rotate all secrets the suspect had access to | Engineering + Infrastructure | New secrets active, old secrets invalid |
| 6 | Verify HMAC chain integrity post-remediation | IC | POST /api/v1/admin/audit/verify returns valid: true |
| 7 | Restore isolated tenants after customer notification | IC | Customers notified, tenant access confirmed restored |
| 8 | Platform health check | Engineering | /readyz, /health/deep pass |
Verify HMAC chain integrity after reverting unauthorized changes:
curl -s -X POST "https://api.arbitex.ai/api/v1/admin/audit/verify" \ -H "Authorization: Bearer $STAFF_TOKEN" \ | jq '{valid, total_entries, errors}'Any chain break after remediation that was not present before indicates the remediation actions themselves introduced an inconsistency — investigate before proceeding.
Restore isolated tenants:
curl -s -X POST "https://api.arbitex.ai/api/staff/emergency/tenant/restore" \ -H "Authorization: Bearer $STAFF_TOKEN" \ -H "Content-Type: application/json" \ -d '{ "org_id": "<org_uuid>", "incident_id": "<incident_id>", "reason": "IR-12: Customer notified, unauthorized access remediated, tenant access restored" }'Platform health check:
curl -s "https://api.arbitex.ai/readyz" | jq '.'curl -s "https://api.arbitex.ai/health/deep" | jq '{status, db, redis, providers}'7. Communication
Section titled “7. Communication”Legal and HR lead all external communication for this playbook. The technical team provides evidence and timeline documentation to legal. Do not communicate directly with affected customers, regulators, or media without legal approval.
Evidence package for legal team
Section titled “Evidence package for legal team”Prepare a structured evidence package for handoff to legal:
IR-12 Technical Evidence PackageIncident ID: [INCIDENT_ID]Prepared by: [IC NAME]Date: [ISO DATE]
1. Suspect staff account details - User ID: [STAFF_USER_ID] - Role: [support/engineering/admin] - Account active since: [DATE] - Account suspended at: [TIMESTAMP]
2. Audit export File: ir12-[INCIDENT_ID]-staff-audit-RESTRICTED.json Window: [START ISO] to [END ISO] Total events: [COUNT]
3. HMAC chain verification Result: [valid: true/false] Total entries verified: [COUNT] Errors (if any): [DESCRIBE]
4. Summary of unauthorized actions - Customer orgs accessed: [LIST] - Policy changes: [YES/NO — describe] - DLP rule changes: [YES/NO — describe] - Credentials created: [YES/NO — list IDs] - Data exports: [YES/NO — describe volume and content]
5. Timeline of key events [Chronological list from audit trail]Customer notification (legal determines timing and content)
Section titled “Customer notification (legal determines timing and content)”If customer data was accessed or exfiltrated, legal determines the notification requirements and drafts the customer communication. Provide legal with:
- List of affected org IDs
- Timeline of unauthorized access (from, to, duration)
- Nature of data accessed (conversation content, DLP scan results, credential data, configuration)
- Volume of data accessed (request count, estimated data size)
- Whether data left Arbitex systems (exfiltration confirmed vs. unauthorized viewing only)
Internal all-hands (after legal clears the statement)
Section titled “Internal all-hands (after legal clears the statement)”Once the staff member has been suspended and legal has cleared communication:
Subject: [SECURITY UPDATE] Staff security incident — account suspended
A staff account has been suspended following a security investigation.The former staff member's access has been fully revoked.
If you used any shared credentials or secrets in the past [N] weeks that[NAME / "this person"] may have had access to, contact the security teamimmediately for rotation instructions.
All affected customer organizations have been or are being notified perour legal obligations.
Questions: [email protected]8. Post-incident
Section titled “8. Post-incident”Evidence preservation checklist:
| Item | Status |
|---|---|
| Complete audit export (full investigation window) saved to secure, legal-hold storage | [ ] |
| HMAC chain verification result before and after remediation recorded | [ ] |
| FIDO key registration history for the suspect account exported | [ ] |
| All IP addresses used by the suspect’s account documented | [ ] |
| All unauthorized actions inventoried with affected org IDs | [ ] |
| Backdoor credentials found and revoked — full list recorded | [ ] |
| Policy and DLP changes reversed — before/after state documented | [ ] |
| Secrets rotation log — which secrets were rotated, when, by whom | [ ] |
| Chain of custody documentation for all evidence artifacts | [ ] |
| Incident timeline document created and signed by IC | [ ] |
Close the incident:
curl -s -X POST "https://api.arbitex.ai/api/staff/incident/<incident_id>/close" \ -H "Authorization: Bearer $STAFF_TOKEN" \ -H "Content-Type: application/json" \ -d '{ "resolution": "Staff account suspended, FIDO keys revoked, unauthorized changes reversed, affected customers notified, evidence package delivered to legal", "root_cause": "<describe: malicious intent / coercion / compromised staff account>", "data_exfiltrated": false }'Access review — use this incident to audit least-privilege across all staff:
Following containment, conduct a full audit of staff permissions:
# List all staff accounts and their rolescurl -s "https://api.arbitex.ai/staff/api/orgs" \ -H "Authorization: Bearer $STAFF_TOKEN" \ | jq '.users[] | {id, email, role, fido_keys: (.fido_keys | length), last_login}'Review the output for: staff members with more access than their role requires, staff accounts not used in the past 90 days (candidates for deactivation), staff members with multiple FIDO keys (each key should be documented).
Lessons learned template:
- What was the initial detection signal? How long was the activity ongoing before detection?
- Was the investigation successfully covert until legal approved visible action?
- Were there any access control gaps that allowed the access — i.e., could least-privilege have prevented or limited this?
- Was the HMAC audit chain sufficient to reconstruct the full scope of activity? Were there any gaps in audit coverage?
- Does staff onboarding or offboarding need to be updated based on this incident?
- Should access to customer data by staff require an open support ticket as a hard control (not just an audit expectation)?
Related playbooks:
- If customer data was accessed: follow IR-1: Compromised Tenant Account for each affected org
- If HMAC chain shows tampering: follow IR-7: HMAC Chain Break Detected
- If DLP rules were modified: follow IR-3: DLP Bypass Detected