IR-6: Model Tampering Suspected

Severity: HIGH (escalate to CRITICAL immediately if the tampered model served production traffic)

Scope: Model configurations, routing rules, or model manifests have been modified without authorization — SHA-256 manifest mismatch, unauthorized model additions, unexpected routing behavior, or model output inconsistent with expected capabilities.

IC role required for: emergency kill switch activation, bulk session termination.

Related documentation: Kill Switch — Kill Switch Operations — Model Routing Configuration — Provider Management — Audit Log Verification

---

1. Detection

Model tampering may be subtle — unauthorized model additions or routing rule changes may not immediately surface as errors. Monitor all of the following signal categories.

Signal	Source	Details
SHA-256 manifest mismatch during model loading	Platform startup log	model manifest hash mismatch log entry with expected vs. actual hash
Unauthorized model config changes in audit log	Audit events	model_config.created or model_config.updated events from unexpected user IDs or at unexpected times
Model producing outputs inconsistent with its declared capabilities	Customer report or automated output quality check	GPT-3.5-class model returning outputs that suggest a different model identity, or a model producing out-of-scope content
Unexpected model_config entries in database	Admin API or direct DB inspection	Models not present in the authorized model catalog for the tenant
Routing rules modified without a corresponding change request	Audit log	routing_rule.created or routing_rule.updated events not matching known change windows
Model returns unexpected model field in API response	API response inspection	The model field in a chat completion response does not match the model the caller requested
Customer reports unusual or dangerous model outputs	Customer escalation	Outputs suggesting jailbroken or modified model behavior

Escalate to CRITICAL immediately if any audit evidence, customer report, or automated detection confirms a tampered model was served to production traffic.

---

2. Severity classification

Classification	Criteria
CRITICAL	Tampered model confirmed served to production traffic
CRITICAL	Unauthorized model added and served to any tenant
HIGH (default)	SHA-256 manifest mismatch detected at load time, before any traffic served
HIGH	Unauthorized model config change detected in audit log, current traffic impact unknown
MEDIUM	Suspicious routing rule change — model itself not tampered, only routing potentially affected

When severity is initially classified as HIGH and investigation confirms production traffic was served, escalate to CRITICAL and notify Legal and Customer Success immediately.

---

3. Immediate actions (first 15 minutes)

Step	Action	Done
1	Declare the incident
2	Kill-switch the suspect model
3	Export recent model config audit events
4	Check whether tampered model was served to production traffic
5	Escalate to CRITICAL if production traffic confirmed

Step 1 — Declare the incident:

curl -X POST https://api.arbitex.ai/api/staff/incident/declare \
  -H "Authorization: Bearer $STAFF_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "title": "IR-6: Model tampering suspected — model_config_id <id>",
    "severity": "high",
    "playbook": "ir-6-model-tampering"
  }'

Step 2 — Kill-switch the suspect model immediately:

Disable the specific model config suspected of tampering. This stops all traffic to that model immediately, regardless of routing rules or fallback chains.

curl -X POST https://api.arbitex.ai/api/v1/admin/kill-switch/providers/<provider>/disable \
  -H "Authorization: Bearer $ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "reason": "security_event"
  }'

If the tampered model uses a known provider but a modified model ID (e.g., an unauthorized model added to an existing provider config), use the Staff emergency kill switch to disable at the feature level:

curl -X POST https://api.arbitex.ai/api/staff/emergency/killswitch \
  -H "Authorization: Bearer $STAFF_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "incident_id": "<incident_id>",
    "target": "model_loading",
    "reason": "Suspected model tampering — IR-6, preventing new model loads while investigating"
  }'

The model loading kill switch blocks the platform from loading any new model configuration from the database until the switch is cleared. Existing loaded model configs continue to serve traffic — use the provider kill switch in combination to stop traffic to specific models.

Step 3 — Export model config audit events:

curl -X POST https://api.arbitex.ai/api/v1/admin/audit/export \
  -H "Authorization: Bearer $ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "action": "model_config.updated",
    "start_date": "<30_days_ago>",
    "end_date": "<today>",
    "format": "jsonl"
  }'

Review the export for:

• Changes made outside of normal change windows
• Changes made by user IDs without model management permissions
• Automated changes with no corresponding change request

Step 4 — Determine whether tampered model served production traffic:

# Check chat completion audit events for the suspect model in the breach window
curl -X POST https://api.arbitex.ai/api/v1/admin/audit/export \
  -H "Authorization: Bearer $ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "action": "chat.completion",
    "start_date": "<tamper_window_start_date>",
    "end_date": "<detection_date>",
    "format": "jsonl"
  }'

If the export contains any chat completion events for the suspect model, production traffic was served. Escalate to CRITICAL and notify Legal and Customer Success immediately.

---

4. Investigation

SHA-256 manifest verification

Verify all model configs against their expected SHA-256 manifests. The platform stores a manifest hash for each registered model config that is checked at load time.

# List all model configs via admin API
curl https://api.arbitex.ai/api/v1/admin/model-configs \
  -H "Authorization: Bearer $ADMIN_TOKEN" | jq '.items[] | {id, provider, model_id, manifest_hash, is_active}'

Compare the returned manifest_hash values against the known-good manifest hashes from:

1. The deployment manifest stored in your infrastructure-as-code repository
2. The values from the last verified deployment (check CI/CD pipeline artifacts)

Any mismatch between the stored hash and the expected hash indicates the model config was modified after initial registration.

Routing rule analysis

Examine all active routing rules for unauthorized changes:

# List all routing rules
curl https://api.arbitex.ai/api/v1/admin/routing \
  -H "Authorization: Bearer $ADMIN_TOKEN" | jq '.rules[]'

# List routing rules modified in the suspect window
curl -X POST https://api.arbitex.ai/api/v1/admin/audit/export \
  -H "Authorization: Bearer $ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "action": "routing_rule.updated",
    "start_date": "<suspect_window_start_date>",
    "end_date": "<detection_date>",
    "format": "jsonl"
  }'

Cross-reference every routing rule change in the audit export with your change request records. Any routing rule change without a corresponding authorized change request is a finding.

Attribution — who made the changes?

For each suspicious model config or routing rule change found in the audit export, identify the actor:

# Look up the user who made a specific model config change
# The audit event will contain user_id — resolve it to an account:
curl "https://api.arbitex.ai/api/v1/admin/users/<user_id>" \
  -H "Authorization: Bearer $ADMIN_TOKEN"

If the user_id in the audit event belongs to:

• A service account or API key: trace which key was used and whether it has been compromised
• A known staff or admin account: determine if their session was compromised (cross-reference with IR-1 playbook)
• An unknown or deleted user ID: the user record may have been deleted after the change — treat as evidence of cover-up

Output impact assessment

If a tampered model served production traffic, assess the potential impact of its outputs:

1. Retrieve all conversation IDs from the tampered model’s chat completion audit events.
2. For each affected conversation, determine: which tenant it belongs to, whether the output contained unexpected or dangerous content, and whether the tenant has reviewed or acted on the output.
3. If the model was expected to be a restricted/safe model but outputs suggest capability beyond the declared model, treat as a potential jailbreak or model substitution attack.

---

5. Containment

Control	Action
Kill-switch suspect model	Complete — done in Immediate Actions
Lock model loading	Complete — done in Immediate Actions (if emergency KS activated)
Verify all remaining model configs	Run manifest hash verification across all active models
Audit all routing rules	Compare all active rules against last known-good state
Suspend model management permissions	Temporarily revoke model config write access for all non-IC users

Verify all remaining model configs have valid manifests:

# List all model configs and check manifest hash against expected values
curl https://api.arbitex.ai/api/v1/admin/model-configs \
  -H "Authorization: Bearer $ADMIN_TOKEN" \
  -H "Content-Type: application/json" | \
  jq -r '.items[] | "\(.id)\t\(.provider)\t\(.model_id)\t\(.manifest_hash)"'

Compare each returned manifest_hash against your deployment-time manifest file. Flag any config where:

• The hash does not match the expected value
• The hash field is null or empty (may indicate manifest verification was bypassed)
• The model ID or provider does not appear in the authorized model catalog

Suspend model management permissions (if investigation is ongoing):

# Terminate all active admin sessions to force re-authentication
# This also clears any compromised session tokens that may have been used to make changes
curl -X POST https://api.arbitex.ai/api/staff/emergency/killswitch \
  -H "Authorization: Bearer $STAFF_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "incident_id": "<incident_id>",
    "target": "admin_model_management",
    "reason": "Restricting model management during IR-6 investigation"
  }'

---

6. Recovery

Do not re-enable any model or remove the model loading lock until all verification steps below have passed.

Step	Action	Verification
Remove unauthorized model configs	Delete any model configs not in the authorized catalog	GET /api/v1/admin/model-configs returns only expected entries
Rollback tampered model configs to known-good state	Restore model config from last verified deployment manifest	Manifest hash matches expected value after restore
Re-verify all SHA-256 manifests	Run full manifest verification across all model configs	All configs return matching manifest hashes
Re-enable models after verification	Clear kill switch for verified models	Test completion returns expected model in response
Run validation test suite	Execute test prompts against re-enabled models	Outputs consistent with model capabilities
Lift model loading lock	Clear the model loading emergency kill switch	Monitor for 15 minutes after lifting

Restore a model config from the known-good manifest:

# Update a tampered model config back to its last known-good state
curl -X PUT https://api.arbitex.ai/api/v1/admin/model-configs/<model_config_id> \
  -H "Authorization: Bearer $ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "provider": "<provider>",
    "model_id": "<model_id>",
    "display_name": "<display_name>",
    "manifest_hash": "<known_good_sha256_hash>",
    "config": { /* config from last verified deployment */ }
  }'

Re-enable the model after verification:

curl -X POST https://api.arbitex.ai/api/v1/admin/kill-switch/providers/<provider>/enable \
  -H "Authorization: Bearer $ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"reason": "IR-6 resolved — model config verified against known-good manifest"}'

Validate model outputs with test prompts:

# Send a deterministic test prompt and verify output is consistent with expected model behavior
curl -X POST https://api.arbitex.ai/v1/chat/completions \
  -H "Authorization: Bearer $TEST_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "<restored_model_id>",
    "messages": [
      {"role": "user", "content": "What is 2 + 2? Reply with only the number."}
    ],
    "temperature": 0,
    "max_tokens": 5
  }'

Verify:

• The model field in the response matches the requested model
• The output is the expected deterministic response for the test prompt
• The response latency is consistent with the declared model tier

Close the incident:

curl -X POST https://api.arbitex.ai/api/staff/incident/<incident_id>/close \
  -H "Authorization: Bearer $STAFF_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "resolution": "Model configs rolled back to known-good. All SHA-256 manifests verified. Kill switch cleared after validation. Attribution identified."
  }'

---

7. Communication

Internal notification (immediate — within 15 minutes)

Send to: Engineering lead, Security lead, Customer Success lead.

Subject: [IR-6 ACTIVE] Model tampering suspected — severity HIGH

Incident ID: <incident_id>
Declared: <timestamp>
Current severity: HIGH [escalate to CRITICAL if production traffic confirmed]

Summary: Unauthorized modification of model configuration detected.
Suspect model(s): <provider>/<model_id> (config ID: <uuid>)
Kill switch activated at: <timestamp>

Current status: Investigating whether tampered model served production traffic.
Model loading locked pending investigation.

Next update: <30 minutes>
IC: <name>

Escalation to CRITICAL (if production traffic confirmed)

Send immediately upon confirmation, additional recipients: Legal, DPO.

Subject: [IR-6 ESCALATED TO CRITICAL] Tampered model served production traffic

Updating severity to CRITICAL for incident <incident_id>.

Confirmed: Tampered model <provider>/<model_id> served production traffic
during the window <start_time> to <end_time UTC>.

Affected tenants: [list or "under investigation"]
Estimated affected conversations: <count>

Immediate actions in progress:
- All affected tenant customer success contacts being notified
- Legal hold initiated for all forensic artifacts
- Legal team assessing breach notification obligations

Next update: <15 minutes>

Customer notification (when production traffic confirmed)

Send to each affected tenant’s designated security contact.

Subject: Security notification — model configuration incident

We are contacting you regarding a security incident that may have affected
your organization's use of the Arbitex platform.

Incident reference: <incident_id>
Window of potential impact: <start_time> to <end_time UTC>

What happened: We detected that a model configuration was modified without
authorization. Conversations in your organization that used this model
during the above window may have received responses from an unverified
model configuration.

Number of conversations potentially affected: <count>
Model affected: <display_name> (configured as <provider>/<model_id>)

What we have done:
- The affected model was taken offline at <detection_time>
- The model configuration has been restored to a verified known-good state
- We are conducting a full investigation into how the unauthorized modification
  occurred

What you should do:
- Review any automated workflows or decisions that used model output during
  the window <start_time> to <end_time UTC> and assess whether re-verification
  is appropriate
- If model outputs were used in a safety-critical context, we recommend
  re-running those inputs against the verified model configuration

We will provide a full incident report within 5 business days.

Arbitex Security Team
incident-<incident_id>@arbitex.ai

---

8. Post-incident

Task	Owner	Timing
Archive all audit exports and forensic artifacts	Security	Before incident close
Root cause analysis — how was the model config modified?	Engineering + Security	Within 72 hours
Review model management access controls	Engineering	Within 1 sprint
Implement manifest verification at request time (not just load time)	Engineering	Evaluate and schedule if not already in place
Customer impact report with affected conversation list	Security	Within 5 business days
Update model management documentation	Docs team	Within 1 sprint
Verify all model configs have manifests (identify any configs missing hashes)	Engineering	Within 24 hours of incident close

Manifest integrity improvement checklist — items to verify are in place after recovery:

Control	Expected behavior	Action if missing
SHA-256 manifest hash stored for every model config	manifest_hash non-null for all active configs	Compute and register manifest hashes for any configs missing them
Manifest hash verified at model load time	Log shows hash verification on startup	Investigate why verification did not catch the tampering
Model config changes require admin role	model_config.updated events only from users with admin role	Audit RBAC grants — remove model management permissions from non-admin users
Model config changes audit logged with actor identity	Audit export shows user_id for every change	Verify audit middleware is applied to model config endpoints
Routing rule changes audit logged	routing_rule.* events present in audit export	Verify audit middleware coverage for routing endpoints