Tier 0 Pre-filter

The Tier 0 pre-filter is a lightweight TF-IDF + logistic regression classifier that runs as Tier 0 of the 5-tier DLP pipeline, before Tiers 1–4. Its sole purpose is to detect obvious prompt injection attempts and short-circuit processing with a sub-10ms decision, eliminating the latency cost of the regex, NER, and DeBERTa stages for clearly malicious inputs.

Overview

The full DLP pipeline (Tier 1 regex → Tier 2 NER → Tier 3 DeBERTa) is accurate but adds meaningful latency to each request. Many prompt injection attempts are structurally obvious — they share surface-level patterns that a fast linear classifier can identify with high confidence before any deep processing begins.

Tier 0 sits at the very front of the intake pipeline. When it fires with sufficient confidence, the remaining tiers are skipped entirely and the request is blocked immediately. When it is uncertain or disabled, processing falls through to the full pipeline unchanged.

How It Works

Incoming request
      |
      v
 [Tier 0: TF-IDF + LR]  <-- sub-10ms
      |
  should_block?
   /        \
 YES          NO
  |            |
  v            v
BLOCK    [Tier 1: Regex]
(skip         |
 T1/T2/T3)   [Tier 2: NER]
              |
         [Tier 3: DeBERTa]
              |
           decision

The classifier calls model.predict_proba([text])[0] and reads the probability at class index 1 (the injection class). Two thresholds govern the result:

• Detection threshold (fixed at 0.50): If prob >= 0.5, the request is flagged as a likely injection (is_injection = true). This is informational only.
• Block threshold (configurable, default 0.95): If is_injection is true and prob >= threshold, should_block is set and the full pipeline is bypassed.

The separation of these thresholds lets you observe detections below the block threshold in your audit stream without acting on them.

The model file is loaded lazily from backend/data/tier0_model.joblib on first use. If the file is absent, the classifier fails open and every request passes through to the full pipeline. The model is trained by scripts/train_tier0.py and carries the internal version tag tfidf_lr_v1.

Configuration

Environment variables

Variable	Default	Description
TIER0_PROMPT_INJECTION_ENABLED	true	Master toggle. Set to false to disable Tier 0 entirely.
TIER0_CONFIDENCE_THRESHOLD	0.95	Minimum probability required to block a request.
TIER0_MODEL_PATH	(built-in path)	Absolute path to an alternate .joblib model file.

Environment variables are evaluated at startup. Changes require a process restart.

System config (runtime)

The confidence threshold can also be updated at runtime via the system config API without restarting the platform. This takes precedence over the environment variable.

Read the current threshold:

curl -s -H "Authorization: Bearer $ADMIN_TOKEN" \
  https://api.arbitex.ai/api/v1/admin/config/tier0_confidence_threshold

Update the threshold:

curl -s -X PUT \
  -H "Authorization: Bearer $ADMIN_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"value": "0.90"}' \
  https://api.arbitex.ai/api/v1/admin/config/tier0_confidence_threshold

The value is stored as a string and parsed as a float at evaluation time. Valid range is 0.50 to 1.00. Values outside this range are rejected with HTTP 422.

Disable Tier 0 at runtime:

The master toggle is an environment variable only and requires a restart. There is no runtime config key for it. To disable without restarting, raise the threshold to 1.00 — no request will ever reach that probability.

Confidence Thresholds

Choosing the right block threshold involves a precision/recall tradeoff:

Threshold	Behavior
0.99	Extremely conservative. Only the most unambiguous injections are blocked. Very low false-positive rate.
0.95	Default. Recommended starting point for production.
0.90	More aggressive blocking. Suitable for high-risk deployments. Monitor false positives.
0.80	Catches a broader range of injection patterns. Higher false-positive risk; use only with active monitoring.
< 0.80	Not recommended for production. Likely to block legitimate requests.

The fixed detection threshold of 0.50 is not configurable. All requests with prob >= 0.50 appear in the audit log with their confidence score, even if should_block is false. Use this stream to tune your block threshold before tightening it.

Fail-Open Behavior

Tier 0 is designed to never cause a false outage. If anything goes wrong, processing continues to the full pipeline:

• Model file missing: Classifier is not loaded. All requests pass through.
• Runtime exception during classification: A warning is logged, the exception is swallowed, and the request continues. should_block is treated as false.
• Model file present but corrupt: joblib.load() raises an exception at startup. The classifier is marked unavailable and all requests fall through.

This means a broken Tier 0 configuration degrades gracefully — requests are not lost, they are simply evaluated by the slower full pipeline. Monitor the tier0_decisions_total OTel counter to detect a sudden drop to zero (indicating the classifier has stopped operating).

Audit Events

Every Tier 0 block generates an audit log entry with event_type: dlp.tier0_block. Pass-through decisions are not logged individually; they are counted in OTel metrics only.

Sample audit log entry:

{
  "event_type": "dlp.tier0_block",
  "timestamp": "2026-04-01T14:22:03.841Z",
  "session_id": "sess_8f3a2c1d",
  "user_id": "usr_00112233",
  "confidence": 0.9731,
  "threshold": 0.95,
  "text_snippet": "Ignore all previous instructions and...",
  "model_version": "tfidf_lr_v1",
  "action": "block",
  "entity_type": "prompt_injection",
  "detector_name": "tier0"
}

The text_snippet field contains the first 120 characters of the input. Full input text is not stored in the audit record; the session replay index retains the full content if session auditing is enabled.

Tier 0 blocks appear in the DLP section of the audit export alongside Tier 1/2/3 findings. Filter by event_type = dlp.tier0_block to isolate them.

Observability

Tier 0 records an OpenTelemetry counter on every decision:

Metric: tier0_decisions_total

Labels:

Label	Values	Description
decision	block, pass	Whether the request was blocked or passed through.
confidence_bucket	high, medium, low	high >= 0.95 / medium >= 0.70 / low < 0.70

Use these labels to build dashboards that show:

• Block rate over time (decision=block)
• Distribution of confidence scores across traffic (confidence_bucket breakdown)
• False-positive investigation triggers (spike in decision=block with confidence_bucket=medium)

A healthy deployment typically shows the majority of traffic in decision=pass, confidence_bucket=low (benign requests score near zero). A spike in confidence_bucket=high combined with decision=block indicates active injection attempts.

Model Training

The model is trained offline using scripts/train_tier0.py. It is not retrained automatically. To update the model:

1. Collect labeled examples (injection vs. benign) in the format expected by the training script.
2. Run scripts/train_tier0.py with your dataset to produce a new .joblib file.
3. Validate the new model on a held-out test set. Check precision at your target threshold before deploying.
4. Replace backend/data/tier0_model.joblib with the new file, or set TIER0_MODEL_PATH to point to the new location.
5. Restart the platform service to load the new model.

The current production model is tfidf_lr_v1. The version tag appears in audit log entries and does not change automatically when the model file is replaced — update the tag in tier0_classifier.py when training a materially different model.

Limitations

• English-centric: The TF-IDF vocabulary is trained primarily on English-language injection examples. Detection rates for non-English injections may be lower.
• Keyword dependence: The model relies on surface-level token patterns. Adversarially rephrased or encoded injections (base64, Unicode homoglyphs, deliberate misspellings) are more likely to bypass Tier 0 and must be caught by the full pipeline.
• No semantic understanding: Tier 0 cannot reason about intent. It makes a binary call based on learned token distributions. Novel injection techniques not represented in training data will pass through.
• Model staleness: The model does not update itself. As injection techniques evolve, periodic retraining is required to maintain detection rates.
• Fixed detection threshold: The 0.50 detection threshold for is_injection is not configurable. Requests near the boundary are logged but not blocked unless they also exceed the block threshold.

Tier 0 is a first line of defense, not a complete solution. Tiers 1–4 of the DLP pipeline remain active for all requests that Tier 0 passes.

Related Pages

• DLP Pipeline Configuration — full 5-tier pipeline architecture and per-tier tuning
• DLP Configuration — policy actions, entity types, and enforcement modes
• DLP Confidence Calibration — threshold tuning across all DLP tiers