Red Team Quickstart

This guide walks you through installing the Arbitex Red Team Framework, running your first scan, and setting up a quality gate. For background on what the framework tests and how scoring works, see the Red Team overview.

---

Prerequisites

• Python 3.10 or later
• A running Arbitex deployment — Outpost (self-hosted) or SaaS — with API access
• An API token with permissions to send requests through your Arbitex instance

---

Installation

1. Install the framework from PyPI:

   pip install arbitex-redteam

2. Verify the installation:

   arbitex-redteam --version

3. Export your API token as an environment variable:

   export ARBITEX_TOKEN=your_api_token_here

---

Run a full scan

Run all 152 probes against your deployment:

arbitex-redteam scan \
  --target-url https://your-arbitex-instance.example.com \
  --auth-token $ARBITEX_TOKEN

The scan runs all three probe categories — prompt injection, DLP evasion, and supply chain — plus the multi-turn campaigns. Depending on your deployment’s response latency, a full scan typically completes in 10–20 minutes.

Output summary:

Scan complete: 152 probes

Category results:
  prompt-injection    58/60 blocked  (ASR: 0.033)
  dlp-evasion         49/50 blocked  (ASR: 0.020)
  supply-chain        41/42 blocked  (ASR: 0.024)

Overall ASR: 0.026 (2.6%)
Status: PASS (threshold: 0.05)

Each probe result records whether the attack was blocked, the probe type, the attack payload, and the model response. Use --output results.json to capture the full result set.

---

Run a targeted scan

Scope the scan to a specific probe category:

arbitex-redteam scan \
  --target-url https://your-arbitex-instance.example.com \
  --auth-token $ARBITEX_TOKEN \
  --category prompt-injection

Available categories:

Value	Probes
prompt-injection	~60 probes: direct, indirect, instruction override, role manipulation, context switching
dlp-evasion	~50 probes: encoding tricks, format manipulation, chunked exfiltration, steganographic patterns, multilingual bypass
supply-chain	~42 probes: system prompt poisoning, tool definition manipulation, RAG injection, few-shot poisoning, plugin abuse

Targeted scans are useful during remediation when you want rapid feedback on a specific attack surface.

---

Run multi-turn campaigns

Multi-turn campaigns test whether attackers can build up to a bypass over an extended conversation, rather than in a single shot:

arbitex-redteam scan \
  --target-url https://your-arbitex-instance.example.com \
  --auth-token $ARBITEX_TOKEN \
  --campaigns crescendo,tap,skeleton-key

Available campaigns:

Campaign	Strategy
crescendo	Gradual trust-building over 5–15 turns, incrementally escalating toward the target behavior
tap	Tree of Attacks with Pruning — branching attack strategies that prune ineffective paths
skeleton-key	Attempts to establish a persistent jailbreak that survives conversation resets

Campaigns can be combined with --category flags to run multi-turn attacks against a specific probe surface.

---

Quality gate

After a scan completes, evaluate whether results meet your ASR threshold:

arbitex-redteam gate --threshold 0.05

• Exit code 0: ASR is at or below the threshold — deployment can proceed
• Exit code 1: ASR exceeds the threshold — deployment should be blocked

The gate command reads the results of the most recent scan in the current working directory. Pair it with --output on the scan command to control which result file is evaluated.

Recommended thresholds:

Threshold	Use case
0.02	Strict — high-security environments
0.05	Recommended — production deployments
0.10	Lenient — initial deployment while tuning

See the CI/CD integration guide for per-category threshold configuration and full pipeline setup.

---

Generate reports

Export scan results in a format suited to your workflow:

arbitex-redteam report --format json --output results.json

Supported formats:

Format	Use case
json	Machine-readable results for custom tooling
html	Human-readable report for stakeholder sharing
markdown	Documentation or PR comment integration
sarif	GitHub Security tab integration

For compliance evidence export, see the compliance mapping page.

---

Configuration file

For persistent settings across runs, create .arbitex-redteam.yml in your project root:

target_url: https://your-arbitex-instance.example.com
auth_token_env: ARBITEX_TOKEN

categories:
  - prompt-injection
  - dlp-evasion
  - supply-chain

campaigns:
  - crescendo
  - tap

threshold: 0.05

output:
  format: json
  path: ./results/redteam-results.json

When a config file is present, CLI flags override config file values. The auth_token_env key references an environment variable name rather than embedding the token value directly — keep secrets out of config files committed to source control.

---

Next steps

• Compliance mapping — map scan results to OWASP LLM Top 10, MITRE ATLAS, NIST AI RMF, and EU AI Act
• CI/CD integration — automate red team testing as a deployment quality gate
• Red Team overview — probe library details, campaign strategies, and scoring methodology