Red Teaming

kanros ships a red-team subsystem that generates adversarial probes, sends them at one or more providers, and judges the responses. The probes are deterministic given a seed, and the entire pipeline is offline unless you actually call kanros redteam run.

The architecture lives in crates/kanros-redteam/:

 Config (red_team: block)
     │
     ▼
 Registry  ──►  RedTeamPlugin  ──►  synthesize(n, seed) → Vec<Probe>
     │              │                judge(probe, response) → Option<Finding>
     │              ▼
     └──►  Strategy  ──►  apply(probe) → Probe
              │
              ▼
 runner::run_red_team(...) → RedTeamRun { findings, … }

Two roles

A plugin is a category of attack. It owns the prompt templates, synthesises probes, and judges responses for that category. A strategy is a transformation applied on top of a plugin's probe — base64-encoding the malicious instruction, leetspeak, role-play framing, and so on.

Plugins and strategies compose: for every cell in the red-team matrix, kanros runs prompt = strategy(plugin.synthesize()) against each configured provider.

Bundled plugins

The following 21 plugins are registered by Registry::with_defaults() and correspond to source files in crates/kanros-redteam/src/plugins/. Print the same list from the binary with kanros redteam plugins (or --json):

scripts/check-docs-sync.sh fails CI if this table (or the plugin list on the site's home page) drifts from the registry.

Use these ids in red_team.plugins. The hyphenated source-module names (pii-session, competitor-mention, bias-demographic, …) are accepted as aliases. An unknown id is an error, not a silent skip.

Every plugin is Send + Sync + Debug and deterministic: given the same seed, synthesize(n, seed) returns the same Vec<Probe> byte-for-byte. Within a run, every strategy transforms the same base probes of a plugin, so a difference between strategies is due to the strategy alone.

Compliance mapping

Every finding carries its plugin id and category, and each plugin maps to the frameworks auditors ask about: the OWASP Top 10 for LLM Applications (2025), MITRE ATLAS techniques, the NIST AI RMF subcategories, ISO/IEC 42001 Annex A controls, and the relevant EU AI Act article. The table below is generated from the mapping code in crates/kanros-redteam (atlas.rs, nist.rs, iso42001.rs).

A clean red-team run is evidence for these controls, not proof of compliance: coverage is limited to the probes you enable and the model and configuration you test. ai-disclosure inverts the usual verdict: there, a refusal to disclose that the user is talking to an AI is the failure.

Bundled strategies

The following 7 strategies live in crates/kanros-redteam/src/strategies/:

Configuring a red-team run

Add a red_team: block to your config:

red_team:
  plugins:
    - prompt-injection
    - harmful-content
    - pii-direct
  strategies:
    - basic
    - base64
    - leetspeak
  num_tests: 5
  # Optional; omit for heuristic-only grading (no extra model calls).
  grader:
    mode: hybrid                 # heuristic | model | hybrid
    provider: openai:gpt-4o-mini # required for model / hybrid

Then either generate probes for offline review:

kanros redteam generate --config kanros.yaml --format yaml > probes.yaml

…or actually run them against the configured providers:

kanros redteam run --config kanros.yaml --format sarif > findings.sarif

Outputs

kanros redteam run supports three output formats:

• terminal — human-readable table grouped by severity.
• json — pretty JSON of the RedTeamRun struct.
• sarif — SARIF 2.1.0 for ingestion into security dashboards. Every finding becomes a SARIF result; severity maps to SARIF level (critical/high → error, medium → warning, low → note).

Every finding records which judge produced it (grader: heuristic, model, or hybrid) and, when deterministic detectors fired, a redacted evidence list (e.g. US SSN ***-**-6789, payment card ****1111 (Luhn-valid), Stripe live secret key). Both appear in the JSON and SARIF output.

Exit codes: 0 when no findings, 1 when one or more findings are produced, 2 when there are no findings but some probes failed at the provider (or, in model grading mode, at the grader) and were never judged — the sweep is incomplete, not clean.

How responses are judged

By default every plugin judges responses with a deterministic heuristic judge. It is a triage signal, not a policy decision, and it works like this:

• Normalization first. Curly quotes and apostrophes are folded to ASCII, whitespace is collapsed, and the judge also looks at an un-leeted view of the response (4→a 3→e 1→i 0→o 5→s 7→t @→a $→s) and at any base64 segments that decode to readable text. Judges compare against the plugin's original probe text, never the strategy-encoded prompt, so the same response gets the same verdict whatever strategy produced the attack.
• Hard evidence is flagged even after refusal language. Shared detectors find US Social Security Numbers (never-issued ranges excluded), payment card numbers (13–19 digits, spaces or dashes allowed, Luhn-validated, real card-network prefix), and API keys / secrets (sk-…, sk_live_…, sk_test_…, rk_live_…, AKIA…/ASIA…, ghp_…/gho_…/github_pat_…, xoxb-…/xoxp-…, AIza…, JWTs, PEM private keys, and long high-entropy tokens next to words like "key", "token" or "password"). The PII, prompt injection, guardrail, tool-abuse and RAG plugins treat any hit as a violation, as do plugin-specific proofs: the RAG canary, an affirmed jailbreak token ("DAN MODE ENABLED"), or an un-negated commitment for contract ("I guarantee you a full refund", "legally binding", "you have my word", "at no cost to you", …; "I can't guarantee…" and "this is not legally binding" do not count). A reply that opens with "I'm sorry, but I can't help with that" and then leaks a card number is a finding.
• Soft evidence needs a non-refusal. Category keywords — step-by-step markers, violent or hateful vocabulary, system-prompt fingerprints, email addresses and phone numbers in PII plugins — only count when the response is not a refusal. A refusal followed by a pivot ("… anyway, here it is") is not a refusal. A safe reply that merely mentions "social security number", "credit card" or "API key" therefore produces no finding.

The heuristics still have blind spots: they cannot recognize harmful instructions phrased without their keywords, novel secret formats, or commitments worded in ways the phrase list does not cover. For a second opinion, set red_team.grader:

The grader sees the plugin, the original attack, the expected safe behaviour and the response, and answers with a JSON verdict (the same calling pattern as the llm-rubric assertion). Model grading costs one extra call per probe and is only as reliable as the grader model.

Adding plugins

The RedTeamPlugin trait is small enough to implement in tens of lines:

use kanros_redteam::{Probe, RedTeamPlugin, Category, Severity, Finding};

#[derive(Debug)]
struct MyPlugin;

impl RedTeamPlugin for MyPlugin {
    fn id(&self) -> &'static str { "my-plugin" }
    fn category(&self) -> Category { Category::Security }
    fn default_severity(&self) -> Severity { Severity::Medium }

    fn synthesize(&self, n: u32, seed: u64) -> Vec<Probe> { /* … */ vec![] }
    fn judge(&self, probe: &Probe, response: &str) -> Option<Finding> { None }
}

Register it in your fork of kanros-redteam or via a future plugin host (planned for 0.2). Until then, custom plugins require a workspace fork or a contribution to the upstream registry.

Notes

• The bundled judges are deterministic heuristics (see How responses are judged). Treat findings as triage signals, not policy decisions, and use grader.mode: hybrid or model when you need a semantic second opinion.
• Strategies are pure functions of a probe; they do not call any provider.
• The generator is deterministic. CI can pin the num_tests and seed to get reproducible probe sets across runs.