kanros run

kanros run executes an evaluation suite. It is the workhorse of the CLI; every other subcommand exists either to feed run (init, migrate), to inspect what it produced (diff, share, cache), or to handle the red-team analogue (redteam run).

Usage

kanros run [OPTIONS]

Options

Flag	Default	Meaning
`-c, --config <PATH>`	`kanros.yaml`	Config file (YAML / TOML / JSON).
`--format <FMT>`	`terminal`	Output format. See list below.
`--no-cache`	off	Disable the response cache for this run.
`--cache-path <PATH>`	XDG cache dir	Custom cache database file.
`--plugins`	off	Enable WASM plugin support (`type: custom` and `type: wasm`).

Global flags --quiet, --verbose, and --json-logs apply to all subcommands.

Output formats

Format	Description
`terminal`	Human-readable table grouped by cell.
`json`	Pretty JSON; the full `RunSummary` struct.
`jsonl`	Newline-delimited JSON; one cell per line.
`junit`	JUnit XML for Jenkins / GitHub Actions test reporters.
`sarif`	SARIF 2.1.0 — feed into security dashboards.
`html`	Self-contained HTML report (one file, all assets inlined).
`csv`	One row per cell, header included.
`markdown`	Human-readable Markdown summary.
`github`	GitHub Actions workflow-command annotations (`::error::` / `::warning::`).

Exit codes

Code	Meaning
`0`	All cells passed their assertions.
`1`	At least one assertion failed (or a red-team probe produced a finding for `redteam run`).
`2`	Config error — could not parse, deserialize, or validate the supplied configuration file.
`3`	Runtime error — provider IO failure, cache IO error, or other recoverable infrastructure problem.
`64`	Usage error — reserved per `sysexits.h`; most clap parse errors currently exit with `2`.

CI/CD pipelines can branch on these codes to distinguish "tests failed" from "your config is broken" from "the provider is down".

Examples

Run the default config and emit a terminal table:

kanros run

Use a non-default config and write a JSONL stream:

kanros run --config configs/regression.yaml --format jsonl > results.jsonl

Disable cache and enable WASM plugins (typical CI invocation when you want to grade on a fresh response):

kanros run --no-cache --plugins

Use a CI-scoped cache database so the per-PR runner does not pollute the developer cache:

kanros run --cache-path ./.kanros-cache.db

What kanros run does

Load and validate the config. Config::load dispatches on file extension to YAML / TOML / JSON, then runs validate() (which enforces the "≥ 1 provider, ≥ 1 prompt, ≥ 1 test, family:model id shape" invariants).
Open the cache (unless --no-cache). The cache is keyed by (provider_id, request); a hit skips the provider call entirely. A miss runs the provider and stores the response.
Open the plugin host (if --plugins). WASM plugins are sandboxed by default — no network, no filesystem, no clock, no env.
Run the eval matrix. For each (prompt, test, provider) cell the runner renders the prompt, calls the provider with the configured retries/timeout, runs every assertion in tests[i].assert, and records the outcome.
Persist the run summary. The summary is stored in the cache database alongside cell-level results so kanros diff and kanros share can find it later. This happens even when --no-cache is set for the response cache.
Emit the report. The chosen format is written to stdout; logs go to stderr.

Performance knobs

The runner: block in kanros.yaml controls concurrency and timeouts:

runner:
  max_concurrency: 32          # global cap on in-flight provider calls
  rate_limit_per_provider: 0   # 0 = no per-provider RPS cap
  timeout_seconds: 120         # per-request timeout
  retries: 3                   # max retries on retryable errors

These are loaded into RunOptions and applied for the whole run.

Cancellation

kanros run runs the eval loop inside a tokio_util::sync::CancellationToken. SIGINT (Ctrl-C) cancels in-flight provider calls and writes a partial summary to stdout before exit. Cells already completed are reported as normal; cells in flight are reported as Error with a cancellation reason.

Caveats

The terminal format truncates long prompts and responses. Use json or jsonl if you need the full text.
The sarif format is intended for redteam run; passing it to kanros run will still produce valid SARIF but the result types are generic assertion failures rather than security findings.
--plugins enables the WASM plugin host. If the host fails to initialise (e.g. a wasmtime version mismatch), kanros warns and continues without plugins. type: custom and type: wasm assertions then return AssertionOutcome::Error.