# Changelog

All notable changes to `kanros` are recorded here. The format follows
[Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project
adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

## [0.1.0-rc.1] - Unreleased

First release candidate for v0.1.0 — **prepared on branch
`release/v0.1.0-rc.1-prep`, not yet tagged**. (An earlier prep cut was
dated 2026-05-30, but the tag was never pushed; this section now also
covers the work that landed since.) Trips every workstream exit gate in
`WORKSTREAMS.md`: 27 providers, 44 assertions across 7 families, the
red-team plugin/strategy fleet, the WASM + JS plugin hosts, RAG metrics,
agent-trace evaluation, the TUI and Web UI shells, a self-hostable share
endpoint, and a cargo-dist release pipeline. PRs #1 through #121
contributed to this release, produced by a fleet of worktree-isolated
sub-agents under orchestrator review.

The `v0.1.0-rc.1` tag will be cut when this branch merges. If no
regressions surface during the 7-day soak after the tag lands, the
contents of this changelog will be re-stamped as `0.1.0` GA on the
release tag.

### Added

#### Providers (27 — workstream B)

- **echo** — built-in identity / upper / lower / reverse (no credentials).
- **openai** — Chat Completions, embeddings, vision (#15).
- **anthropic** — Messages API including tool use and vision (#14).
- **azure-openai** — Azure OpenAI deployment routing with API version pinning (#25).
- **ollama** — local chat + embeddings, batched via parallel requests (#13).
- **http** — generic OpenAI-compatible HTTP with `json_path` response mapping (#22).
- **llama-cpp**, **lm-studio**, **vllm** — local self-hosted endpoints (#54).
- **subprocess** — shell-out provider with explicit env-var forwarding (#43, #56).
- **bedrock** — AWS Bedrock Converse API, SigV4-signed (#18).
- **browser** — Chromium DevTools Protocol via `chromiumoxide` (#55).
- **cloudflare** — Workers AI chat + embeddings (#45).
- **cohere** — Command + Embed family (#29).
- **deepseek** — DeepSeek API (#37).
- **fireworks** — Fireworks AI inference (#32).
- **groq** — Groq fast inference (#24).
- **huggingface** — Inference API (#44).
- **mistral** — La Plateforme + Codestral (#28).
- **openrouter** — OpenRouter model routing (#30).
- **perplexity** — Perplexity Sonar (#36).
- **replicate** — async prediction polling (#42).
- **together** — Together AI (#33).
- **voyage** — Voyage embeddings-only (#41).
- **vertex** — Google Gemini / Vertex AI (#16, key-auth + service-account).
- **websocket** — generic WS provider via `tokio-tungstenite` (#55).
- **xai** — xAI Grok (#40).

#### Assertions (workstream C)

- **Deterministic** (22): `equals`, `not-equals`, `contains`, `icontains`,
  `not-contains`, `starts-with`, `ends-with`, `regex`, `not-regex`,
  `contains-all`, `contains-any`, `contains-none`, `contains-json`,
  `is-json`, `is-sql`, `is-xml`, `json-schema`, `length`, `word-count`,
  `cost`, `latency` (#39).
- **Similarity**: `similarity` (cosine via `ProviderRef`), `levenshtein` (#11).
- **Model-graded**: `llm-rubric`, `factuality`, `model-graded-closedqa`,
  `answer-relevance` — robust JSON-island parsing with one retry (#21).
- **NLP metrics**: `bleu`, `rouge-n`, `meteor` (#26).
- **Code**: `javascript` (boa-sandboxed), `python` (subprocess), `webhook` (#34, #56).
- **Composite**: `assert-set`, `select-best` (#39).
- **Safety**: `moderation`, `classifier` (#31).
- **RAG**: Ragas-style decomposed metrics — context-precision,
  context-recall, faithfulness, answer-relevance (#38, #69).
- **Custom (WASM + JS)**: WASM Component Model host (#23) and embedded
  JavaScript grader runner (#67); both wired into the runner with
  capability-gated sandboxing.

#### Red-team (workstream E)

- Trait layer, plugin/strategy registry, and `kanros redteam` CLI bootstrap (#35, #49).
- Plugins: `prompt-injection`, `pii:direct`, `pii:session`, `pii:api-query`,
  `bias`, `hallucination`, `competitor`, `contract`, plus four
  harmful-content category plugins (#60, #64, #65).
- Strategies: `basic`, `base64`, `crescendo`, `roleplay`, `prompt-injection` (#63).

#### Runner & cache (workstream D)

- Bounded global concurrency via `tokio::sync::Semaphore` (#19).
- Per-provider rate limiting (sliding 1-second window).
- Retries with exponential backoff + jitter; respects `Retry-After`.
- Per-request timeout per `config.runner.timeout_seconds`.
- Ctrl+C cancellation — in-flight cells drain, partial `RunSummary` preserved.
- `run_with_cancellation(config, CancellationToken)` API.
- SQLite-backed response cache with blob compression (#27).
- PluginHost wired into the runner; `--plugins` CLI flag (#65 follow-up).

#### CLI (workstream G/J)

- `kanros run` — eval suite with config-file or CLI overrides.
- `kanros validate` — parse and check without running.
- `kanros schema` — print the JSON Schema for `kanros.yaml`.
- `kanros version` — binary + trait-layer versions.
- `kanros init` — scaffold a new project (#48).
- `kanros completions <shell>` — bash, zsh, fish, powershell, elvish (#48).
- `kanros auth set/get/delete` — system-keyring credential storage (#48).
- `kanros migrate` — promptfoo config migration (#57).
- `kanros generate tests --from spec.md` — model-graded test synthesis (#50).
- `kanros redteam generate` — synthesize red-team probes from policy (#49).
- `kanros diff` — compare two run summaries (#59).
- `kanros share` — upload run summaries to the optional share endpoint (#73).
- Polished help text, completion install hints, version output (#72).

#### RAG & agent eval (workstream I)

- `kanros-rag` crate: chunking, retrieval-trace evaluation, citation
  attribution, Ragas-style metrics (#38, #69).
- `kanros-agent` crate: agent-trace evaluation primitives — tool-call
  sequence assertions, plan-vs-execution drift (#66).

#### Datasets (workstream G)

- CSV, JSONL, HuggingFace dataset loaders with row-level
  variable-mapping and column-renaming (#62).

#### Templating (workstream G)

- `kanros-templating` minijinja-backed `Engine`, URI loaders for
  `file://`, `https://`, `hf://` (#61).

#### Reports & outputs (workstream G)

- Formatters: `terminal`, `json`, `jsonl`, `junit`, `sarif`, `html`
  (single self-contained file), `markdown`, `csv`, GitHub annotations
  (#12, #17, #46, #47).
- `generate_schema()` — JSON Schema draft 2020-12 for `kanros.yaml` (#47).

#### Plugin hosts (workstream H)

- WASM Component Model host via `wasmtime` with capability-based
  sandbox (default-deny FS/network/clock/env, opt-in `allow_http`) (#23).
- Embedded JavaScript grader runner via `boa_engine` (#67).
- Sample plugin at `examples/plugins/contains-len/`.

#### Web UI (workstream F/J)

- `kanros-tui` — ratatui live RunSummary view (#68).
- `kanros-webui-server` — axum + rust-embed Web UI server for the
  RunSummary viewer (#71).

#### Sharing

- `kanros-share` upload flow + optional self-hostable endpoint (#73).
- SHA-256 content addressing; auth tokens via `OsRng`.

#### Release pipeline (workstream J)

- `cargo-dist` 0.28 across 6 targets: macOS arm64/x86_64, Linux
  glibc/musl, Linux arm64 (cross), Windows x86_64 (#20, #70).
- `release.yml` triggers on `v[0-9]+.*` tags or `workflow_dispatch`.
  Produces tarballs/zips, shell + PowerShell installers, SHA256 sums.
- musl static linking via `+crt-static`; cargo-dist auto-bundles
  README/LICENSE/CHANGELOG.

#### Benchmarks (workstream K)

- `kanros-bench` Criterion harness covering the §11 performance bars:
  assertion eval, cell-pool concurrency, cache I/O, config parse (#74).

#### Landed after the 2026-05-30 prep cut (PRs #101–#121)

- **Web UI live tail** — cell completions streamed over Server-Sent
  Events (#101), rich cell detail + source-config viewer + client-side
  filters (#113), and a side-by-side run-diff page (#114).
- **Plugin marketplace** — `kanros-plugin-registry` discovery + install
  client for registry.kanros.dev (#103).
- **Release-profile binary-size tuning** — `lto = "fat"`,
  `codegen-units = 1`, `panic = "abort"`, `strip = "symbols"`; the
  release binary lands at 23.23 MB stripped, under the 30 MB §11 bar
  (#105). §11 performance bars verified vs promptfoo (#104).
- **Share server** — `kanros-share-server` backend for share.kanros.dev
  (#106), extended with auth, multi-user, and project ownership (#118).
- **Security: wasmtime 44.0.2** — `kanros-plugin` bumped from
  wasmtime 27.0.0 to 44.0.2, clearing the open RUSTSEC advisories
  (#107). Sandbox-escape proof demo in `examples/wasm-sandbox/` (#109).
- **CLI P2 omnibus** — `kanros eval` alias, promptfoo-compatible output,
  `kanros cache gc`, `kanros doctor`, and `kanros run --watch` (#108).
- **Keyring fallback for provider credentials** — providers resolve
  API keys from the system keyring when env vars are absent (#110).
- **Core thaw** — `prompt` / `completion` / `usage` / `cost` / `retry` /
  `config_source` fields added to the result types under the
  post-freeze review process (#111).
- **GitHub Action** — kanros packaged as a GitHub Action under
  `integrations/` (#115).
- **Runner** — budget caps, provider fallback chains, and retry
  visibility in the run summary (#116).
- **Agent loop** — a real agent loop with tools, a scripted provider
  for deterministic tests, and CLI wiring (#117).
- **VSCode extension** — a real extension under `editor/vscode/`
  (schema-backed completion/validation) (#119).
- **OpenAI + Anthropic** — vision input, structured output, and batch
  API support (#120).
- **Streaming** — `StreamingProvider` trait with native SSE streaming
  for OpenAI/Anthropic, plus a `--stream` flag on `kanros run` that
  pre-streams the first cell to stderr (#121).

### Changed

- **MSRV bumped to 1.88** (from 1.85) to clear RUSTSEC-2026-0009 on `time`.
- CI pinned at rustc 1.95.
- PR matrix trimmed to ubuntu-only to stay within the org's Actions
  minute cap; macOS/Windows tests available via `workflow_dispatch`.
- License allowlist expanded: `Apache-2.0`, `MIT`, `BSD-2-Clause`,
  `BSD-3-Clause`, `ISC`, `Unicode-DFS-2016`, `Unicode-3.0`, `CC0-1.0`,
  `Zlib`, `MPL-2.0`, `CDLA-Permissive-2.0` (webpki-roots), `Unlicense`
  (boa_engine).
- Workspace clippy: `pedantic` + `nursery` with targeted `allow`s
  (`collapsible_if`, `module_name_repetitions`, `must_use_candidate`,
  `missing_errors_doc`, `missing_panics_doc`).
- `kanros-provider-vertex` upgraded from key-auth-only to full
  service-account auth flow.

### Fixed

- `kanros-cli` duplicate `thiserror` dependency removed (#58).
- `kanros-cli` duplicate `mod` declarations in `main.rs` (#51).
- `kanros-report` rustfmt + clippy follow-ups around `checked_div` (#52, #53).
- `kanros-providers` merged-conflict marker cleanup in the `all`
  feature list and HuggingFace crate (#44 follow-up, post-#62 cleanup).

### Security

- `SECURITY.md` published with reporting address, threat model,
  sandboxing notes, cryptography choices, and a RUSTSEC tracking table (#75).
- `deny.toml` tightened: `openssl`, `openssl-sys`, `native-tls`
  explicitly denied; `wildcards = "deny"` for direct deps;
  `unknown-registry` and `unknown-git` denied; yanked crates denied (#75).
- All TLS routed through `rustls`; no `openssl` in the tree.
- Provider config structs with API keys / tokens / AWS credentials
  implement a manual `Debug` impl that prints `"***"`.
- Plugin sandbox: WASM plugins receive no FS / network / clock / env by
  default; explicit `allow_http: [...]` prefix list required for
  outbound HTTP.
- `boa_engine` JavaScript assertions run with a loop-iteration budget;
  pure-Rust engine — no native FFI.

### Performance

- Criterion benchmark suite at `crates/kanros-bench/benches/` covering
  the §11 performance bars (#74). See `bench_assertion_eval`,
  `bench_cell_pool`, `bench_cache_io`, `bench_config_parse`. Targets
  documented in `WORKSTREAMS.md` §K.

### Known Issues

- ~~**Critical:** `kanros-plugin` pinned to `wasmtime 27.0.0` with 15
  open RustSec advisories.~~ **Resolved:** `kanros-plugin` moved to
  wasmtime 44.0.2 in #107; the remaining wasmtime 27 pin in the
  `examples/wasm-sandbox` demo crate, plus the `deny.toml` MIT-0
  allow-list entry and stale skip cleanups, are fixed on the
  `release/v0.1.0-rc.1-prep` branch (2026-07-13). See `AUDIT.md`.
- WASM plugin host's `Custom` assertion wiring landed in #23 / #65 but
  the embedded JS grader (#67) is the recommended path for most users
  during the rc-soak — wasmtime startup cost dominates for short
  assertion bodies.
- macOS/Windows CI runs only via `workflow_dispatch` to conserve
  Actions minutes; nightly cron pending.
- `kanros.dev` (docs.kanros.dev, share.kanros.dev, registry.kanros.dev)
  was registered on 2026-07-13; no services are deployed on those hosts
  yet, so the docs site, hosted share endpoint, and plugin registry are
  build-it-yourself / self-hosted for now.
- `bedrock` and `vertex` credentials come from env vars or the system
  keyring fallback (#110); deeper `kanros auth` flows (e.g. OAuth
  device flow for Vertex) remain planned for v0.2.

### Acknowledgements

This release was produced by a fleet of worktree-isolated sub-agents
across 11 workstreams, dispatched and reviewed by an orchestrating
Claude instance. See `WORKSTREAMS.md` for the parallel-workstream plan
and `STABILITY.md` for the post-A4 trait-freeze policy. Contributor
attribution will be finalized at GA — see the GitHub Insights pane for
the per-PR list of authors.

## [0.1.0-alpha.1] — 2026-05-28

First pre-release with end-to-end usability. Trips the alpha.1 gate from
`WORKSTREAMS.md` §8.3 (3 providers + 10 deterministic assertions + a
basic runner + terminal output) and over-delivers on most of them.

### Providers (7)

- **echo** — built-in identity / upper / lower / reverse for testing without credentials.
- **openai** — Chat Completions API. API key from `OPENAI_API_KEY` or `config.api_key`.
- **anthropic** — Messages API, including tool use and vision for claude-3+.
- **ollama** — local Ollama server. Chat + embeddings (batched via parallel requests).
- **bedrock** — AWS Bedrock Converse API. SigV4-signed; works across all Bedrock vendors.
- **vertex** — Google Gemini API (key-auth public endpoint). Full Vertex service-account auth in alpha.2.
- **http** — generic OpenAI-compatible HTTP for vLLM/LM Studio/llama.cpp/proxies; `json_path` mapping for non-standard response shapes.

### Assertion families

- **Deterministic** (17): `equals`, `not-equals`, `contains`, `icontains`, `not-contains`, `starts-with`, `ends-with`, `regex`, `not-regex`, `contains-all`, `contains-any`, `contains-none`, `contains-json`, `is-json`, `length`, `word-count`, `cost`/`latency` (performance bucket).
- **Similarity**: `similarity` (cosine, embeds via `ProviderRef`), `levenshtein` (byte-level edit distance).
- **Model-graded**: `llm-rubric`, `factuality`, `model-graded-closedqa`, `answer-relevance` — driven by a grader provider, robust JSON-island parsing with one retry.
- **Custom (WASM)**: `custom` plugins via the new plugin host (see below).

### Runner

- Bounded global concurrency via `tokio::sync::Semaphore`.
- Per-provider rate limiting (sliding 1-second window).
- Retries with exponential backoff + jitter; respects `Retry-After`.
- Per-request timeout per `config.runner.timeout_seconds`.
- Ctrl+C cancellation — in-flight cells drain, partial `RunSummary` is preserved.
- New `run_with_cancellation(config, CancellationToken)` API alongside the existing `run(config)`.

### Output formatters

- `terminal` (default), `json` (pretty), `jsonl` (header + per-cell lines), `junit` (suites grouped by provider × prompt), `sarif` (SARIF 2.1.0; findings for failing/errored assertions only), `html` (single self-contained file; CSS inlined, no `<script>` tags).

### Plugin system (WASM, MVP)

- `wasmtime` Component Model host (`kanros-plugin`), WIT interface for assertion plugins.
- Capability-based sandbox: default-deny FS / network / clock / env. Plugins opt-in to outbound HTTP via `allow_http: [...]` prefix list.
- Sample plugin at `examples/plugins/contains-len/` (Rust → `wasm32-wasip2` via `cargo-component`).
- Runner wiring still pending: `Custom` assertions currently skip with `plugins disabled` — host is plumbed but not yet auto-instantiated in `kanros run`. Tracked as a follow-up sub-issue under workstream H.

### Release pipeline

- `cargo-dist` 0.28 wired across 6 targets: macOS arm64 / x86_64, Linux glibc / musl, Linux arm64 (cross), Windows x86_64.
- `release.yml` triggers on `v[0-9]+.*` tags or `workflow_dispatch`. Produces tarballs/zips, shell + PowerShell installers, SHA256 sums per artifact. Draft Release; team promotes to published.
- musl static linking via `+crt-static`.

### Infrastructure

- MSRV: **1.88** (bumped from 1.85 to clear RUSTSEC-2026-0009 on `time`).
- CI pinned at rustc **1.95**; PR matrix trimmed to ubuntu-only to stay within the org's Actions minute cap. macOS/Windows tests available via `workflow_dispatch`.
- License allowlist includes Apache-2.0, MIT, BSD-2/3-Clause, ISC, Unicode-DFS-2016/3.0, CC0-1.0, Zlib, MPL-2.0, CDLA-Permissive-2.0 (webpki-roots).
- Workspace clippy: `pedantic` + `nursery` (with targeted `allow`s for `collapsible_if`, `module_name_repetitions`, a few others).

### Known gaps for alpha.2

- SQLite cache (`kanros-cache` stub exists; sub-issue under workstream D).
- Streaming pipe to TUI (workstream F).
- Remaining assertion families: NLP metrics (BLEU/ROUGE/METEOR), code (`javascript`, `python`, `webhook`), composite (`assert-set`, `select-best`), RAG, safety.
- Web UI shell (workstream F).
- Red-team plugins + strategies (workstream E).

### Process notes

The 13 PRs that make up this release were produced by 13 worktree-isolated
sub-agents across three waves, dispatched and reviewed by an orchestrating
Claude instance. See `WORKSTREAMS.md` for the parallel-workstream plan and
`STABILITY.md` for the post-A4 trait-freeze policy.

[Unreleased]: https://github.com/kanros/kanros/compare/v0.1.0-alpha.1...HEAD
[0.1.0-rc.1]: https://github.com/kanros/kanros/compare/v0.1.0-alpha.1...HEAD
[0.1.0-alpha.1]: https://github.com/kanros/kanros/releases/tag/v0.1.0-alpha.1
