WASM plugins

kanros supports user-authored assertion plugins compiled to WebAssembly Component Model. Plugins run inside a wasmtime sandbox with no filesystem, no network, no clock, and no environment variables by default. They opt in to capabilities (today, just outbound HTTP) via the plugin's config: block.

The plugin host lives in kanros-plugin. The WIT interface plugins implement is assertion.wit, reproduced in full below.

When to write a WASM plugin

• You need to grade something that is not in the bundled assertion catalogue.
• The custom logic is non-trivial — many lines of code, or it has its own state, or it depends on a real library.
• You want strong isolation: the plugin runs in a sandboxed component and cannot accidentally exfiltrate data through the filesystem.

For one-off grader snippets, prefer JavaScript plugins (smaller cycle time, no build step).

The WIT interface

Every plugin exports the kanros:plugin/assertion world, which has one export interface and one optional import:

package kanros:plugin@0.1.0;

interface assertion {
    record assertion-input {
        output: string,
        rendered-prompt: string,
        vars-json: string,
        provider-id: string,
        config-json: string,
    }

    variant assertion-output {
        pass,
        fail(string),
        error(string),
        skipped(string),
    }

    evaluate: func(ctx: assertion-input) -> assertion-output;
}

interface http {
    get: func(url: string) -> result<list<u8>, string>;
    post: func(url: string, body: list<u8>) -> result<list<u8>, string>;
}

world assertion-plugin {
    import http;       // only available when `allow_http` is configured
    export assertion;
}

The four variants of assertion-output map 1:1 to AssertionOutcome::{Pass, Fail, Error, Skipped}. The fail variant carries a reason string that is surfaced in reports.

Authoring a plugin (Rust)

1. Add a new crate (anywhere — the plugin does not have to live in the kanros workspace).
2. Add wit-bindgen as a build dependency and pin the .wit to package kanros:plugin@0.1.0.
3. Implement the assertion interface.
4. Build for the wasm32-wasip2 target.

A minimal Rust plugin:

// Cargo.toml: crate-type = ["cdylib"]
wit_bindgen::generate!({
    path: "wit/assertion.wit",
    world: "assertion-plugin",
});

struct MyPlugin;

impl exports::kanros::plugin::assertion::Guest for MyPlugin {
    fn evaluate(ctx: exports::kanros::plugin::assertion::AssertionInput)
        -> exports::kanros::plugin::assertion::AssertionOutput
    {
        if ctx.output.contains("HELLO") {
            return exports::kanros::plugin::assertion::AssertionOutput::Pass;
        }
        exports::kanros::plugin::assertion::AssertionOutput::Fail(
            "expected the output to contain HELLO".into(),
        )
    }
}

export!(MyPlugin);

Build:

cargo build --target wasm32-wasip2 --release

The artifact is target/wasm32-wasip2/release/my_plugin.wasm.

Using a plugin

Reference the plugin in your kanros config via a wasm:// URI:

assert:
  - type: wasm
    plugin:
      source: "wasm://./plugins/my_plugin.wasm"
    config:
      threshold: 0.5

Then run with the plugin host enabled:

kanros run --plugins

--plugins is required because plugins execute arbitrary WASM code; we default to off to make the security posture explicit. Without it, every wasm/custom assertion reports error (and the run exits 1) — a plugin assertion is never counted as a pass when it did not run. The same applies when the .wasm file cannot be found.

Capability sandbox

By default, a plugin gets no filesystem, no network, no clock, and no environment variables. The host.allow_http: configuration key in the plugin's config: opens the http import above:

- type: wasm
  plugin:
    source: "wasm://./plugins/my_plugin.wasm"
  config:
    allow_http:
      - "https://api.example.com/v1/grade"

URLs are matched against the allowlist before each request; anything else returns an error string from http::get / http::post.

The custom vs wasm assertion

type: wasm is an alias of type: custom: both take the same plugin: and config: fields, call the same WIT interface, and are evaluated the same way. Use whichever reads better in your config.

Determinism

The runner does not call plugins concurrently with itself for the same cell. A plugin that depends on global state inside the wasm module behaves predictably. If you need cross-cell state, persist it via http (allowlisted) — the sandbox has no filesystem.

Failure modes

Notes

• The WIT package version is 0.1.0. We will bump it when the interface changes; plugins compiled against an older version can fail to load. Always document the interface version the plugin targets in its README.
• The Component Model is required (wasm32-wasip2). Pre-component WASM binaries from wasm32-unknown-unknown will not load.
• A plugin SDK crate is planned for 0.2 to make authoring less manual; until then, the path above (wit-bindgen + cargo build --target wasm32-wasip2) is the supported workflow.