Vision input

Some kanros providers (OpenAI's gpt-4o, gpt-4-turbo, Anthropic's Claude 3+, and others) accept multimodal input — text and images in the same prompt. kanros surfaces vision through a simple template marker that any prompt can embed:

prompts:
  - id: describe-image
    template: |
      Describe what you see in this image:
      {{ image:./fixtures/cat.png }}

At runtime the marker is recognised by [kanros_core::preprocess_vision_template], the file is read from disk, base64-encoded, and attached to the CompletionRequest as a VisionContent block. The provider then translates that into the correct wire format:

Capability gating

Providers that don't support vision (echo, ollama with text-only models, Anthropic Claude 2.x, OpenAI gpt-3.5-turbo, ...) return ProviderError::VisionUnsupported when a vision payload is present. The runner surfaces this as a clean per-cell error rather than silently dropping the image.

Path resolution

Relative paths in {{ image:PATH }} are resolved against the directory containing the kanros config (so ./fixtures/cat.png always means "alongside the YAML file"). Absolute paths are passed through unchanged.

Supported MIME types

The MIME type is inferred from the file extension:

• .png — image/png
• .jpg, .jpeg — image/jpeg
• .gif — image/gif
• .webp — image/webp

Other extensions return VisionError::UnknownMime at render time.

Programmatic API

use kanros_core::{CompletionRequest, VisionContent, ImageData};

let req = CompletionRequest::user_with_vision(
    "what is in this image?",
    vec![VisionContent {
        url_or_data: ImageData::Url("https://example.com/cat.png".into()),
        mime: "image/png".into(),
    }],
);

ImageData::Url(...) is forwarded verbatim to the provider (which fetches the URL itself); ImageData::Base64(...) is inlined.