Providers

A provider in kanros is anything that turns a CompletionRequest into a CompletionResponse. The trait lives in kanros-core::provider::Provider and is implemented once per backend in a dedicated crate under crates/kanros-providers/.

Providers are referenced in config by id, in family:model form:

providers:
  - id: openai:gpt-4o-mini
  - id: anthropic:claude-3-5-haiku
  - id: ollama:llama3.1

The segment before the colon (openai, anthropic, ollama, …) is the provider family; it selects which crate handles the request. The segment after the colon is forwarded verbatim as the model id.

Bundled provider families

The workspace currently ships the following provider crates. Every name below corresponds to a directory in crates/kanros-providers/ and to a provider family id.

Inline provider config

Per-provider configuration is forwarded as a JSON value:

providers:
  - id: openai:gpt-4o-mini
    config:
      base_url: "https://api.openai.com/v1"
      timeout_ms: 30000
  - id: ollama:llama3.1
    config:
      base_url: "http://localhost:11434"

The config block is provider-specific and each provider crate documents its own keys in its README. Unknown keys are rejected — a typo fails the run rather than being silently ignored.

The key for pointing a family at a different host is base_url, not endpoint. For OpenAI-compatible third-party endpoints, give the full prefix that precedes the API path, including any version segment: the provider appends /chat/completions to whatever you supply. For example, Alibaba DashScope's OpenAI-compatible mode takes https://dashscope-intl.aliyuncs.com/compatible-mode/v1.

Generation parameters

Sampling and output-shape parameters can be written directly on the provider entry, or inside its config: block. When the same key appears in both places, the provider-level value wins.

providers:
  - id: openai:gpt-4o-mini
    temperature: 0
    top_p: 1
    max_tokens: 512
    seed: 42
    stop: ["\n\n"]
  - id: anthropic:claude-3-5-haiku-latest
    config:
      temperature: 0.2      # same keys also work inside config:
      max_tokens: 1024
prompts:
  - "Answer in one sentence: {{ question }}"
tests:
  - vars: { question: "What is the boiling point of water at sea level?" }
    assert:
      - type: icontains
        value: "100"

These values apply to every request made through the provider: eval cells, model-graded assertion graders, red-team probes and agent loops. Out-of-range or wrongly typed values fail kanros validate (exit code 2) instead of being silently ignored.

For anthropic, bedrock and vertex, max_tokens is also a native config key that sets the provider's default; the per-request value derived from the table above overrides that default.

fallback is also accepted on a provider entry: a list of provider ids tried in order when the provider fails with a retryable error after exhausting its retries.

providers:
  - id: openai:gpt-4o-mini
    fallback: [anthropic:claude-3-5-haiku-latest]

Credentials

For network providers, kanros looks for an API key in this order:

1. config.api_key in the provider's inline config block. Write it as env:NAME to read the key from the environment variable NAME instead of committing a secret — for example api_key: env:OPENAI_API_KEY.
2. The provider-specific environment variable (e.g. OPENAI_API_KEY, ANTHROPIC_API_KEY, GROQ_API_KEY).
3. The system keyring (see kanros auth), under the service name kanros and account name equal to the provider family.

If none of the three resolve, the provider cannot be constructed: the run stops before any cell executes, prints which variable or keyring entry it looked for, and exits with code 2. Errors a provider returns while a cell runs (HTTP failures, rate limits after the retry budget) are recorded on that cell's provider_error field instead and make the run exit with code 1.

The env:NAME form works for every credential key a provider accepts: api_key, api_token, access_key_id, secret_access_key and session_token. If NAME is not set, the run stops with a clear error naming the variable; kanros does not fall back to the next source.

providers:
  - id: bedrock:anthropic.claude-3-5-haiku-20241022-v1:0
    config:
      region: us-east-1
      access_key_id: env:AWS_ACCESS_KEY_ID
      secret_access_key: env:AWS_SECRET_ACCESS_KEY

Adding a provider

Each provider is a small crate that implements kanros_core::provider::Provider:

use async_trait::async_trait;
use kanros_core::{CompletionRequest, CompletionResponse, Provider, ProviderError};

#[derive(Debug)]
struct MyProvider { /* … */ }

#[async_trait]
impl Provider for MyProvider {
    fn id(&self) -> &str { "myprovider:demo" }

    async fn complete(&self, req: CompletionRequest) -> Result<CompletionResponse, ProviderError> {
        // …
        todo!()
    }
}

Register the provider with the kanros-providers registry (see crates/kanros-providers/src/lib.rs). Once registered, users can refer to it by id from any kanros config.

The echo provider

The echo:identity provider is special: it returns the rendered prompt verbatim. It has no dependencies, no I/O, and no per-call cost. It is the right pick for smoke tests, schema fixtures, and CI bootstraps.

providers:
  - id: echo:identity
prompts:
  - "say {{ word }}"
tests:
  - vars: { word: hello }
    assert:
      - type: contains
        value: hello