Providers

A provider in kanros is anything that turns a CompletionRequest into a CompletionResponse. The trait lives in kanros-core::provider::Provider and is implemented once per backend in a dedicated crate under crates/kanros-providers/.

Providers are referenced in config by id, in family:model form:

providers:
  - id: openai:gpt-4o-mini
  - id: anthropic:claude-3-5-haiku
  - id: ollama:llama3.1

The segment before the colon (openai, anthropic, ollama, …) is the provider family; it selects which crate handles the request. The segment after the colon is forwarded verbatim as the model id.

Bundled provider families

The workspace currently ships the following provider crates. Every name below corresponds to a directory in crates/kanros-providers/ and to a provider family id.

Inline provider config

Per-provider configuration is forwarded as a JSON value:

providers:
  - id: openai:gpt-4o-mini
    config:
      temperature: 0.0
      max_tokens: 512
      top_p: 0.9
  - id: ollama:llama3.1
    config:
      endpoint: "http://localhost:11434"

The config block is provider-specific. Each provider crate documents its own keys in its README; the OpenAI family, for instance, accepts the usual temperature, top_p, max_tokens, seed, plus an endpoint override for proxies.

Credentials

For network providers, kanros looks for an API key in this order:

1. config.api_key in the provider's inline config block (not recommended for source control).
2. The provider-specific environment variable (e.g. OPENAI_API_KEY, ANTHROPIC_API_KEY, GROQ_API_KEY).
3. The system keyring (see kanros auth), under the service name kanros and account name equal to the provider family.

If none of the three resolve, the provider returns an error and the cell exits with the provider_error field set. The whole run exits with code 3.

Adding a provider

Each provider is a small crate that implements kanros_core::provider::Provider:

use async_trait::async_trait;
use kanros_core::{CompletionRequest, CompletionResponse, Provider, ProviderError};

#[derive(Debug)]
struct MyProvider { /* … */ }

#[async_trait]
impl Provider for MyProvider {
    fn id(&self) -> &str { "myprovider:demo" }

    async fn complete(&self, req: CompletionRequest) -> Result<CompletionResponse, ProviderError> {
        // …
        todo!()
    }
}

Register the provider with the kanros-providers registry (see crates/kanros-providers/src/lib.rs). Once registered, users can refer to it by id from any kanros config.

The echo provider

The echo:identity provider is special: it returns the rendered prompt verbatim. It has no dependencies, no I/O, and no per-call cost. It is the right pick for smoke tests, schema fixtures, and CI bootstraps.

providers:
  - id: echo:identity
prompts:
  - "say {{ word }}"
tests:
  - vars: { word: hello }
    assert:
      - type: contains
        value: hello