Batch API

For long-running evaluations against OpenAI you can ship the entire eval matrix as a single batch job. Batch jobs are cheaper (50 % off list price) and have a 24-hour SLA, so they're ideal for nightly regression suites that don't need real-time feedback.

In-process batch (kanros run --batch)

$ kanros run --batch --batch-poll-interval 30
Submitted batch: batch_abc123
status=in_progress progress=12/500
status=in_progress progress=412/500
status=completed progress=500/500

This:

  1. Serialises every (prompt × test × provider) cell into a single JSONL file (one chat-completion request per line).
  2. Uploads the file to POST /v1/files and creates a batch job via POST /v1/batches with completion_window: "24h".
  3. Polls GET /v1/batches/{id} every --batch-poll-interval seconds (default 30) until the job is completed, failed, expired, or cancelled.
  4. Downloads the per-line results, parses each output chunk into an EvalResult, and emits the run summary as if it had run synchronously.

Mixed-provider configs are rejected at submit time — the batch API ships one provider family per job. v1 supports openai only.

Fire-and-forget (kanros batch)

For workflows where the submitting machine doesn't stay online, the kanros batch subcommands separate submission from result fetching:

# 1. Submit and capture the id.
$ ID=$(kanros batch submit --config kanros.yaml)

# 2. Later (from any machine with the same config), check progress.
$ kanros batch status "$ID"
status=in_progress progress=412/500

# 3. When complete, download results.
$ kanros batch fetch "$ID" > results.jsonl

Each submit line in the input file is correlated with its output by a custom_id of the shape p{prompt_index}-t{test_index}-{provider_id}, so the cell origin survives the round trip.

Implementation notes

  • Multipart upload is hand-rolled — the kanros workspace does not opt into reqwest's multipart feature.
  • Polling uses tokio's sleep and is cancellation-safe.
  • The first failure to fetch the result file aborts the run; partial results are not currently merged into the cache.