kanros migrate

Translate an existing promptfoo configuration (promptfooconfig.yaml) into a kanros configuration in one command.

kanros migrate <INPUT> [--output <PATH> | --stdout] [--strict]

What it does

kanros migrate reads the input file, parses the legacy schema, and emits an equivalent kanros YAML. The translator:

• Maps providers: 1:1 and translates each provider's config: block (see Provider config below).

• Maps prompts: 1:1, converting any file:// / https:// URIs as-is.

• Carries tests: file references over: tests: file://cases.csv and - file://more.jsonl list entries are kept (bare paths such as tests: cases.csv gain a file:// prefix), since kanros loads tests from files natively. Globs and non-data sources (e.g. file://gen.py) are kept with a # TODO(migrate): comment.

• Walks every tests[i].assert[j] and translates each assertion to its kanros equivalent — renaming fields where only the spelling differs (regex value: → pattern:, contains-any value: → values:, …) and rewriting inversions kanros spells differently (see Assertions).

• Translates defaultTest → default_test, including its vars: and appended assert: list.

• Inserts # TODO(migrate): comments at the top of the output wherever a promptfoo construct has no direct kanros equivalent, including one line per assertion that could not be migrated.

• Never leaves a test with zero checks. If every assertion of a test (or of defaultTest) had to be dropped, a failing placeholder takes their place, so the test fails with a TODO(migrate) reason until you port the checks by hand:

  assert:
  - type: javascript
    code: 'return { pass: false, reason: "TODO(migrate): this test''s promptfoo assertion(s) `is-valid-openai-tools-call` could not be migrated automatically, ..." };'
    metric: migrate-todo
  

  A test that had no assertions in promptfoo either is listed in the summary: kanros run records such cells as errors (see kanros run).

The command prints a summary line on success:

✓ migrated → kanros.yaml  (3 providers, 5 prompts, 24 tests)

If TODOs were inserted, the line also notes the count:

✓ migrated → kanros.yaml  (3 providers, 5 prompts, 24 tests) (2 TODO comment(s) — review before running)

A migration summary follows on stderr (so --stdout output stays a clean config), listing every dropped assertion by location:

migration summary: 3 test(s), 1/3 assertion(s) migrated, 2 dropped, 1 failing placeholder(s) inserted where a test lost every assertion
  dropped: tests[0] ("only unsupported").assert[0] `is-valid-openai-tools-call`: no kanros equivalent
  dropped: tests[1] ("mixed").assert[1] `not-similar`: no kanros equivalent (kanros has no inverted form of `similar`)
warning: 1 test(s) have no assertions — `kanros run` reports them as errors: tests[2] ("no checks")

With --strict, a non-empty dropped: list makes the command exit 1.

Examples

kanros migrate legacy-config.yaml
kanros migrate legacy-config.yaml --output configs/regression.yaml
kanros migrate legacy-config.yaml --stdout | less
kanros migrate legacy-config.yaml --strict   # CI: fail if anything was dropped

Provider config

promptfoo keeps sampling settings and credentials in a camelCase config: block; kanros uses snake_case and lifts generation parameters to the provider entry itself. Given:

providers:
  - id: openai:chat:gpt-4o-mini
    config:
      apiKey: '{{ env.OPENAI_API_KEY }}'
      apiBaseUrl: https://api.openai.com/v1
      temperature: 0
      maxTokens: 256
      topP: 0.9
      stopSequences: ["END"]
      responseFormat: { type: json_object }
      frequency_penalty: 0.5

kanros migrate emits:

# TODO(migrate): provider `openai:gpt-4o-mini`: config key `frequency_penalty` is not supported by the kanros provider and was dropped.

providers:
- id: openai:gpt-4o-mini
  temperature: 0
  top_p: 0.9
  max_tokens: 256
  stop:
  - END
  response_format:
    type: json_object
  config:
    api_key: env:OPENAI_API_KEY
    base_url: https://api.openai.com/v1

The rules:

Provider ids

What survives translation

Assertions

Model-graded assertions (llm-rubric, factuality, context-*, …) require an explicit grader provider: in kanros. promptfoo falls back to an implicit default grader when the field is omitted — the migrator fills in the config's first provider instead and flags it with a # TODO(migrate): comment so you can pick a different grader deliberately. Bare provider strings (provider: openai:gpt-4o) are wrapped into the { id: … } form kanros uses.

What needs human review

The migrator writes a # TODO(migrate): comment at the top of the output for every construct that does not have a clean kanros equivalent (dropped provider config keys, implicit graders, …). Look for these markers and resolve them before piping the file into kanros run:

• Inline JavaScript graders that depend on Node.js built-ins (fs, path, http). kanros runs JS via embedded boa_engine, which exposes no Node APIs. Rewrite the grader as a pure function or port it to a WASM plugin (see WASM plugins).
• Custom dataset loaders. kanros loads tests: from CSV, JSONL, YAML/JSON and Hugging Face datasets. Glob patterns (file://tests/*.yaml) and script or spreadsheet sources (file://gen.py, Google Sheets) are flagged with a TODO: list each file explicitly, or export the cases to one of the supported formats.
• Cloud-only features — team mode, remote run history. kanros does not have a server side; for run comparisons see kanros diff, for sharing see kanros share.
• Cost / latency assertions with units other than USD / ms. kanros is strict on units.

Sanity workflow after migration

kanros migrate legacy-config.yaml
grep -n 'TODO' kanros.yaml           # resolve all TODOs
kanros validate --config kanros.yaml # check shape
kanros run --config kanros.yaml --no-cache  # smoke test

Then start iterating.

Notes

• The migrator is one-shot. It does not watch the source for changes; re-run it after any meaningful edit to the source config.
• The output is deterministic given a fixed input. Diff-friendly.
• If the input file cannot be read or the YAML parser rejects it, migrate exits with code 2 and prints the underlying error.