Documentation.

Quick start

Install V0 from a master checkout. It is not published to PyPI.

cd /tmp
git clone https://github.com/Wesley3141/Averth.git Averth
cd Averth
python3 -m venv .venv-evidence
.venv-evidence/bin/python -m pip install ./evidence-v0
.venv-evidence/bin/averth-evidence --help

In a scratch directory, make a tiny repo, a checks file with 2 checks, and a runs file with 1 run.

mkdir -p /tmp/v0qs/repo
cat > /tmp/v0qs/repo/test_smoke.py <<'EOF'
def test_smoke():
    assert 1 + 1 == 2
EOF

Write checks.json and runs.json in /tmp/v0qs. The runs file names /tmp/v0qs/repo as its working directory.

{
  "task_intent": "smoke test",
  "checks": [
    {"id": "S1", "check_type": "recorded_run",
     "required_command": "python -m pytest -q test_smoke.py::test_smoke",
     "scope_limitation": "Proves only that this one test passed on this revision."},
    {"id": "S2", "check_type": "artifact_presence",
     "artifact_path": "test_smoke.py",
     "scope_limitation": "Proves only that the file exists in the snapshot."}
  ]
}

{
  "final_edit_marker": "2026-11-04T00:00:00Z",
  "runs": [
    {"label": "smoke run",
     "command": "python -m pytest -q test_smoke.py::test_smoke",
     "working_directory": "/tmp/v0qs/repo",
     "start_timestamp": "2026-11-04T01:00:00Z",
     "end_timestamp": "2026-11-04T01:01:00Z",
     "exit_code": 0, "stdout": "1 passed", "stderr": ""}
  ]
}

The repo holds one file, test_smoke.py, with a test that asserts 1 + 1 == 2.

cd /tmp/v0qs
/tmp/Averth/.venv-evidence/bin/averth-evidence --repo repo \
  --checks checks.json --runs runs.json --out out --report-id smoke

Expected output, verified 2026-11-05:

counts: {'VERIFIED': 2}
unknown rate: 0.0
runs: 1 (0 superseded, retained, never cited)

The ledger and report land in out/. Next, read Install and run and Write checks.

Back to top

Status

V0 is merged to master (commit a50f7a0, merge of PR 2, waves 1 to 3, averth-evidence 0.3.0, 53 tests). It is not published to PyPI, and no registry install command exists.

The meter is the live product path: the local CLI from the repository on master, and the ChatGPT plugin path. Plugin 1.2.0 is live in the ChatGPT listing and on the MCP server at mcp.getaverth.com.

V0 has 0 live benchmarks and 0 paying customers. It has no validated repair and no measured savings. See Limits for the full list.

Wave 1, wave 2, and wave 3 mean build waves, not releases or dates. Merged to master meant the pull request to master was merged. Published release means a package is published to a registry.

Install and run

For the meter, install from the repository on master. Do not use the meter install command as a V0 install command.

pip install git+https://github.com/Wesley3141/Averth.git
averth trace runs.json --format langfuse

They are different paths, and no V0 package is published yet. V0 packaging in the checkout names package averth-evidence 0.3.0 with console script averth-evidence; install it only from that checkout per evidence-v0/INSTALL.md, not from a registry.

git clone https://github.com/Wesley3141/Averth.git
cd Averth
cd evidence-v0
python3 cli.py --repo <repo path> --checks <checks.json> \
  --out <out dir>

Run the V0 tests from the Averth repository root, with a Python that has pytest. To check, run python -m pytest --version first.

If pytest is missing, install it into a virtual environment with python3 -m venv .venv, then .venv/bin/python -m pip install pytest. Then use .venv/bin/python for the test commands.

Wave 2 total of 32 is 9 original plus 10 wave 1 adapter plus 13 wave 2 tests.

Re-run the tests after any local change. Do not quote a test count from another checkout as the count for yours.

python3 -m venv .venv
.venv/bin/python -m pip install pytest
.venv/bin/python -m pytest -q evidence-v0/tests/

From evidence-v0/, the same suite is: ../.venv/bin/python -m pytest -q tests/ (with a Python that has pytest).

Packaged install, from the merged master checkout, not published: the master checkout defines package averth-evidence 0.3.0 and console script averth-evidence, per evidence-v0/INSTALL.md. It is not published to PyPI and not a release.

git clone https://github.com/Wesley3141/Averth.git
cd Averth
python3 -m venv .venv-evidence
.venv-evidence/bin/python -m pip install ./evidence-v0
.venv-evidence/bin/averth-evidence --help

When a package is published, this section will name the published version and the exact registry install command, with a changelog entry. Until then, any install command that claims to install V0 from a package registry is invented, and must not be used or quoted.

The V0 judge path is read-only by default and runs only when the operator triggers it. Judging alone never modifies the target repo, except that an operator-chosen output directory may live under it if the operator explicitly puts it there. No settings file, git hook, CI configuration, PR, or production service is ever touched by judging alone.

An operator-approved allowlist command is the exception: it runs exactly what the operator listed, including a command that deletes files; see Runner allowlist. An allowlist is an operator-written file listing exact commands that may run, and nothing else; the full definition is in Runner allowlist.

Meter

The meter reads traces you already have and produces a cost receipt. It does not fetch private traces on its own.

You supply sanitized records. The meter answers four questions: what did each usable result cost, how much of that cost was retries, how much did context grow, and how much of the work yielded a usable result.

What the meter measures

The meter answers four questions:

• Total fully loaded cost, meaning the total cost in the supplied records, including model, tool, and human time costs where supplied, divided by accepted outcomes. Accepted means the outcome the operator counts as usable, not merely a run that finished.
• Repeated attempts, tool calls, and human time behind failed runs, counted separately from first-attempt cost. A retry-heavy workflow is visible as retry cost, not hidden inside a total.
• How input grows as history accumulates across turns, including after a failed tool call, when later model calls resend the accumulated history.
• The share of attempts, tokens, or spend that produced a usable result, as defined by the supplied records. It is not a claim about quality in general.

Input formats

Supported trace formats for the meter are Langfuse, LangSmith, OpenTelemetry, JSONL, Claude Code session JSONL (one session file at a time), and Codex CLI rollout JSONL (one rollout file at a time). Each importer has its own limits.

Read the importer docstring in the repository before quoting a figure from that format. A format being supported does not mean every field in it is measured, and an accepted field does not promise its own report or a return on investment calculation.

Dollar labels

Every dollar in a meter report carries a provenance label. A figure quoted without its label is an incomplete quote.

Context tax, the extra token cost from resending accumulated history in later calls, on fallback-priced runs prints as not applicable, not as zero.

A missing outcome is unknown, not a failure. With no accepted outcomes, the meter prints $0.00 per success. Treat that as undefined, not as a real cost.

Historical policy replay reports historical stopped-run spend, failed-run exposure, and successful-run collateral. It is not projected savings and not live enforcement. Completed-run history does not establish the cost avoided by an intervention partway through execution.

Meter example, labelled

As a labelled example only, the public mini-SWE-agent trace set reproduced in wave 1 work was 12 attempts, 9 trace-labelled completions, 32 retries, 1.94 dollars ESTIMATED per trace-labelled success, 9.4x context compounding, and 21 percent yield. Completions there are trace labels, not accepted outcomes, and the dollar figure is ESTIMATED, not a bill.

Do not reuse those numbers as a result for any other workflow, model, or developer.

Evidence verifier

V0 takes three inputs: a human-written acceptance-check file (interview guidance: 3 to 5 checks; the tool does not enforce a count), a repo directory, and optionally a recorded-run ledger, a JSONL trace file, or both.

A ledger is the JSON list of recorded runs V0 judges. In this page, operator means the person running V0, and workflow owner means the person who owns the workflow being judged and agrees permission and scope.

It outputs one JSON ledger plus one Markdown report from the same judgments.

The report has a mandatory LIMITATIONS section. A check file is seeded by a human. V0 does not auto-extract checks from a transcript, issue, or PR, and an agent summary is never a check.

Evidence states

Check types

Trust tiers

Trust tiers, one per item:

Judging rules and execution

Judging rules that change a result

Command normalisation: only a Python interpreter token may be dropped when comparing commands. npm, cargo, make, go, and other first tokens are kept.

Before that fix, npm, cargo, and make all collapsed to the same normalised form and could cite the same run. After the fix, wave 3 tests prove the point on fixture runs: npm and cargo each cite their own run, a go check with no go run is NOT OBSERVED, and a mixed pytest, npm, and cargo run set is judged only from each check's own run. Those tests are not a result from a real multi-runner workflow.

A ledger that mixes runners must use the fixed matcher, or its VERIFIED statuses for colliding runners must be labelled status only. A status labelled status only means: quote it only as the recorded status, with its scope limitation, and never as proof that that runner's own command was observed.

Judging rules, one per item:

• After-final-edit: the final-edit marker comes from the --final-edit-marker flag first.

• Else the ledger marker from the runs file, else the newest file modification time in the repo snapshot.

• Runs started before the marker are superseded, retained, and never cited. A run with no parseable timestamp is not citable.

• Digest mismatch: if a check names an artifact with an expected digest and the snapshot digest differs, a would-be VERIFIED is downgraded to NOT OBSERVED with a digest_mismatch flag.

• For artifact_presence, a digest mismatch is FAILED, because the scan itself observed the mismatch. A non-zero cited run stays FAILED.

• A digest mismatch never converts a recorded_run row to FAILED. Only an artifact_presence digest mismatch is FAILED.

• Scope limit: a successful shell command is never proof of complete correctness, and every VERIFIED row carries the check's scope limitation verbatim.

Those three rules are applied together: final-edit first, then digest, then scope.

Execution, and what is never executed

Execution rules, one per item:

• Without the execute flag, no command runs at all. V0 judges imported runs only.

• With the execute flag, without an allowlist only pytest invocations ever run. Flags are only -q, -v, --tb=short, or --tb=line.

• Test paths or test nodes must be relative, inside the repo, and the test file must exist. A test node is a test path such as file.py::test_name. Runs use the operator-chosen interpreter, with no shell.

• The CLI execute flag allowlist extension is merged in wave 3. It lets the CLI run any exactly listed command, any runner, under the same no shell, exact match, and confirmation rules.

• No arbitrary command from a README, PR, issue, or transcript is ever executed on either path.

• Skipped or refused checks: a non-allowlisted recorded_run check that applies is skipped with a printed note, and its check stays NOT OBSERVED. Refusal never aborts the report.
• Misspelled node: a misspelled test node inside an existing test file runs and records FAILED with the pytest exit code; that FAILED is a failure of that test invocation, not a verdict that the product failed a test.
• Missing file: a missing test file is refused by the allowlist and the check stays NOT OBSERVED; that refusal is not a verdict on the product.
• Allowlist path: the separate runner allowlist path is described in the Runner allowlist section, and it is never automatic and never reads a command from a transcript.

Those rules together are why execution in V0 is narrow by design.

Judge commands

python3 cli.py --repo <repo path> --checks <checks.json> \
  --ingest runs.jsonl --out <out dir>

python3 cli.py --repo <repo path> --ingest runs.jsonl \
  --infer-checks --out <out dir>

python3 cli.py --repo <repo path> --checks <checks.json> \
  --execute --allowlist examples/npm-allowlist.json --out <out dir>

python3 cli.py --repo <repo path> --checks <checks.json> \
  --runs ledger.json --trace trace.jsonl --out <out dir>

The allowlist file must be operator-written and list the exact command. See Runner allowlist.

Write checks

V0 cannot auto-extract checks from intent, and it must not treat an agent summary as a check. If no checks are written, V0 has nothing to judge, and that is correct behaviour, not a gap to paper over. Write 3 to 5 checks, not 1 and not 20.

That count is interview guidance; the tool does not enforce it. One check is a slogan.

Twenty checks is a test suite relabelled. If you have more than 5, pick the 5 that would change the decision if they failed.

The interview, in order:

1. Write the intent in one sentence, in user terms, not the agent summary, before looking at any run output.
2. Say what done looks like, in observable terms: for each part of the intent, what would a person see, in a file, in a command result, or in a named procedure, if that part actually worked. Tests pass in general is not an answer. A named test command is.
3. Say what failure looks like, in observable terms: an empty run exiting 0, a skipped test, a file never written, a deploy that shipped without the feature.

4. Pick evidence per check: recorded_run (exact command, exit code, after final edit), artifact_presence (artifact path, expected digest if known), or external (named tester, procedure, result).

5. Write scope per check: what this check proves, and what it does not prove. A passing command proves only that the named command exited 0 on that revision. It never proves correctness in general.
6. Count check: 3 to 5 checks, as above (guidance, not a tool rule).

If no evidence of that kind exists, write the check anyway and let V0 return NOT OBSERVED. Do not invent a record to fill the gap.

A complete checks file

The file below is examples/no-test-checks.json, in the checkout, in full. Copy it and edit the commands and paths for your workflow.

{
  "checks": [
    {
      "check_type": "recorded_run",
      "expected_evidence": "a recorded run of that command, exit 0, after the final edit",
      "id": "C1",
      "required_command": "python -m reportgen --input data.csv --output report.md",
      "scope_limitation": "Proves only that the named command exited 0 on that revision, not that the totals are right.",
      "source": "check interview, operator written",
      "test_target": null,
      "text": "report generator runs on the known input"
    },
    {
      "artifact_path": "report.md",
      "check_type": "artifact_presence",
      "expected_evidence": "report.md present in the repo snapshot after the run",
      "id": "C2",
      "scope_limitation": "Proves only that a file with that name exists, not that its contents are correct.",
      "source": "check interview, operator written",
      "text": "report.md exists after the run"
    },
    {
      "artifact_path": "report.md",
      "check_type": "artifact_presence",
      "expected_evidence": "report.md digest matches the digest of a manually verified expected report for the known input",
      "id": "C3",
      "scope_limitation": "Proves only that the output matches that one known input, not all inputs.",
      "source": "check interview, operator written",
      "text": "report.md matches the manually verified expected report"
    },
    {
      "check_type": "external",
      "expected_evidence": "a named human confirmation record (tester, procedure, result) supplied by the operator before judging; none is included in this template, so this check correctly returns EXTERNAL-UNVERIFIABLE until one is supplied",
      "id": "C4",
      "scope_limitation": "Scoped to exactly that tester, procedure, and revision. It does not transfer to other inputs or revisions.",
      "source": "check interview, operator written",
      "text": "report totals verified by hand for the known input"
    },
    {
      "check_type": "recorded_run",
      "expected_evidence": "a recorded run of that command with a non-zero exit code, after the final edit",
      "id": "C5",
      "required_command": "python -m reportgen --input missing.csv --output report.md",
      "scope_limitation": "Proves only that the missing-input case fails loudly, not that all bad inputs fail loudly.",
      "source": "check interview, operator written",
      "test_target": null,
      "text": "missing input fails loudly instead of writing a silent empty report"
    }
  ],
  "task_intent": "a command line tool reads a CSV file and writes a Markdown summary report, one row per category and a total, with no tests in the repo"
}

Example: a workflow with no tests

Intent: a small command line tool reads a CSV file and writes a Markdown summary report, with one row per category and a total. There are no tests in the repo, and no test suite to cite. The operator must not claim tests pass, because no tests exist.

1. Exact command python -m reportgen --input data.csv --output report.md, exit 0, after the final edit. Scope: proves only that the named command exited 0 on that revision, not that the totals are right.
2. Artifact path report.md exists in the repo snapshot after the run. Scope: proves only that a file with that name exists, not that its contents are correct.

3. The same report.md, with an expected digest matching a manually verified expected report for a small known input.

   The shipped example file carries no expected_digest field, so the tool does not enforce that digest match. Scope: proves only that a file with that name exists.

4. Report totals verified by hand for the known input, with no human confirmation record supplied, so it returns EXTERNAL-UNVERIFIABLE.
5. Exact command python -m reportgen --input missing.csv --output report.md, expected to fail loudly. With no recorded run, it returns NOT OBSERVED.

A template run with no recorded runs and no supplied human confirmation gives NOT OBSERVED 2 (Checks 1 and 5) and FAILED 2 (Checks 2 and 3, against a repo snapshot with no report.md).

Against a snapshot containing report.md, Checks 2 and 3 would not be FAILED. It also gives EXTERNAL-UNVERIFIABLE 1 (Check 4), unknown rate 60.0 percent, and 0 VERIFIED.

Unknown rate means NOT OBSERVED plus EXTERNAL-UNVERIFIABLE, divided by all checks. That is the template result, not a pass, and the template must not be quoted as a pass.

Example

The fixture is a small widget with add, subtract, and multiply, plus edge tests, a hardware claim, a build claim, and a migration claim. The multiply function is intentionally buggy in the fixture: it returns 7 for inputs that should multiply to 12, so a multiply test that checks the correct product must fail.

That bug is part of the fixture design, not a V0 error. The check file has 8 checks, written by a human before judging, each with a scope limitation stating what it proves and what it does not prove.

{
  "task_intent": "widget add, subtract and multiply behave as their names state; the full test suite is green; edge cases are covered by a run test",
  "checks": [
    {"id": "C1", "check_type": "recorded_run",
     "required_command": "python -m pytest -q widget_tests/test_widget.py::test_add",
     "scope_limitation": "Proves only that add(2, 3) == 5 in this one test. It does not prove add is correct in general."},
    {"id": "C2", "check_type": "recorded_run",
     "required_command": "python -m pytest -q widget_tests/test_widget.py::test_subtract"},
    {"id": "C3", "check_type": "recorded_run",
     "required_command": "python -m pytest -q widget_tests/test_widget.py::test_multiply"}
  ]
}

Imported mode uses recorded runs only, with no execution. It judged 5 runs, with 1 superseded and retained, never cited.

The two VERIFIED rows are add and subtract. The two FAILED rows are multiply and the full suite, because multiply fails.

The two NOT OBSERVED rows are edge tests, which were never run in any recorded run, and the build, which was never run at all.

The EXTERNAL-UNVERIFIABLE row is the hardware claim, with no named tester record. The NOT APPLICABLE row is the migration claim, with a recorded reason that no database exists in the fixture.

Unknown rate means checks that are NOT OBSERVED or EXTERNAL-UNVERIFIABLE, divided by all checks.

Executed mode uses the same checks, with the execute flag and a Python that has pytest.

The edge tests move from NOT OBSERVED to VERIFIED, because V0 actually ran them in this mode and they passed. Running the check is what moves the row, not relabelling it.

Multiply and the full suite stay FAILED. Hardware stays EXTERNAL-UNVERIFIABLE. Build stays NOT OBSERVED, because no build was run.

Read both results together and the lesson is clear: the same checks, the same code, and a different evidence set give a different unknown rate and a different VERIFIED count. Neither result proves the widget is correct in general, proves edge coverage in general, proves the hardware claim, or proves a build artifact exists.

A report that quotes only the VERIFIED count, without the mode, the unknown rate, and the scope limitations, is an incomplete report.

Plugin tool

The plugin tool verify_evidence is the V0 evidence verifier. It is in plugin version 1.2.0, live in the ChatGPT listing.

Inputs are evidence JSON with human-written checks, plus optional ingest text in any supported shape. It judges in memory only. It does not execute anything, access a repo, guard a merge, or repair code.

Outputs are a JSON ledger plus a Markdown report. Unknown stays unknown. Inferred checks are always NOT OBSERVED, never human acceptance.

Limits: tested only on a local fixture and public traces, not on a real developer workflow. No validated repair, no measured savings, no paying customer. No staging deployment record is verified.

Example, illustration only, not a result. Input is evidence JSON naming one check, exact command npm test, plus ingest text with that command and exit code 0. Output is a ledger with that check VERIFIED, scoped only to the cited run, plus a Markdown report.

Generic ingestion

Wave 3 is merged to master. This section describes its mechanism as merged, not as a released package. A raw log is accepted directly as run records by generic ingestion; the recorded run converter remains available for converting a raw harness, CI, or log record into ledger shape.

Accepted shapes, wave 3: no pre-conversion once merged

Since wave 3 merged, the sender does not have to pre-convert to ledger shape. Accepted inputs, all as local files or text the operator names, are:

• a JSON object with a runs list, or a bare JSON list of records;
• JSONL, one JSON object per line, as run records (JSONL as a trace file under the trace flag is summary only, see Trace summary vs verdict);
• pretty-printed JSON objects, one after another;
• plain text logs, in key and value blocks separated by blank lines, or shell style lines with a command and an exit code, for example: $ npm test exit 0;
• OpenTelemetry-style span records, with spanId, traceId, attributes, status, and start and end times in ISO, epoch seconds, epoch milliseconds, or nanoseconds;
• meter-style event records, with a type field and token or tool fields, for example: {"type": "tool_event", "tool": "search"} with no command, which stays an event, or the same record with a command field added, which becomes a run.

A span is a timed operation record from a tracing system, not itself a record that a command ran. That is the span, run, and event distinction, stated once: a span is a timed operation; a run is a recorded command with an exit code or a recorded missing exit code; an event is a record with no command, kept as an event, never as a run.

A span becomes a run only when its attributes carry an explicit command.

A span name alone is never promoted to a command, because a span name is not evidence a command ran. Promoting it could create a false VERIFIED from a status code alone.

A meter-style event is kept as an event, not a run, unless it also carries a command. A command-bearing event with no exit code becomes a run recorded with a missing exit code, never assumed to have passed.

For span records, status code maps to an exit code only as OK to 0, ERROR to 1, and UNSET to missing. UNSET never becomes a pass or a fail.

Example, rule illustration only, not a result. A span record whose attributes carry "command": "npm test" and status OK becomes a run with exit code 0.

A span with only "name": "npm test" and no command in attributes becomes no run. An event {"type": "tool_event", "tool": "search", "command": "npm test"} with no exit code becomes a run recorded with a missing exit code, never assumed to have passed.

Preservation rules

Original record IDs are kept in the raw record, and the run label falls back in this order: label, id, record_id, run_id, spanId, traceId, case_id, then the command itself. That fallback exists only so a run still has a readable label when no label field is present; it does not create evidence.

Command strings are kept verbatim, and normalisation for matching drops only a Python interpreter token, as in the wave 1 matcher fix.

Timestamps are kept verbatim in the raw record and parsed where possible, including ISO 8601 with Z or offset, naive ISO treated as UTC, YYYY-MM-DD HH:MM:SS, YYYY/MM/DD HH:MM:SS, MM/DD/YYYY HH:MM:SS, and epoch seconds, milliseconds, microseconds, and nanoseconds, as numbers or numeric strings. Any other form is unparseable: it is kept as text and the run is not citable, never given an invented timestamp.

Unknown fields are kept in the raw record. They are never silently dropped, and never turned into a verdict.

Runners, judged only from their own run

Runner detection returns the first token of the command for known runners. Known test and build runners include pytest, python, python3, npm, yarn, pnpm, node, cargo, go, make, gradle, gradlew, maven, mvn, dotnet, phpunit, php, rspec, ruby, jest, vitest, bun, deno, ctest, bazel, buck2, swift, xcodebuild, docker, podman, tox, and nox. Shell commands use sh, bash, or zsh as first tokens, and a .sh script path maps to shell.

For any other first token, the runner is generic. Generic is not a failure: the run is still recorded, still matched by exact command, and still judged only from its own exit code. When there is no command, the runner is unknown, which is a different state from generic: generic means an unlisted command was recorded, unknown means no command was recorded at all.

A check is never judged from another runner's run, because verifier matching is exact command matching, not runner matching. That is the fix that stops the old collision in which npm, cargo, and make could cite the same run.

Inferred checks are not human checks

Each distinct normalised observed command can generate one INFERRED check row, labelled INFERRED, with its source stated as inferred from observed runs, not human written. Inferred rows appear only when the operator passes the wave 3 --infer-checks option; they are not generated automatically. The verifier always returns NOT OBSERVED for an inferred row, with flag inferred_not_human.

An inferred row never becomes VERIFIED, even if its command exited 0. That is deliberate: observing that a command ran is not the same as a human accepting what it proved.

Because an inferred row judges NOT OBSERVED, it counts toward the unknown rate like any NOT OBSERVED row: it adds to both the unknown numerator and the total check denominator, so inferred rows inflate the unknown rate by design, and they are for observe-only listing, not for a verdict.

Human-written checks are still required for a verdict. If no human checks are written, V0 has observed runs and artifacts to report, but nothing to judge as VERIFIED or FAILED on a human criterion.

Failure tolerance, the whole point

Malformed lines, unknown fields, unknown runners, missing exit codes, mixed-language runs, and runs with no tests must all produce a ledger, not a crash, with the unknown rate computed and reported. Unknown rate means checks that are NOT OBSERVED or EXTERNAL-UNVERIFIABLE, divided by all checks. A run with a missing exit code is recorded with that gap, not assumed to have passed.

A workflow with no tests does not auto-extract human checks and does not pass; observed commands generate INFERRED rows only, always NOT OBSERVED, see the no tests example in Check writing. Ingestion never executes anything: a command string in the input is data only, copied into the ledger, never passed to a shell.

Execution happens only through the allowlist path in Judging rules and execution.

What generic ingestion does not change

Generic ingestion changes what V0 can read and record. It does not change what a VERIFIED row means, it does not create a Tier 3 attestation, it does not turn a trace summary into a verdict, it does not prove correctness, acceptance, value, time saved, or willingness to pay, and it does not supply a live benchmark, a paying customer, a validated repair, or measured savings.

An expected digest is checked only against the local repo snapshot; a remote URL is never accepted as a digest or as artifact evidence. Those remain at zero, as in Status and Limits.

Recorded run converter

The recorded run converter, adapter 1, converts a raw harness, CI, or log record into the V0 recorded-run ledger shape that the V0 command line accepts with its runs flag. An adapter is a small piece that converts one external record shape into a form V0 can judge.

Adapter numbers come from an earlier list of seven candidate adapters; this page documents only adapters 1, 3, and 7, and does not document the other four here.

It never executes anything. A command string in the input is data only: it is copied into the output ledger, never passed to a shell, never run, never imported.

If a transcript contains a destructive command, the converter records the text and does nothing with it. That property is covered by a wave 1 test in the checkout, on fixture input only, not a live verification.

Input format A: structured JSON

A JSON object with a runs list, or a bare JSON list of run objects. Each run object may carry label, command (required), working_directory, start_timestamp, end_timestamp (ISO 8601, or null), exit_code (integer, required), stdout, stderr, and output_digest. If output_digest is absent, it is computed as SHA-256 of stdout plus stderr.

Runs with no start timestamp are kept, with a warning on stderr. Runs with a present but unparseable timestamp are also kept and get no converter warning; V0 itself treats any run with no parseable start timestamp as not citable.

{
  "runs": [
    {"label": "npm run", "command": "npm test",
     "start_timestamp": "2026-11-04T20:00:00Z",
     "end_timestamp": "2026-11-04T20:05:00Z",
     "exit_code": 0, "stdout": "ok", "stderr": "",
     "working_directory": "/tmp/example-app"},
    {"label": "cargo run", "command": "cargo test",
     "start_timestamp": "2026-11-04T20:00:00Z",
     "end_timestamp": "2026-11-04T20:06:00Z",
     "exit_code": 1, "stdout": "", "stderr": "1 test failed",
     "working_directory": "/tmp/example-app"}
  ]
}

Input format B: delimited log text

Records are separated by blank lines. Each record is a set of key and value lines, one per line, with these keys: label, command, working_directory, start_timestamp, end_timestamp, exit_code, stdout, stderr. The stdout and stderr values are literal text on their lines, not commands to run.

A record without a command or without an exit_code is rejected with an error naming the record number. No other parsing is attempted: the converter does not guess a command from free prose, and it does not split a shell pipeline into runs.

label: npm run
command: npm test
working_directory: /tmp/example-app
start_timestamp: 2026-11-04T20:00:00Z
end_timestamp: 2026-11-04T20:05:00Z
exit_code: 0
stdout: ok
stderr:

label: cargo run
command: cargo test
working_directory: /tmp/example-app
start_timestamp: 2026-11-04T20:00:00Z
end_timestamp: 2026-11-04T20:06:00Z
exit_code: 1
stdout:
stderr: 1 test failed

Run the converter as follows, from evidence-v0/:

python3 recorded_run_converter.py \
  --input examples/recorded-run-converter-input.json --out ledger.json
python3 recorded_run_converter.py \
  --input examples/recorded-run-converter-input.log \
  --format log \
  --out ledger.json --final-edit-marker 2026-11-04T12:00:00Z

Next step: judge the ledger with the CLI, as in Quick start, passing ledger.json with --runs.

Output is a JSON object with final_edit_marker (from the flag, or null) and a runs list in exactly the shape the V0 loader accepts: label, command, working_directory, start_timestamp, end_timestamp, exit_code, stdout, stderr, output_digest. Limits, stated plainly: a converted run is Tier 1 evidence (see Trust tiers), the same as any imported run.

Conversion does not make a log trustworthy, does not prove the command ran where claimed, and does not prove correctness.

A run with no timestamp, or with a timestamp before the final edit marker, will not be cited by V0. If the source log itself is incomplete, the V0 output is a high unknown rate, not a pass.

Runner allowlist

Wave 3 status for this section: the standalone allowlist runner described here can already run one exactly listed command, any runner. The CLI execute flag allowlist extension is merged in wave 3. The runner allowlist, adapter 3, is also available as a standalone runner; the command line execute flag likewise accepts an operator-written allowlist, and without one it stays pytest-only.

Nothing here runs automatically, and nothing here reads a command from a transcript, a README, a PR, or an issue. The operator writes an allowlist file for one workflow, names the approver (the date is recorded by convention in the example but is not enforced by the loader), lists the exact commands that may run, and only then runs the allowlist runner with an explicit confirmation flag.

A command that is not listed, exactly, is refused. There is no wildcard, no pattern, and no 'any npm command' rule.

{
  "workflow": "example-app npm tests",
  "approved_by": "operator name, fill in before use",
  "approved_date": "2026-11-04",
  "runner": "npm",
  "allowed_flags": ["test"],
  "allowed_relative_paths": [],
  "timeout_seconds": 300,
  "sandbox": "run only in the repo checkout, operator watches output, no network assumed",
  "commands": [{"label": "npm tests", "command": "npm test"}]
}

Validation rules, all required for a command to pass:

• The command, split with shlex, must equal one listed command, token for token. No extra flags, no extra paths.
• The first token must equal the allowlist runner.
• timeout_seconds, if present, must be a positive integer.

Execution, in the allowlist runner only: subprocess without a shell, current working directory set to the repo directory, output captured, timeout applied, and the result written as one ledger-shaped run record, with source allowlist_recorder and trust tier 2. Tier 2 means this tool observed the run locally. It is not tamper-proof provenance, and it is not a Tier 3 CI attestation.

Quoting limit, read before approving a list: listed commands are split with shlex (the shell-word splitter), so a quoted argument loses its quotes. Prefer a script file over a command with quoted arguments, or the command V0 runs will differ from the command you wrote.

python3 allowlist_runner.py --repo <dir> \
  --allowlist examples/npm-allowlist.json --command "npm test" \
  --out run.json --i-approved-this-allowlist

Without --i-approved-this-allowlist, the runner prints what it would run and runs nothing.

Next step: merge the resulting record into a ledger object with a runs list, then judge it with the CLI, as in Quick start, passing the ledger with --runs.

Safety limit, read before approving any list: an allowlist does not make a command safe in general. It makes a named, operator-approved command runnable in a named repo checkout.

If the listed command itself deletes files, the allowlist will run it, because the operator approved that exact command. Review the listed commands before approving them, and never approve a list copied from an untrusted source.

PR converter

The PR evidence converter, adapter 7, is fixture-based in wave 1. It makes no network call, reads no GitHub API, and fetches no live PR.

It converts a fixture JSON file into two outputs, and it keeps the two kinds of material strictly separate: ledger runs, and Tier 0 claims. A claim in a comment is not evidence, however specific it sounds.

{
  "pr_number": 999,
  "repo": "example/repo",
  "files_changed": ["src/main.rs"],
  "ci_runs": [{"label": "CI cargo test", "command": "cargo test",
    "start_timestamp": "2026-11-04T20:00:00Z",
    "end_timestamp": "2026-11-04T20:05:00Z",
    "exit_code": 0, "stdout": "ok", "stderr": ""}],
  "review_comments": [{"author": "A. Reviewer",
    "body": "I ran the full suite and it passed, and I checked the device manually.",
    "claims": ["full suite passed", "device checked manually"]}]
}

Ledger runs are runs recorded in a CI log, in recorded-run shape: label, command, working_directory, start and end timestamps, exit code, stdout, stderr, output_digest (computed if absent). These become Tier 1 ledger runs, exactly like any imported run. A CI run with no parseable start timestamp is kept, and V0 will not cite it.

Tier 0 claims are anything claimed in a PR body, a diff summary, or a review comment that is not backed by a CI run record. These are listed, with author and text, and are never turned into ledger runs. A claim stays Tier 0 unless the whole claim string exactly equals the command string of a CI run in the fixture (which by validation carries an integer exit code).

Timestamp parseability is not checked at this step; a CI run with no parseable timestamp is kept in the ledger but V0 will not cite it. A claim such as 'I ran cargo test and it passed' does not equal 'cargo test' and stays Tier 0.

Run the PR converter as follows, from evidence-v0/:

python3 pr_evidence_converter.py --input examples/pr-fixture.json \
  --ledger-out ledger.json --claims-out claims.json

Next step: judge the ledger with the CLI, as in Quick start, passing ledger.json with --runs.

Live version, in merged master but not live-verified: it would fetch the full diff, the CI check runs, and the review thread through the GitHub API with the operator permission, convert each CI check run to the same ledger shape, and list every other claim as Tier 0. A live-fetch implementation exists in merged master (pr_live_fetch.py), with mocked tests only, is not live-verified, and is not in a released package.

Until a live version is tested against a real PR, every report from this converter must say fixture-based, not live-verified, and must not be cited as a merge certifier. V0 is intended for PR triage only in this fixture form, never a merge decision on its own.

Trace summary vs verdict

The V0 trace flag takes a JSONL trace file, one JSON object per line, each with type and case_id where present, and summarises it only. It counts lines, cases, model events, tool events, retry events, and end events, and totals input and output tokens on model events.

It never judges a check from a trace, never turns a trace label into an accepted outcome, and never computes a cost per accepted outcome from a trace with no acceptance join. Verdicts in a run that also supplies a trace come from the fixture checks and the recorded runs, not from the trace.

Lines that do not parse as JSON are counted as unparseable inside events_by_type, not dropped and not reported as zero. For the trace-summary flag, OpenTelemetry pretty-printed JSON, LangSmith and Langfuse exports, and custom JSON shapes need a converter before V0 can even summarise them.

A trace normaliser for those shapes is merged wave 2 work (trace_normaliser.py): it converts a JSON array or pretty-printed objects to JSONL summary shape, inventing neither type nor case_id.

Wave 3 generic ingestion, merged, extends that idea to recorded runs: it accepts those shapes directly as run records, with the same no invention rule, and counts unparseable records instead of dropping them. A summary is never a verdict, and a verdict is never a trace count.

python3 trace_normaliser.py --input trace.json --out trace.jsonl
python3 cli.py --repo <repo path> --checks <checks.json> \
  --runs ledger.json --trace trace.jsonl --out <out dir>

Trace example, labelled: the public mini-SWE-agent trace summarised in wave 1 work was 938 lines, 12 cases, 447 model events, 447 tool events, 32 retry events, 12 end events, 5,182,465 input tokens, and 69,883 output tokens. Those are event counts from that trace file. They are not a V0 verdict, not a cost figure, and not a result for any other trace.

What to supply

Six items for a judged report:

If evidence is missing inside a runnable invocation, V0's output is a report with a high unknown rate, stating exactly what evidence is missing, for which check. That report is still useful as triage. It is not a pass, and it must not be quoted as one.

A missing repo directory or a missing check file makes the command error out instead of producing a report. Item 5 is an operator process requirement, not a CLI gate; the CLI enforces the repo, checks, and output arguments, not permission records.

Privacy and local only

V0 has no upload path. It makes no network call, no model call, and takes no credentials. For the plugin path only, as described on the site: submitted ledgers are processed in memory and never stored by Averth tools, but ChatGPT conversation retention is separate, and that retention is not Averth storage.

That retention statement is a plugin-path statement, not a V0 tool behaviour. For both paths, supply sanitized metadata only. Remove customer identifiers, sensitive free text, credentials, and personal data before supplying any record.

Case IDs, tool names, and retry reasons may contain sensitive information. Do not claim an export is automatically sanitized.

The V0 repo snapshot maps every file under the repo to a SHA-256 digest, excluding version control, caches, and virtual environments, and excluding the ledger and report output files themselves. It records digests, not file contents, in the ledger.

It does not exclude secrets that are inside a file that is snapshotted, so secrets must be excluded by the operator before supplying a repo snapshot, and by the operator scope in the What to supply section.

If a secret is in the supplied records, V0 will digest and report on it as supplied. That is an operator responsibility, not a V0 sanitisation claim.

Local-only framing for prospect and customer material: nothing derived from a prospect's shared materials goes in the public repository, not even generalised documentation with synthetic fixtures based on those materials, unless the owner explicitly authorises that use. The examples in this documentation use the public fixture and public traces only, not any prospect workflow.

Limits

V0 classifies evidence for checks a human wrote. It does not repair code, guard a merge, monitor a fleet, enforce a budget, or prove correctness, acceptance, value, time saved, or willingness to pay.

Nothing here is a customer result. No PyPI publication, no live-verified PR fetch, no real-developer-workflow run, no validated repair, no measured improvement or savings, 0 live benchmarks, 0 paying customers.

As stated in Status, V0 has only run on a local fixture and public traces, not on a real developer workflow, and the limits stated there still apply. It does not prove test quality, correctness beyond the named assertions, acceptance by any human or operator, value, time saved, willingness to pay, or that a firmware image boots on untested hardware.

It does not repair anything, guard anything, monitor anything continuously, or certify that unobserved tests never occurred.

Executed mode in the command line tool is pytest only without an allowlist; with an operator-approved allowlist, any exactly listed command may run (see Evidence verifier and Runner allowlist). That allowlist extension is merged in wave 3. Trace mode is summary only, never a verdict (see Trace summary vs verdict).

Generic ingestion does not change that: it widens what can be recorded, and unknown input stays unknown (see Generic ingestion). The PR converter is fixture-based, not live-verified, and must not be cited as a merge certifier (see PR evidence converter).

No Tier 3 row is ever produced, because no independent CI or isolated runner with signed attestation is wired in.

Tier 2 is stronger observation, not tamper-proof provenance. Tier 4 is scoped to exactly the named tester, procedure, hardware, and revision, and does not transfer. Inferred rows are always NOT OBSERVED, never human acceptance.

Still a limit on any prod claim: no PyPI publication (see Install and run), no live-verified PR fetch (see PR evidence converter), no real-developer-workflow run, no validated repair, and no measured improvement or savings. Demo package formal second-pass sign-off is recorded in the wave 2 report, and the investor brief v2 qualification pass was done in wave 2; neither is a customer install, pilot, or validation on a real workflow.

Waves 1, 2, and 3 are merged to master.

A live PR fetch implementation (see PR evidence converter) is mocked only, not live-verified. The live repair benchmark stays blocked pending a newly authorised model evaluation environment, never local credentials. This documentation is staging preparation, not a release note, and the install path in it must not be presented as live.

If a claim in this documentation does not match the tool in the checkout, the tool wins, and the documentation must be corrected. Report a mismatch by naming the section, the sentence, and the tool file and behaviour that contradicts it.