AverthAverth / The details
Inside the ledger.
The ChatGPT tools, their data requirements, and the local CLI. Older simulated examples are retained below with their assumptions.
ChatGPT plugin reference
Four tools. Data you supply.
Averth analyzes submitted run records and replays thresholds against history. It does not automatically connect to your agents, billing accounts, or trace providers.
Analyze a batch of runs
analyze_run_batch accepts a JSON array of run records or an Averth export_ledger object with an attempts array. It reports fully loaded cost per accepted outcome, model/retry/tool/human costs, and expensive-tail share. Token and model analysis depends on the supplied fields.
| Field | Input |
|---|---|
run_id | Run identifier |
model_cost | Model or API cost |
tool_cost | External tool cost |
retry_cost | Retry-related cost |
human_cost | Human labor cost |
human_minutes | Time spent on human work |
retries | Retry count |
input_tokens | Input token count |
output_tokens | Output token count |
waste_tokens | Tokens recorded as waste |
total_cost | Recorded overall cost |
success | Recorded outcome flag |
reopened | Whether a result was reopened |
business_value | Supplied business-value information |
model_name | Model attribution |
Accepted fields do not guarantee dedicated reports: business_value does not establish a promised ROI calculation. Aggregate token counts cannot reconstruct per-step context growth.
A missing outcome is unknown, not a failure. With no accepted outcomes, the meter prints $0.00 per success. Treat that as undefined, not as a real cost.
Replay a historical policy
simulate_policy takes a supplied ledger and a policy with at least one supported threshold:
{
"max_cost_per_attempt": 1.00,
"yield_floor": 0.10
}max_cost_per_attempt identifies attempts above the dollar threshold. yield_floor identifies token yield below a fraction between 0 and 1. max_cost_per_run and min_yield are unsupported.
The replay identifies historical runs that would have stopped, their associated spend, failed-run exposure, and collateral: successful runs caught by the policy. That spend is not projected savings. Completed-run history cannot establish exactly what a live intervention would have saved.
No retry-count policy, routing policy, fallback configuration, or live enforcement tool is exposed.
Explain the cost layers
explain_cost_layers returns a static educational explanation of context compounding, external tool spend, token yield, heavy-tail outliers, and model attribution with human labor. Its 2,000-run workload figures are simulated, not customer production results.
The get-started skill supplies operating instructions and interpretation guidance. It does not add another executable backend capability.
Verify evidence
verify_evidence is the V0 evidence verifier, in plugin version 1.2.0, live in the ChatGPT listing. Inputs: evidence JSON with human-written checks, plus optional ingest text in any supported shape (JSON, JSONL, pretty JSON, plain log, OpenTelemetry-style span records, meter-style event records). Behaviour: judges in memory only, no execution, no repo access, no merge guard, no repair. Outputs: a JSON ledger plus a Markdown report; unknown stays unknown; inferred checks are always NOT OBSERVED, never human acceptance. Limits: tested only on a local fixture and public traces, not on a real developer workflow; no validated repair, no measured savings, no paying customer.
Example, illustration only, not a result: input is evidence JSON naming one check, exact command npm test, plus ingest text with that command and exit code 0. Output is a ledger with that check VERIFIED, scoped only to the cited run, plus a Markdown report. A check with no cited run is NOT OBSERVED, never assumed to have passed.
Access and data handling
The plugin can analyze records and replay ledgers you explicitly supply. It has no exposed access to provider invoices, billing accounts, previous ledgers, live agents, source code, or deployment logs. It cannot automatically fetch Langfuse, LangSmith, or OpenTelemetry traces, enforce budgets, change routing, or edit, delete, or export stored Averth records. No installed-version endpoint is exposed.
Other connected ChatGPT integrations remain separate. Selecting Averth does not itself authorize access to email, Drive, GitHub, or other accounts.
Submitted ledgers are processed in memory and not stored by Averth's tools. That does not establish ChatGPT conversation retention. Share metadata only after redacting customer identifiers, sensitive free text, credentials, payment details, and personal data. Exported case IDs, tool names, and retry reasons are not guaranteed to be sanitized.
Use the local CLI below when traces need to remain on your machine.
Cost per outcome
A model bill is one line
in the ledger.
An accepted outcome carries the cost of the attempts behind it. Count the retries, external calls, and review time alongside the model spend.
| Dashboard cost | $0.20 |
|---|---|
| Metered cost, including the dashboard | $0.30 |
| Human review, assumed loaded rate | $0.52 |
| Simulated total: metered + review | $0.82 |
In this simulation, metered cost is 1.5× the dashboard cost. Human review adds a separate assumed cost. Use your own loaded rate when measuring your workload.
Averth marks missing data as UNKNOWN. Cost per accepted outcome needs accepted-outcome labels; a completed trace alone does not establish success.
Illustrative simulation, not customer results. The previous headline, “$6,200/month your dashboard never shows,” described this simulated example; it is not a savings estimate for your team. These legacy website figures differ from the plugin v1.1 example.
The expensive tail
Find the runs that
carry the bill.
An average hides the spread. Rank attempts by cost to see which runs deserve a closer look.
What gets counted
Five places costs accumulate.
Model spend
Input and output tokens, attributed to the model and attempt.
Retries & failures
Spend on attempts that did not produce an accepted outcome.
Tools & API calls
The external calls recorded in the run, with their costs.
Context growth
The token cost of carrying more context through each step. Context grew 4.7× in the simulated example.
Human review
Escalation and handling time, priced at your loaded rate.
Where Averth fits
Keep your traces.
Add the cost of the outcome.
For the local CLI, start with an exported trace file. It reads Langfuse, LangSmith, and OpenTelemetry exports on your machine. The ChatGPT plugin does not automatically fetch them.
| Question | Averth | Your existing workflow |
|---|---|---|
| What did a call cost? | Includes model spend in the ledger. | Keep the call-level view in Langfuse or your token dashboard. |
| What did an outcome cost? | Rolls up attempts using accepted-outcome labels. | A spreadsheet needs the outcome joins and cost rollup. |
| Where did spend accumulate? | Separates retries, tools, and human review. | Bring trace data and your review-rate assumptions. |
| What is missing? | Flags unknown inputs rather than presenting gaps as free. | Check usage coverage before comparing totals. |
| What should I test first? | Ranks cost findings by dollars. | Test the change against cost and output quality. |
Averth meters and diagnoses costs. It does not change your agent or fix it automatically.
The method
From trace file to cost breakdown.
Read the traces
Run the CLI on an export you already have. The local meter processes the file on your machine.
Inspect the receipt
See cost per attempt, the expensive tail, and the largest cost drivers. Start with the highest-cost finding.
Check the evidence
Missing inputs stay
UNKNOWN. Historical policy replay reports spend on runs that would have stopped, plus collateral; it does not promise future savings.
Open source
Read the code. Run the meter.
MIT-licensed. Install from GitHub and start with a trace export. The local metering path needs no hosted account.
Have traces already?
Install, then point the CLI at your file.
pip install git+https://github.com/Wesley3141/Averth.git
averth trace runs.json --format langfuseInstrumenting an agent?
The Tracker records attempts, tool calls, and escalations.
from averth import Tracker
t = Tracker("support-agent")
t.start_attempt(case_id="case-1042")
# ... your agent runs ...
t.log_tool_call("zendesk.search", cost=0.004)
t.log_escalation(1.6, "low confidence")
t.end_attempt(success=True)
print(t.receipt())API note: this retained example uses receipt(); the current Tracker API uses pnl(). Follow the repository quickstart for current usage.
Merged to master, not a release
Evidence verifier: merged, experimental, not a release.
V0 classifies checks you write as VERIFIED, FAILED, NOT OBSERVED, EXTERNAL-UNVERIFIABLE, or NOT APPLICABLE, from run records and artifacts you supply.
It is experimental: merged to master, tested only on a local fixture and public traces, with no real workflow run, no validated repair, and 0 paying customers. Full detail and limits are in the Docs.