2 Commits

Author SHA1 Message Date
7216d3cc31 feat: add benchmark aggregator and reporting (task 0030)
All checks were successful
CI / test (pull_request) Successful in 51s
CI / test (push) Successful in 50s
Add bench/aggregate.ts: the pure aggregator seam that rolls the
accumulated sample store into a readable comparison. aggregate produces
a headline table (one row per arm, cost-equivalent tokens as the
headline plus raw tokens, turns, duration, success rate, coverage, and
imputed cost as a de-emphasized secondary column), per-tier and
per-token-component breakdowns, and the separate bonus table; renderReport
renders it as stable text. Cost-equivalent tokens are weighted at render
time from the four retained components per ADR 0014, and incomplete
coverage is annotated rather than hidden. Unit-tested against synthetic
append-only sample stores.
2026-07-16 12:46:03 -04:00
d445b2bd2b docs: add benchmark harness spec, ADRs, and task breakdown
All checks were successful
CI / test (pull_request) Successful in 50s
CI / test (push) Successful in 49s
Add the benchmark-harness spec, three supporting ADRs (cost-equivalent
token metric, single-user seed, guard-based tool isolation), and the
0022-0030 task breakdown that slices the harness into foundational
seams, an integrating single-cell runner, and the reporting layer.
2026-07-15 10:04:28 -04:00