bench:run gains a --skill <path> flag, threaded into the existing
BuildArmOptions.skillPath, so a skill variant can be A/B'd against the
shipped SKILL.md with the same binary and harness — the mechanism used to
validate the find-then-act rewrite — without mutating the shipped file.
Add the maintainer-facing `bench:report` command, the offline counterpart
to `bench:run`: it opens the sample store, drains it, aggregates against the
scored suite and bonus definitions, and prints the aggregator's comparison to
stdout. It renders whatever has accumulated, annotating incomplete coverage
rather than blocking on a complete matrix.
`parseReportArgs` is the pure argument seam (`--store`, `--help`, plus a
`--self-review` / `--no-self-review` variant selector). Unlike `bench:run` the
boundary is offline — it reads only the local store, no host or Agent SDK — so
the whole `runReportCommand` is deterministic and unit-tested, not smoke-run.
Move `DEFAULT_STORE_ROOT` to `store.ts` as the single source of truth,
re-exported from `run.ts` for its existing importers.
Add the maintainer-facing command that runs one chosen benchmark cell on
demand, so only the token budget available at that moment is spent.
runCells (bench/run-loop.ts) runs one (arm, task) cell for a batch of
trials — defaulting to five with a reporting floor of three — by driving
the existing single-cell runner and the append-only sample store rather
than reimplementing orchestration. Re-running a cell deepens it: trial
numbering continues past the highest trial the cell already holds and the
new samples append, so a cell's sample size grows across sittings without
overwriting prior runs.
bench/run.ts is the command: parseRunArgs is the pure, unit-tested
argument seam, and runBenchCommand is the live boundary that resolves host
access, resolves the scored suite against the host's self-review support,
selects the task, and drives the run loop. It is invoked via the new
bench:run npm script, run under tsx (a new devDependency) because the
harness's .js-specifier imports need a TypeScript-aware runner. The Claude
Agent SDK is now declared as an optional peerDependency — documented but
neither installed for package consumers nor pulled into CI.
Every arm runs on the driver's single fixed model; the command exposes no
per-cell model override that could break cross-arm comparability. The
default store root bench/results/ is gitignored.
